Install

pip install claudette

Getting started

Anthropic’s Python SDK will automatically be installed with Claudette, if you don’t already have it.

import os
# os.environ['ANTHROPIC_LOG'] = 'debug'

To print every HTTP request and response in full, uncomment the above line.

from claudette import *

Claudette only exports the symbols that are needed to use the library, so you can use import * to import them. Alternatively, just use:

import claudette

…and then add the prefix claudette. to any usages of the module.

Claudette provides models, which is a list of models currently available from the SDK.

models

['claude-3-opus-20240229',
 'claude-3-5-sonnet-20241022',
 'claude-3-haiku-20240307']

For these examples, we’ll use Sonnet 3.5, since it’s awesome!

model = models[1]

Chat

The main interface to Claudette is the Chat class, which provides a stateful interface to Claude:

chat = Chat(model, sp="""You are a helpful and concise assistant.""")
chat("I'm Jeremy")

Hello Jeremy, nice to meet you.

• id: msg_015oK9jEcra3TEKHUGYULjWB
• content: [{'text': 'Hello Jeremy, nice to meet you.', 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 19, 'output_tokens': 11, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

r = chat("What's my name?")
r

Your name is Jeremy.

• id: msg_01Si8sTFJe8d8vq7enanbAwj
• content: [{'text': 'Your name is Jeremy.', 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 38, 'output_tokens': 8, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

r = chat("What's my name?")
r

Your name is Jeremy.

• id: msg_01BHWRoAX8eBsoLn2bzpBkvx
• content: [{'text': 'Your name is Jeremy.', 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 54, 'output_tokens': 8, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

As you see above, displaying the results of a call in a notebook shows just the message contents, with the other details hidden behind a collapsible section. Alternatively you can print the details:

print(r)

Message(id='msg_01BHWRoAX8eBsoLn2bzpBkvx', content=[TextBlock(text='Your name is Jeremy.', type='text')], model='claude-3-5-sonnet-20241022', role='assistant', stop_reason='end_turn', stop_sequence=None, type='message', usage=In: 54; Out: 8; Cache create: 0; Cache read: 0; Total: 62)

Claude supports adding an extra assistant message at the end, which contains the prefill – i.e. the text we want Claude to assume the response starts with. Let’s try it out:

chat("Concisely, what is the meaning of life?",
     prefill='According to Douglas Adams,')

According to Douglas Adams,42. Philosophically, it’s to find personal meaning through relationships, purpose, and experiences.

• id: msg_01R9RvMdFwea9iRX5uYSSHG7
• content: [{'text': "According to Douglas Adams,42. Philosophically, it's to find personal meaning through relationships, purpose, and experiences.", 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 82, 'output_tokens': 23, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

You can add stream=True to stream the results as soon as they arrive (although you will only see the gradual generation if you execute the notebook yourself, of course!)

for o in chat("Concisely, what book was that in?", prefill='It was in', stream=True):
    print(o, end='')

It was in "The Hitchhiker's Guide to the Galaxy" by Douglas Adams.

chat = AsyncChat(model)
await chat("I'm Jeremy")

async for o in await chat("Concisely, what is the meaning of life?",
                          prefill='According to Douglas Adams,', stream=True):
    print(o, end='')

According to Douglas Adams,  it's 42. But in my view, there's no single universal meaning - each person must find their own purpose through relationships, personal growth, contribution to others, and pursuit of what they find meaningful.

Prompt caching

If you use mk_msg(msg, cache=True), then the message is cached using Claude’s prompt caching feature. For instance, here we use caching when asking about Claudette’s readme file:

chat = Chat(model, sp="""You are a helpful and concise assistant.""")

nbtxt = Path('README.txt').read_text()
msg = f'''<README>
{nbtxt}
</README>
In brief, what is the purpose of this project based on the readme?'''
r = chat(mk_msg(msg, cache=True))
r

Claudette is a high-level wrapper for Anthropic’s Python SDK that automates common tasks and provides additional functionality. Its main features include:

1. A Chat class for stateful dialogs
2. Support for prefill (controlling Claude’s initial response words)
3. Convenient image handling
4. Simple tool use API integration
5. Support for multiple model providers (Anthropic, AWS Bedrock, Google Vertex)

The project is notable for being the first “literate nbdev” project, meaning its source code is written as a detailed, readable Jupyter Notebook that includes explanations, examples, and teaching material alongside the functional code.

The goal is to simplify working with Claude’s API while maintaining full control, reducing boilerplate code and manual work that would otherwise be needed with the base SDK.

• id: msg_014rVQnYoZXZuyWUCMELG1QW
• content: [{'text': 'Claudette is a high-level wrapper for Anthropic\'s Python SDK that automates common tasks and provides additional functionality. Its main features include:\n\n1. A Chat class for stateful dialogs\n2. Support for prefill (controlling Claude\'s initial response words)\n3. Convenient image handling\n4. Simple tool use API integration\n5. Support for multiple model providers (Anthropic, AWS Bedrock, Google Vertex)\n\nThe project is notable for being the first "literate nbdev" project, meaning its source code is written as a detailed, readable Jupyter Notebook that includes explanations, examples, and teaching material alongside the functional code.\n\nThe goal is to simplify working with Claude\'s API while maintaining full control, reducing boilerplate code and manual work that would otherwise be needed with the base SDK.', 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 4, 'output_tokens': 179, 'cache_creation_input_tokens': 7205, 'cache_read_input_tokens': 0}

The response records the a cache has been created using these input tokens:

print(r.usage)

Usage(input_tokens=4, output_tokens=179, cache_creation_input_tokens=7205, cache_read_input_tokens=0)

We can now ask a followup question in this chat:

r = chat('How does it make tool use more ergonomic?')
r

According to the README, Claudette makes tool use more ergonomic in several ways:

1. It uses docments to make Python function definitions more user-friendly - each parameter and return value should have a type and description

2. It handles the tool calling process automatically - when Claude returns a tool_use message, Claudette manages calling the tool with the provided parameters behind the scenes

3. It provides a toolloop method that can handle multiple tool calls in a single step to solve more complex problems

4. It allows you to pass a list of tools to the Chat constructor and optionally force Claude to always use a specific tool via tool_choice

Here’s a simple example from the README:

def sums(
    a:int,  # First thing to sum 
    b:int=1 # Second thing to sum
) -> int: # The sum of the inputs
    "Adds a + b."
    print(f"Finding the sum of {a} and {b}")
    return a + b

chat = Chat(model, sp=sp, tools=[sums], tool_choice='sums')

This makes it much simpler compared to manually handling all the tool use logic that would be required with the base SDK.

• id: msg_01EdUvvFBnpPxMtdLRCaSZAU
• content: [{'text': 'According to the README, Claudette makes tool use more ergonomic in several ways:\n\n1. It uses docments to make Python function definitions more user-friendly - each parameter and return value should have a type and description\n\n2. It handles the tool calling process automatically - when Claude returns a tool_use message, Claudette manages calling the tool with the provided parameters behind the scenes\n\n3. It provides atoolloopmethod that can handle multiple tool calls in a single step to solve more complex problems\n\n4. It allows you to pass a list of tools to the Chat constructor and optionally force Claude to always use a specific tool viatool_choice\n\nHere\'s a simple example from the README:\n\n```python\ndef sums(\n a:int, # First thing to sum \n b:int=1 # Second thing to sum\n) -> int: # The sum of the inputs\n "Adds a + b."\n print(f"Finding the sum of {a} and {b}")\n return a + b\n\nchat = Chat(model, sp=sp, tools=[sums], tool_choice=\'sums\')\n```\n\nThis makes it much simpler compared to manually handling all the tool use logic that would be required with the base SDK.', 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 197, 'output_tokens': 280, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 7205}

We can see that this only used ~200 regular input tokens – the 7000+ context tokens have been read from cache.

print(r.usage)

Usage(input_tokens=197, output_tokens=280, cache_creation_input_tokens=0, cache_read_input_tokens=7205)

chat.use

In: 201; Out: 459; Cache create: 7205; Cache read: 7205; Total: 15070

Tool use

Tool use lets Claude use external tools.

We use docments to make defining Python functions as ergonomic as possible. Each parameter (and the return value) should have a type, and a docments comment with the description of what it is. As an example we’ll write a simple function that adds numbers together, and will tell us when it’s being called:

def sums(
    a:int,  # First thing to sum
    b:int=1 # Second thing to sum
) -> int: # The sum of the inputs
    "Adds a + b."
    print(f"Finding the sum of {a} and {b}")
    return a + b

Sometimes Claude will say something like “according to the sums tool the answer is” – generally we’d rather it just tells the user the answer, so we can use a system prompt to help with this:

sp = "Never mention what tools you use."

We’ll get Claude to add up some long numbers:

a,b = 604542,6458932
pr = f"What is {a}+{b}?"
pr

'What is 604542+6458932?'

To use tools, pass a list of them to Chat:

chat = Chat(model, sp=sp, tools=[sums])

To force Claude to always answer using a tool, set tool_choice to that function name. When Claude needs to use a tool, it doesn’t return the answer, but instead returns a tool_use message, which means we have to call the named tool with the provided parameters.

r = chat(pr, tool_choice='sums')
r

Finding the sum of 604542 and 6458932

ToolUseBlock(id=‘toolu_014ip2xWyEq8RnAccVT4SySt’, input={‘a’: 604542, ‘b’: 6458932}, name=‘sums’, type=‘tool_use’)

• id: msg_014xrPyotyiBmFSctkp1LZHk
• content: [{'id': 'toolu_014ip2xWyEq8RnAccVT4SySt', 'input': {'a': 604542, 'b': 6458932}, 'name': 'sums', 'type': 'tool_use'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: tool_use
• stop_sequence: None
• type: message
• usage: {'input_tokens': 442, 'output_tokens': 53, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

Claudette handles all that for us – we just call it again, and it all happens automatically:

chat()

The sum of 604542 and 6458932 is 7063474.

• id: msg_01151puJxG8Fa6k6QSmzwKQA
• content: [{'text': 'The sum of 604542 and 6458932 is 7063474.', 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 524, 'output_tokens': 23, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

You can see how many tokens have been used at any time by checking the use property. Note that (as of May 2024) tool use in Claude uses a lot of tokens, since it automatically adds a large system prompt.

chat.use

In: 966; Out: 76; Cache create: 0; Cache read: 0; Total: 1042

We can do everything needed to use tools in a single step, by using Chat.toolloop. This can even call multiple tools as needed solve a problem. For example, let’s define a tool to handle multiplication:

def mults(
    a:int,  # First thing to multiply
    b:int=1 # Second thing to multiply
) -> int: # The product of the inputs
    "Multiplies a * b."
    print(f"Finding the product of {a} and {b}")
    return a * b

Now with a single call we can calculate (a+b)*2 – by passing show_trace we can see each response from Claude in the process:

chat = Chat(model, sp=sp, tools=[sums,mults])
pr = f'Calculate ({a}+{b})*2'
pr

'Calculate (604542+6458932)*2'

chat.toolloop(pr, trace_func=print)

Finding the sum of 604542 and 6458932
[{'role': 'user', 'content': [{'type': 'text', 'text': 'Calculate (604542+6458932)*2'}]}, {'role': 'assistant', 'content': [TextBlock(text="I'll help you break this down into steps:\n\nFirst, let's add those numbers:", type='text'), ToolUseBlock(id='toolu_01St5UKxYUU4DKC96p2PjgcD', input={'a': 604542, 'b': 6458932}, name='sums', type='tool_use')]}, {'role': 'user', 'content': [{'type': 'tool_result', 'tool_use_id': 'toolu_01St5UKxYUU4DKC96p2PjgcD', 'content': '7063474'}]}]
Finding the product of 7063474 and 2
[{'role': 'assistant', 'content': [TextBlock(text="Now, let's multiply this result by 2:", type='text'), ToolUseBlock(id='toolu_01FpmRG4ZskKEWN1gFZzx49s', input={'a': 7063474, 'b': 2}, name='mults', type='tool_use')]}, {'role': 'user', 'content': [{'type': 'tool_result', 'tool_use_id': 'toolu_01FpmRG4ZskKEWN1gFZzx49s', 'content': '14126948'}]}]
[{'role': 'assistant', 'content': [TextBlock(text='The final result is 14,126,948.', type='text')]}]

The final result is 14,126,948.

• id: msg_0162teyBcJHriUzZXMPz4r5d
• content: [{'text': 'The final result is 14,126,948.', 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 741, 'output_tokens': 15, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

Structured data

If you just want the immediate result from a single tool, use Client.structured.

cli = Client(model)

def sums(
    a:int,  # First thing to sum
    b:int=1 # Second thing to sum
) -> int: # The sum of the inputs
    "Adds a + b."
    print(f"Finding the sum of {a} and {b}")
    return a + b

cli.structured("What is 604542+6458932", sums)

Finding the sum of 604542 and 6458932

[7063474]

This is particularly useful for getting back structured information, e.g:

class President:
    "Information about a president of the United States"
    def __init__(self, 
                first:str, # first name
                last:str, # last name
                spouse:str, # name of spouse
                years_in_office:str, # format: "{start_year}-{end_year}"
                birthplace:str, # name of city
                birth_year:int # year of birth, `0` if unknown
        ):
        assert re.match(r'\d{4}-\d{4}', years_in_office), "Invalid format: `years_in_office`"
        store_attr()

    __repr__ = basic_repr('first, last, spouse, years_in_office, birthplace, birth_year')

cli.structured("Provide key information about the 3rd President of the United States", President)

[President(first='Thomas', last='Jefferson', spouse='Martha Wayles', years_in_office='1801-1809', birthplace='Shadwell', birth_year=1743)]

Images

Claude can handle image data as well. As everyone knows, when testing image APIs you have to use a cute puppy.

fn = Path('samples/puppy.jpg')
display.Image(filename=fn, width=200)

We create a Chat object as before:

chat = Chat(model)

Claudette expects images as a list of bytes, so we read in the file:

img = fn.read_bytes()

Prompts to Claudette can be lists, containing text, images, or both, eg:

chat([img, "In brief, what color flowers are in this image?"])

In this adorable puppy photo, there are purple/lavender colored flowers (appears to be asters or similar daisy-like flowers) in the background.

• id: msg_01LHjGv1WwFvDsWUbyLmTEKT
• content: [{'text': 'In this adorable puppy photo, there are purple/lavender colored flowers (appears to be asters or similar daisy-like flowers) in the background.', 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 110, 'output_tokens': 37, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

The image is included as input tokens.

chat.use

In: 110; Out: 37; Cache create: 0; Cache read: 0; Total: 147

Alternatively, Claudette supports creating a multi-stage chat with separate image and text prompts. For instance, you can pass just the image as the initial prompt (in which case Claude will make some general comments about what it sees), and then follow up with questions in additional prompts:

chat = Chat(model)
chat(img)

What an adorable Cavalier King Charles Spaniel puppy! The photo captures the classic brown and white coloring of the breed, with those soulful dark eyes that are so characteristic. The puppy is lying in the grass, and there are lovely purple asters blooming in the background, creating a beautiful natural setting. The combination of the puppy’s sweet expression and the delicate flowers makes for a charming composition. Cavalier King Charles Spaniels are known for their gentle, affectionate nature, and this little one certainly seems to embody those traits with its endearing look.

• id: msg_01Ciyymq44uwp2iYwRZdKWNN
• content: [{'text': "What an adorable Cavalier King Charles Spaniel puppy! The photo captures the classic brown and white coloring of the breed, with those soulful dark eyes that are so characteristic. The puppy is lying in the grass, and there are lovely purple asters blooming in the background, creating a beautiful natural setting. The combination of the puppy's sweet expression and the delicate flowers makes for a charming composition. Cavalier King Charles Spaniels are known for their gentle, affectionate nature, and this little one certainly seems to embody those traits with its endearing look.", 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 98, 'output_tokens': 130, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

chat('What direction is the puppy facing?')

The puppy is facing towards the left side of the image. Its head is positioned so we can see its right side profile, though it appears to be looking slightly towards the camera, giving us a good view of its distinctive brown and white facial markings and one of its dark eyes. The puppy is lying down with its white chest/front visible against the green grass.

• id: msg_01AeR9eWjbxa788YF97iErtN
• content: [{'text': 'The puppy is facing towards the left side of the image. Its head is positioned so we can see its right side profile, though it appears to be looking slightly towards the camera, giving us a good view of its distinctive brown and white facial markings and one of its dark eyes. The puppy is lying down with its white chest/front visible against the green grass.', 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 239, 'output_tokens': 79, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

chat('What color is it?')

The puppy has a classic Cavalier King Charles Spaniel coat with a rich chestnut brown (sometimes called Blenheim) coloring on its ears and patches on its face, combined with a bright white base color. The white is particularly prominent on its face (creating a distinctive blaze down the center) and chest area. This brown and white combination is one of the most recognizable color patterns for the breed.

• id: msg_01R91AqXG7pLc8hK24F5mc7x
• content: [{'text': 'The puppy has a classic Cavalier King Charles Spaniel coat with a rich chestnut brown (sometimes called Blenheim) coloring on its ears and patches on its face, combined with a bright white base color. The white is particularly prominent on its face (creating a distinctive blaze down the center) and chest area. This brown and white combination is one of the most recognizable color patterns for the breed.', 'type': 'text'}]
• model: claude-3-5-sonnet-20241022
• role: assistant
• stop_reason: end_turn
• stop_sequence: None
• type: message
• usage: {'input_tokens': 326, 'output_tokens': 92, 'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}

Note that the image is passed in again for every input in the dialog, so that number of input tokens increases quickly with this kind of chat. (For large images, using prompt caching might be a good idea.)

chat.use

In: 663; Out: 301; Cache create: 0; Cache read: 0; Total: 964

Other model providers

You can also use 3rd party providers of Anthropic models, as shown here.

models_aws

['anthropic.claude-3-opus-20240229-v1:0',
 'anthropic.claude-3-5-sonnet-20241022-v2:0',
 'anthropic.claude-3-sonnet-20240229-v1:0',
 'anthropic.claude-3-haiku-20240307-v1:0']

from anthropic import AnthropicBedrock

ab = AnthropicBedrock(
    aws_access_key=os.environ['AWS_ACCESS_KEY'],
    aws_secret_key=os.environ['AWS_SECRET_KEY'],
)
client = Client(models_aws[-1], ab)

chat = Chat(cli=client)
chat("I'm Jeremy")

models_goog

['claude-3-opus@20240229',
 'claude-3-5-sonnet-v2@20241022',
 'claude-3-sonnet@20240229',
 'claude-3-haiku@20240307']

from anthropic import AnthropicVertex
import google.auth

project_id = google.auth.default()[1]
gv = AnthropicVertex(project_id=project_id, region="us-east5")
client = Client(models_goog[-1], gv)

chat = Chat(cli=client)
chat("I'm Jeremy")

Extensions

• Pydantic Structured Ouput

• __Report an issue

models

['claude-3-opus-20240229',
 'claude-3-5-sonnet-20241022',
 'claude-3-haiku-20240307',
 'claude-3-5-haiku-20241022']

cli = Client(models[1])

cli.structured("Classify the sentiment of the following movie review: <Review>The movie is bad </Review>", get_sentiment)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[1], line 1
----> 1 cli.structured("Classify the sentiment of the following movie review: <Review>The movie is bad </Review>", get_sentiment)

File /usr/local/lib/python3.11/site-packages/claudette/core.py:270, in structured(self, msgs, tools, obj, ns, **kwargs)
    268 "Return the value of all tool calls (generally used for structured outputs)"
    269 tools = listify(tools)
--> 270 res = self(msgs, tools=tools, tool_choice=tools, **kwargs)
    271 if ns is None: ns=mk_ns(*tools)
    272 if obj is not None: ns = mk_ns(obj)

File /usr/local/lib/python3.11/site-packages/claudette/core.py:251, in __call__(self, msgs, sp, temp, maxtok, prefill, stream, stop, tools, tool_choice, **kwargs)
    237 @patch
    238 @delegates(messages.Messages.create)
    239 def __call__(self:Client,
   (...)
    248              tool_choice:Optional[dict]=None, # Optionally force use of some tool
    249              **kwargs):
    250     "Make a call to Claude."
--> 251     if tools: kwargs['tools'] = [get_schema(o) for o in listify(tools)]
    252     if tool_choice: kwargs['tool_choice'] = mk_tool_choice(tool_choice)
    253     msgs = self._precall(msgs, prefill, stop, kwargs)

File /usr/local/lib/python3.11/site-packages/claudette/core.py:251, in <listcomp>(.0)
    237 @patch
    238 @delegates(messages.Messages.create)
    239 def __call__(self:Client,
   (...)
    248              tool_choice:Optional[dict]=None, # Optionally force use of some tool
    249              **kwargs):
    250     "Make a call to Claude."
--> 251     if tools: kwargs['tools'] = [get_schema(o) for o in listify(tools)]
    252     if tool_choice: kwargs['tool_choice'] = mk_tool_choice(tool_choice)
    253     msgs = self._precall(msgs, prefill, stop, kwargs)

File /usr/local/lib/python3.11/site-packages/toolslm/funccall.py:91, in get_schema(f, pname)
     89 def get_schema(f:callable, pname='input_schema')->dict:
     90     "Generate JSON schema for a class, function, or method"
---> 91     schema = _get_nested_schema(f)
     92     desc = f.__doc__
     93     assert desc, "Docstring missing!"

File /usr/local/lib/python3.11/site-packages/toolslm/funccall.py:81, in _get_nested_schema(obj)
     79 for n, o in d.items():
     80     if n != 'return' and n != 'self':
---> 81         _process_property(n, o, props, req, defs)
     83 schema = dict(type='object', properties=props, title=obj.__name__ if isinstance(obj, type) else None)
     84 if req: schema['required'] = list(req)

File /usr/local/lib/python3.11/site-packages/toolslm/funccall.py:69, in _process_property(name, obj, props, req, defs)
     66 if obj.default is empty: req[name] = True
     68 if hasattr(obj.anno, '__origin__'):
---> 69     p.update(_handle_container(obj.anno.__origin__, obj.anno.__args__, defs))
     70 else:
     71     p.update(_handle_type(obj.anno, defs))

TypeError: 'NoneType' object is not iterable

TypeError: 'NoneType' object is not iterable
Traceback (most recent call last):

  File "/usr/local/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-1-9b203029b6ec>", line 1, in <module>
    cli.structured("Classify the sentiment of the following movie review: <Review>The movie is bad </Review>", get_sentiment)

  File "/usr/local/lib/python3.11/site-packages/claudette/core.py", line 270, in structured
    res = self(msgs, tools=tools, tool_choice=tools, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/claudette/core.py", line 251, in __call__
    if tools: kwargs['tools'] = [get_schema(o) for o in listify(tools)]
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/claudette/core.py", line 251, in <listcomp>
    if tools: kwargs['tools'] = [get_schema(o) for o in listify(tools)]
                                 ^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/toolslm/funccall.py", line 91, in get_schema
    schema = _get_nested_schema(f)
             ^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/toolslm/funccall.py", line 81, in _get_nested_schema
    _process_property(n, o, props, req, defs)

  File "/usr/local/lib/python3.11/site-packages/toolslm/funccall.py", line 69, in _process_property
    p.update(_handle_container(obj.anno.__origin__, obj.anno.__args__, defs))

TypeError: 'NoneType' object is not iterable

AI Prompt

what’s the problem here?

AI Response

The error is occurring because we need to modify our get_sentiment function to work properly with Claude’s structured output format. The current implementation using Literal type isn’t being handled correctly. Let’s fix it:

def get_sentiment(
    text: str,  # The text to analyze
) -> dict:
    "Analyzes the sentiment of a movie review"
    return {
        "sentiment": "neg" if "bad" in text.lower() else "pos"
    }

The main changes are: 1. We’re accepting the text as input rather than returning a predefined sentiment 2. We’re returning a dictionary with the sentiment value 3. We’ve removed the Literal type hint which was causing issues with the schema generation

Alternatively, if you want to use the structured output approach with Claude, we could use a class instead:

class SentimentResult:
    "Result of sentiment analysis"
    def __init__(self,
                 sentiment: str,  # The sentiment (pos/neg) of the text
                ):
        self.sentiment = sentiment

Would you like to try either of these approaches?

class SentimentResult:
    "Result of sentiment analysis"
    def __init__(self,
                 sentiment: str,  # The sentiment (pos/neg) of the text
                ):
        assert sentiment in ["pos", "neg"], "Invalid sentiment"
        
        self.sentiment = sentiment

cli.structured("Classify the sentiment of the following movie review: <Review>The movie is bad </Review>", SentimentResult)

[<__main__.SentimentResult at 0x72ea002e1e90>]

r = cli.structured("Classify the sentiment of the following movie review: <Review>The movie is bad </Review>", SentimentResult)

r

[<__main__.SentimentResult at 0x72eab07b53d0>]

r.sentiment

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[1], line 1
----> 1 r.sentiment

AttributeError: 'list' object has no attribute 'sentiment'

AttributeError: 'list' object has no attribute 'sentiment'
Traceback (most recent call last):

  File "/usr/local/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-1-9d4c274308b4>", line 1, in <module>
    r.sentiment

AttributeError: 'list' object has no attribute 'sentiment'

r[0].sentiment

'neg'

def classify_review(review):
    r = cli.structured(f"Classify the sentiment of the following movie review: <Review>{review['text']}</Review>", SentimentResult)
    return r[0].sentiment

classify_review(reviews_embds[-1])

'neg'

def add_pred(review):
    return {**review, "pred": classify_review(review)}

llm_results = L(reviews_embds).map(add_pred)

llm_results

(#100) [{'text': 'plot : derek zoolander is a male model . \nhe is also very dumb and impressionable . \nfor that reason , he is secretly hired and trained ( so secret , that even he doesn\'t know about it ) by an underground fashion syndicate to assassinate the prime minister of malaysia , who wants to abolish child labor in his country . \nwill zoolander fulfill the dirty deed ? \nwill zoolander ever grace the world his new " look " ? \nis this a funny movie ? \nfind out below . . . \ncritique : there\'s a place in our world for " stupid comedies " . \nfilms which don\'t pretend to take themselves seriously , are based on idiotic premises and filled with dumb jokes . \nthere\'s also a time for them and many would argue that this difficult period of our history , might be an ideal circumstance during which to " relax " by watching something so completely frivolous . \nwell , if you\'re in the mood for some seriously mindless entertainment , ben stiller and his cast of many have assembled one of the more original dumb comedies in some time . \nof course , comedies as such are usually very subjective and i could see how some will see this film simply as stupid and unfunny , while others will grant them the " stupid " , but consider it funny instead . \ni personally enjoyed it for the most part , cracked up loudly during a couple of specific sequences and loved the derek zoolander character and the unrestrained whipping they released upon the fashion industry . \nsnap ! \nit was also nice to see several real-life models with small roles in the movie , not taking themselves too seriously . \nand if you\'re the type of person who likes celebrity cameos in films , well , don\'t look any further because dozens of famous faces show up here including vince vaughn , billy zane , winona ryder , christian slater , david duchovny , natalie portman and many , many others . \ni especially liked andy dick\'s complete make-over as the masseuse . \nhi-larious ! \nbut with all films of this type , there is bound to be some stuff that simply doesn\'t work . \na few particulars which didn\'t strike my fancy included the bulimia and " orgy " sequence with christine taylor , the break-dance fighting , and i also never get why they use real countries in plots like this ( why not just " make up " a country , instead of zeroing in on a certain people ? ) . \ni also could have done with less of the taylor character in general , since she wasn\'t all that interesting and seemed to slow things down every now and then ( more zoolander , dude ! ) . \nbut those few missteps were nothing compared to some of the more memorable scenes which definitely did work for me ! \ni almost pissed myself during the " gas station " disaster , absolutely adored the " walk-off " contest ( with david bowie as the judge , no less ) and appreciated many of zoolander\'s moronic one-liners ( " i was a merman . . . a \nmerman ! ! " ) . \nand i dare you to get his " monkey " photo shoot out of your head after seeing this movie ( " you\'re a monkey , derek . . . a \nmonkey ! " ) \ni also really liked the soundtrack and the pace of the film , both of which zipped and zagged , and established a nice rhythm to it all . \nagain , it\'s to note that this movie is dumb and not for everyone , but my guess is that if you laughed at the trailer , you will likely enjoy many of the quips in the actual picture as well . \nif you thought the trailer was dumb , skip this dodo bird and go rent austin powers again or something , a film from which there is an obvious influence here . \nblue steel , baby . . . yeah ! ! ! \nwhere\'s joblo coming from ? \naustin powers ( 7/10 ) - austin powers 2 ( 7/10 ) - deuce bigalow ( 7/10 ) - dude , where\'s my car ( 7/10 ) - freddy got fingered ( 5/10 ) - jay & silent bob strike back ( 8/10 ) - joe dirt ( 5/10 ) - meet the parents ( 8/10 ) - say it isn\'t so ( 3/10 ) \n', 'label': 'pos', 'embedding': [-0.028036825, -0.029044345, -0.047237873, 0.03214113, -0.005614088, 0.017529253, 0.007937407, 0.06114289, -0.018761829, 0.01684007, 0.0102760885, 0.022791898
========== TRUNCATED ==========
, -0.018393734, 0.029368952, 0.033914838, 0.0031390204, 0.053070612, -0.008919553, 0.029266886, 0.05452007, -0.0045006457, -0.06055649, -0.007895349, -0.03148518, -0.012017405, -0.035939064, 0.053935137, -0.07043659, 0.0029372668, -0.031107077, -0.021694716, 0.010721458, -0.039267533, -0.009802533, 0.05328437, -0.00858736, 0.00025471667, -0.0056747845, 0.044799235, -0.03857471, 0.0029413486, -0.04211427, 0.0022101186, 0.02241039, 0.018380161, -0.039794218, -0.022303186, 0.0346868, 0.029725887, 0.024404744, -0.04131337, -0.03219357, -0.0073991297, -0.005194563, -0.012993291, 0.018319614, 0.04653389, 0.015097829, -0.042021785, 0.033300966, -0.031226614, 0.027941037, 0.0070020626, -0.04302886, -0.016536027, -0.020188984, 0.04914125, 0.0081786495, -0.04490889, 0.020907514, -0.009434235, -0.0142212575, 0.047198955, 0.014853108, -0.023429299, 0.034682527, -0.084128655, -0.05055864, 0.07898019, -0.03895906, 0.11140129, -0.020291248, 0.05838838, 0.049867656, 0.04075819, 0.030516526, 0.066105805, 0.014409356, 0.011970263, 0.049177345, -0.013555308, 0.081342384, -0.00762423, -0.016328195, 0.014574487, -0.037175406, -0.0069948956, -0.008100563, -0.045961626, -0.009694281, -0.03278084, -0.018160239, -0.05202766, 0.0033189934, -0.0143624, -0.02671746, -0.001415489, -0.0047006086, -0.03224215, 0.014450342, 0.018371584, 0.018401045, 0.0055655516, -0.025224572, 0.015616917, 0.011846861, 0.00087809766, -0.05174142, 0.028903967, 0.0317321, 0.0055511137, -0.020039504, 0.009217652, -0.0075617577, 0.0013215569, 0.011287277, -0.03759058, -0.04326035, 0.0023132262, 0.005914851, 0.008933322, -0.014059064, -0.018902147, -0.028862504, -0.04993121, 0.01685796, -0.03562105, -0.019611944, 0.03822628, -0.021241069, -0.049925905, 0.027365098, 0.0032447774, 0.015983174, 0.044786885, 0.0017810394, -0.010186617, 0.0184125, -0.033125237, 0.010395376, 0.010963465, 0.0019406489, -0.016690668, -0.027291028, -0.040989026, 0.035864193, -0.04056406, -0.025016997, 0.01782647, -0.045990877, -0.02014531, -0.042198494, 0.0016370984, -0.0043466715, -0.019281795, 0.040232096, -0.011553848, -0.0522375, 0.014337117, -0.047861494, -0.05443393, -0.043306522, -0.0011884038, 0.020994298, 0.0205555, 0.015920509, -0.0020248282, -0.015946405, 0.007036882, 0.037424076, -0.042692408, -0.028045679, -0.022651706, -0.018605804, 0.061427973, -0.017790169, -0.0016451592, 0.015617046, -0.011951815, 0.057211805, 0.007342377, -0.0851792, 0.013083405, -0.043980345, 0.041450884, 0.029633973, 0.05452041, -0.05286796, -0.027135216, -0.0010516717, 0.024978315, 0.009902785, -0.027558176, -0.03661595, -0.0016361757, -0.009407849, -0.010746422, -0.00022211972, -0.014222683, -0.0051546367, -0.00023844474, 0.0035851959, 0.030983545, 0.018156303, -0.023779836, -0.018027946, -0.01814223, -0.014069596, -0.0033181321, 0.030819938, 0.003032134, -0.0062368484, 0.007540357, -0.020149812, 0.007868793, -0.02044547, 0.055769444, 0.016953874, -0.0317939, 0.007494329, -0.0050064838, 0.0025758962, -0.006226743, 0.03198515, -0.017085038, -0.028089255, -0.0021957676, 0.01799674, -0.025389675, -0.055563636, 0.018478252, -0.037858266, 0.087167695, -0.015485936, 0.06074542, 0.061953817, -0.028854333, -0.010893407, -0.037852608, -0.018422268, -0.033275485, 0.04706845, -0.0057725003, 0.03397722, -0.025772488, -0.0022985341, 0.022223232, -0.060510606, -0.011889764, 0.04926235, -0.0027368963, -0.004637742, 0.017194577, 0.044450004, 0.026138898, 0.04933249, -0.03422178, -0.0246854, -0.036936354, 0.037677344, 0.0062181093, 0.01729093, 0.019595413, -0.03493916, 0.011960312, -0.03956942, -0.024383888, 0.06777718, -0.010914413, -0.04620698, 0.0034036774, -0.045193076, 0.010701642, -0.008690606, -0.02563803, -0.04715476, 0.009559938, 0.018104384, -0.01613881, 0.037632063, -0.02254838, 0.03373061, 0.045927748, -0.0415655, -0.024590513, -0.038102556, -0.008331217, -0.04021093, 0.04263146, -0.030696195, -0.0032756466, 0.1096495, -0.009997374, -0.010033237, -0.00048540175, 0.052174386, 0.029515378, -0.055459205, -0.0008010739, -0.044277433, 0.011924893, -0.045417912, 0.011100777, 0.0072399713, -0.044269584], 'pred': 'pos'}...]
========== MIDDLE OF OUTPUT TRUNCATED ==========

AI Prompt

now let’s calculate the accuracy by comparing pred with label

AI Response

Here’s how we can calculate the accuracy by comparing the predictions with the actual labels:

correct = sum(1 for r in llm_results if r['pred'] == r['label'])
total = len(llm_results)
accuracy = correct / total

print(f"Accuracy: {accuracy:.2%} ({correct}/{total} correct)")

This will show us how well the LLM performed compared to our previous ML approaches. Let’s run this and see how it compares to the SVM (93%) and Random Forest (95%) results we got earlier!

correct = sum(1 for r in llm_results if r['pred'] == r['label'])
total = len(llm_results)
accuracy = correct / total

print(f"Accuracy: {accuracy:.2%} ({correct}/{total} correct)")

Accuracy: 97.00% (97/100 correct)

AI Prompt

Okay, an LLM directly is slightly better on sentiment classification. But this is an easy task for an LLM right?

AI Response

Yes, exactly! Sentiment analysis is a relatively straightforward task for modern LLMs because:

1. It’s one of the most basic NLP tasks - distinguishing positive from negative sentiment is something LLMs are inherently good at since they’re trained on vast amounts of text with emotional content
2. Movie reviews are particularly clear-cut examples - they usually explicitly state whether something is good or bad
3. The task doesn’t require much domain knowledge or complex reasoning

However, there are some key trade-offs to consider: 1. Speed: The ML approach with embeddings is much faster at inference time 2. Cost: Each LLM API call costs money, while running a trained ML model is essentially free 3. Scalability: You could process millions of reviews with the ML approach very efficiently

This is why the embedding + ML classifier approach is still valuable, especially when: - You need to process large volumes of text - You need real-time classification - You want to minimize costs - The classification task is more specialized or domain-specific where you can train on specific data

Would you like to try a more complex classification task where the ML approach might be more competitive with the LLM?

AI Prompt

Do you see tasks where embeddings + ML might be more competitive with LLMs?

AI Response

Yes, there are several scenarios where embeddings + ML could be more competitive or even outperform LLMs:

1. Domain-Specific Classification:
   • Medical document classification (diagnosis codes, procedure types)
   • Legal document categorization (contract types, case categories)
   • Technical support ticket routing Where the categories are very specific and the LLM might not have deep domain expertise
2. Multi-label Classification:
   • Product categorization (items belonging to multiple departments)
   • Content tagging (articles with multiple topics) Where items can belong to multiple categories simultaneously and the relationships between categories matter
3. Fine-grained Classification:
   • Emotion detection (beyond just positive/negative)
   • Customer feedback categorization (specific issue types)
   • Academic paper classification by subfield Where the distinctions between categories are subtle and require specific training
4. High-Stakes Classification:
   • Fraud detection
   • Security threat classification
   • Safety-critical systems Where you need consistent, deterministic results and can’t risk the occasional “creative” responses that LLMs might generate

Would you like to try implementing one of these more challenging scenarios to compare the approaches?