General Tips on Prompting¶
Before we get into some big applications of schema engineering I want to equip you with the tools for success. This notebook is to share some general advice when using prompts to get the most of your models.
Before you might think of prompt engineering as massaging this wall of text, almost like coding in a notepad. But with schema engineering you can get a lot more out of your prompts with a lot less work.
Classification¶
For classification we've found theres generally two methods of modeling.
- using Enums
- using Literals
Use an enum in Python when you need a set of named constants that are related and you want to ensure type safety, readability, and prevent invalid values. Enums are helpful for grouping and iterating over these constants.
Use literals when you have a small, unchanging set of values that you don't need to group or iterate over, and when type safety and preventing invalid values is less of a concern. Literals are simpler and more direct for basic, one-off values.
In [1]:
Copied!
import instructor
from openai import OpenAI
from enum import Enum
from pydantic import BaseModel, Field
from typing_extensions import Literal
client = instructor.patch(OpenAI())
# Tip: Do not use auto() as they cast to 1,2,3,4
class House(Enum):
Gryffindor = "gryffindor"
Hufflepuff = "hufflepuff"
Ravenclaw = "ravenclaw"
Slytherin = "slytherin"
class Character(BaseModel):
age: int
name: str
house: House
def say_hello(self):
print(
f"Hello, I'm {self.name}, I'm {self.age} years old and I'm from {self.house.value.title()}"
)
resp = client.chat.completions.create(
model="gpt-4-1106-preview",
messages=[{"role": "user", "content": "Harry Potter"}],
response_model=Character,
)
resp.model_dump()
import instructor from openai import OpenAI from enum import Enum from pydantic import BaseModel, Field from typing_extensions import Literal client = instructor.patch(OpenAI()) # Tip: Do not use auto() as they cast to 1,2,3,4 class House(Enum): Gryffindor = "gryffindor" Hufflepuff = "hufflepuff" Ravenclaw = "ravenclaw" Slytherin = "slytherin" class Character(BaseModel): age: int name: str house: House def say_hello(self): print( f"Hello, I'm {self.name}, I'm {self.age} years old and I'm from {self.house.value.title()}" ) resp = client.chat.completions.create( model="gpt-4-1106-preview", messages=[{"role": "user", "content": "Harry Potter"}], response_model=Character, ) resp.model_dump()
Out[1]:
{'age': 17, 'name': 'Harry Potter', 'house': <House.Gryffindor: 'gryffindor'>} In [2]:
Copied!
resp.say_hello()
resp.say_hello()
Hello, I'm Harry Potter, I'm 17 years old and I'm from Gryffindor
In [3]:
Copied!
class Character(BaseModel):
age: int
name: str
house: Literal["Gryffindor", "Hufflepuff", "Ravenclaw", "Slytherin"]
resp = client.chat.completions.create(
model="gpt-4-1106-preview",
messages=[{"role": "user", "content": "Harry Potter"}],
response_model=Character,
)
resp.model_dump()
class Character(BaseModel): age: int name: str house: Literal["Gryffindor", "Hufflepuff", "Ravenclaw", "Slytherin"] resp = client.chat.completions.create( model="gpt-4-1106-preview", messages=[{"role": "user", "content": "Harry Potter"}], response_model=Character, ) resp.model_dump()
Out[3]:
{'age': 11, 'name': 'Harry Potter', 'house': 'Gryffindor'} Arbitrary properties¶
Often times there are long properties that you might want to extract from data that we can not specify in advanced. We can get around this by defining an arbitrary key value store like so:
In [4]:
Copied!
class Property(BaseModel):
key: str = Field(description="Must be snake case")
value: str
class Character(BaseModel):
age: int
name: str
house: Literal["Gryffindor", "Hufflepuff", "Ravenclaw", "Slytherin"]
properties: list[Property]
resp = client.chat.completions.create(
model="gpt-4-1106-preview",
messages=[{"role": "user", "content": "Snape from Harry Potter"}],
response_model=Character,
)
resp.model_dump()
class Property(BaseModel): key: str = Field(description="Must be snake case") value: str class Character(BaseModel): age: int name: str house: Literal["Gryffindor", "Hufflepuff", "Ravenclaw", "Slytherin"] properties: list[Property] resp = client.chat.completions.create( model="gpt-4-1106-preview", messages=[{"role": "user", "content": "Snape from Harry Potter"}], response_model=Character, ) resp.model_dump()
Out[4]:
{'age': 38,
'name': 'Severus Snape',
'house': 'Slytherin',
'properties': [{'key': 'role', 'value': 'Potions Master'},
{'key': 'patronus', 'value': 'Doe'},
{'key': 'loyalty', 'value': 'Dumbledore'},
{'key': 'played_by', 'value': 'Alan Rickman'}]} Limiting the length of lists¶
In later chapters we'll talk about how to use validators to assert the length of lists but we can also use prompting tricks to enumerate values. Here we'll define a index to count the properties.
In this following example instead of extraction we're going to work on generation instead.
In [5]:
Copied!
class Property(BaseModel):
index: str = Field(..., description="Monotonically increasing ID")
key: str = Field(description="Must be snake case")
value: str
class Character(BaseModel):
age: int
name: str
house: Literal["Gryffindor", "Hufflepuff", "Ravenclaw", "Slytherin"]
properties: list[Property] = Field(
...,
description="Numbered list of arbitrary extracted properties, should be exactly 5",
)
resp = client.chat.completions.create(
model="gpt-4-1106-preview",
messages=[{"role": "user", "content": "Snape from Harry Potter"}],
response_model=Character,
)
resp.model_dump()
class Property(BaseModel): index: str = Field(..., description="Monotonically increasing ID") key: str = Field(description="Must be snake case") value: str class Character(BaseModel): age: int name: str house: Literal["Gryffindor", "Hufflepuff", "Ravenclaw", "Slytherin"] properties: list[Property] = Field( ..., description="Numbered list of arbitrary extracted properties, should be exactly 5", ) resp = client.chat.completions.create( model="gpt-4-1106-preview", messages=[{"role": "user", "content": "Snape from Harry Potter"}], response_model=Character, ) resp.model_dump()
Out[5]:
{'age': 38,
'name': 'Severus Snape',
'house': 'Slytherin',
'properties': [{'index': '1',
'key': 'position_at_hogwarts',
'value': 'Potions Master'},
{'index': '2', 'key': 'patronus_form', 'value': 'Doe'},
{'index': '3', 'key': 'loyalty', 'value': 'Albus Dumbledore'},
{'index': '4', 'key': 'played_by', 'value': 'Alan Rickman'},
{'index': '5', 'key': 'final_act', 'value': 'Protecting Harry Potter'}]} Defining Multiple Entities¶
Now that we see a single entity with many properties we can continue to nest them into many users
In [6]:
Copied!
from collections.abc import Iterable
class Character(BaseModel):
age: int
name: str
house: Literal["Gryffindor", "Hufflepuff", "Ravenclaw", "Slytherin"]
resp = client.chat.completions.create(
model="gpt-4-1106-preview",
messages=[{"role": "user", "content": "Five characters from Harry Potter"}],
response_model=Iterable[Character],
)
for character in resp:
print(character)
from collections.abc import Iterable class Character(BaseModel): age: int name: str house: Literal["Gryffindor", "Hufflepuff", "Ravenclaw", "Slytherin"] resp = client.chat.completions.create( model="gpt-4-1106-preview", messages=[{"role": "user", "content": "Five characters from Harry Potter"}], response_model=Iterable[Character], ) for character in resp: print(character)
age=11 name='Harry Potter' house='Gryffindor' age=11 name='Hermione Granger' house='Gryffindor' age=11 name='Ron Weasley' house='Gryffindor' age=11 name='Draco Malfoy' house='Slytherin' age=11 name='Neville Longbottom' house='Gryffindor'
In [7]:
Copied!
from collections.abc import Iterable
class Character(BaseModel):
age: int
name: str
house: Literal["Gryffindor", "Hufflepuff", "Ravenclaw", "Slytherin"]
resp = client.chat.completions.create(
model="gpt-4-1106-preview",
messages=[{"role": "user", "content": "Five characters from Harry Potter"}],
stream=True,
response_model=Iterable[Character],
)
for character in resp:
print(character)
from collections.abc import Iterable class Character(BaseModel): age: int name: str house: Literal["Gryffindor", "Hufflepuff", "Ravenclaw", "Slytherin"] resp = client.chat.completions.create( model="gpt-4-1106-preview", messages=[{"role": "user", "content": "Five characters from Harry Potter"}], stream=True, response_model=Iterable[Character], ) for character in resp: print(character)
age=11 name='Harry Potter' house='Gryffindor' age=11 name='Hermione Granger' house='Gryffindor' age=11 name='Ron Weasley' house='Gryffindor' age=17 name='Draco Malfoy' house='Slytherin' age=11 name='Luna Lovegood' house='Ravenclaw'
Defining Relationships¶
Not only can we define lists of users, but with lists of properties we can also easily define lists of references. It's one of the more interesting things I've learned about prompting.
In [8]:
Copied!
class Character(BaseModel):
id: int
name: str
friends_array: list[int] = Field(description="Relationships to their friends using the id")
resp = client.chat.completions.create(
model="gpt-4-1106-preview",
messages=[{"role": "user", "content": "5 kids from Harry Potter"}],
stream=True,
response_model=Iterable[Character],
)
for character in resp:
print(character)
class Character(BaseModel): id: int name: str friends_array: list[int] = Field(description="Relationships to their friends using the id") resp = client.chat.completions.create( model="gpt-4-1106-preview", messages=[{"role": "user", "content": "5 kids from Harry Potter"}], stream=True, response_model=Iterable[Character], ) for character in resp: print(character)
id=1 name='Harry Potter' friends_array=[2, 3, 4, 5, 6] id=2 name='Hermione Granger' friends_array=[1, 3, 4, 5] id=3 name='Ron Weasley' friends_array=[1, 2, 4, 6] id=4 name='Neville Longbottom' friends_array=[1, 2, 3, 5] id=5 name='Luna Lovegood' friends_array=[1, 2, 4, 6] id=6 name='Draco Malfoy' friends_array=[1, 3, 5]
With the tools we've discussed, we can find numerous real-world applications in production settings. These include extracting action items from transcripts, generating fake data, filling out forms, and creating objects that correspond to generative UI. These simple tricks will be highly useful.
Missing Data¶
The Maybe pattern is a concept in functional programming used for error handling. Instead of raising exceptions or returning None, you can use a Maybe type to encapsulate both the result and potential errors.
This pattern is particularly useful when making LLM calls, as providing language models with an escape hatch can effectively reduce hallucinations.
In [9]:
Copied!
from typing import Optional
class Character(BaseModel):
age: int
name: str
class MaybeCharacter(BaseModel):
result: Optional[Character] = Field(default=None)
error: bool = Field(default=False)
message: Optional[str]
from typing import Optional class Character(BaseModel): age: int name: str class MaybeCharacter(BaseModel): result: Optional[Character] = Field(default=None) error: bool = Field(default=False) message: Optional[str]
In [10]:
Copied!
def extract(content: str) -> MaybeCharacter:
return client.chat.completions.create(
model="gpt-3.5-turbo",
response_model=MaybeCharacter,
messages=[
{"role": "user", "content": f"Extract `{content}`"},
],
)
def extract(content: str) -> MaybeCharacter: return client.chat.completions.create( model="gpt-3.5-turbo", response_model=MaybeCharacter, messages=[ {"role": "user", "content": f"Extract `{content}`"}, ], )
In [11]:
Copied!
extract("Harry Potter")
extract("Harry Potter")
Out[11]:
MaybeCharacter(result=Character(age=17, name='Harry Potter'), error=False, message=None)
In [12]:
Copied!
user = extract("404 Error")
if user.error:
raise ValueError(user.message)
user = extract("404 Error") if user.error: raise ValueError(user.message)
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) /Users/jasonliu/dev/instructor/docs/tutorials/2-tips.ipynb Cell 20 line 4 <a href='vscode-notebook-cell:/Users/jasonliu/dev/instructor/docs/tutorials/2-tips.ipynb#X25sZmlsZQ%3D%3D?line=0'>1</a> user = extract("404 Error") <a href='vscode-notebook-cell:/Users/jasonliu/dev/instructor/docs/tutorials/2-tips.ipynb#X25sZmlsZQ%3D%3D?line=2'>3</a> if user.error: ----> <a href='vscode-notebook-cell:/Users/jasonliu/dev/instructor/docs/tutorials/2-tips.ipynb#X25sZmlsZQ%3D%3D?line=3'>4</a> raise ValueError(user.message) ValueError: 404 Error