Use Translation for Paraphrasing
Large Language Models are sensitive to the way that they are prompted. When prompted incorrectly, they might perform much worse despite having the information or capability to respond to the prompt. We can help find semantically similar prompts by performing back translation - where we translate our prompts to another language and back to encourage more diversity in the rephrased prompts.
Prompt paraphrasing 1. provides some ways for us to improve on the phrasing of our prompts to do so.
We can implement this using instructor as seen below.
import instructor
from openai import AsyncOpenAI
from pydantic import BaseModel
import random
client = instructor.from_openai(AsyncOpenAI())
class TranslatedPrompt(BaseModel):
translation: str
async def translate_prompt(prompt: str, from_language: str, to_language: str):
return await client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "system",
"content": f"""
You are an expert translation assistant.
You are going to be given a prompt and
asked to translate it from {from_language}
to {to_language}. Paraphrase and use
synonyms where possible, especially for
the examples.
""",
},
{"role": "user", "content": f"Prompt: {prompt}"},
],
response_model=TranslatedPrompt,
)
async def generate_permutation(prompt: str, language: str) -> str:
tranlated_prompt = await translate_prompt(prompt, "english", language)
backtranslated_prompt = await translate_prompt(
tranlated_prompt.translation, language, "english"
)
return backtranslated_prompt.translation
async def generate_prompts(
prompt: str, languages: list[str], permutations: int
) -> list[str]:
coros = [
generate_permutation(prompt, random.choice(languages))
for _ in range(permutations)
]
return await asyncio.gather(*coros)
if __name__ == "__main__":
import asyncio
prompt = """
You are an expert system that excels at Sentiment
Analysis of User Reviews.
Here are a few examples to refer to:
1. That was a fantastic experience I had! I'm
definitely recommending this to all my friends
// Positive
2. I think it was a passable evening. I don't think
there was anything remarkable or off-putting for me.
// Negative
3. I'm horrified at the state of affairs in this new
restaurant // Negative
Sentence: This was a fantastic experience!
"""
languages = ["french", "spanish", "chinese"]
permutations = 2
generated_prompts = asyncio.run(generate_prompts(prompt, languages, permutations))
for prompt in generated_prompts:
print(prompt)
"""
You are an expert system specializing in user review sentiment analysis. Here are a few examples to guide you: 1. It was an exceptional experience! I will definitely recommend it to all my friends // Positive 2. I think it was a mediocre evening. There wasn't anything outstanding or particularly bad for me // Negative 3. I am horrified by the condition of things in this new restaurant // Negative Sentence: It was an amazing experience!
"""
"""
You are an expert system that excels in User Review Sentiment Analysis.
Here are some reference examples:
1. I had an amazing experience! I will definitely recommend it to all my friends.
// Positive
2. I think it was an average evening. I don’t believe there was anything remarkable or unpleasant about it for me.
// Negative
3. I am horrified by the situation at this new restaurant.
// Negative
Sentence: This was a fantastic experience!
"""
"""
You are an expert system skilled in conducting user
review sentiment analysis.
Here are some examples for reference:
1. That was an awesome experience! I'll definitely
recommend it to all my friends // Positive
2. I think it was an okay evening. I don't find
anything particularly outstanding or unpleasant.
// Neutral
3. I am very shocked by the condition of this new
restaurant // Negative
Sentence: This was a wonderful experience!
"""