Introduction

Prompt engineering–the art, science, and craft of coaxing useful, relevant output from generative artificial intelligence (AI), such as large language models (LLMS)–has emerged as a valuable new skill in the age of artificial intelligence. Although LLMs such as ChatGPT are designed to be ‘conversational’ and hence easy to use simply by typing in natural-sounding questions and comments, advanced techniques give much more control over how the model will behave.

Prompt engineering is a new and rapidly evolving field, and is useful both for individual users of LLMs and developers who want to build applications on top of LLM technology to craft specific user experiences without any need for, or in addition to, training custom models, which is much more expensive and time-consuming.

This post will not try to review all of the prompt engineering techniques that have been described–for that, see this handy review article. Instead, I’ll focus on the all-important concept of context, which can be broken down into two components instructions and data. I’ll show you how a powerful new tool, Prompt Poet, can be used to manage these components, allowing you to build powerful, extensible, and reusable prompts to greatly boost the power of your LLM applications without the need for model fine-tuning.

Prompt Poet was published by the creators of Character.ai, a platform and makerspace for personalized conversational AIs, which was recently acquired by Google. This suggests the exciting possibility that Prompt Poet could represent the future direction of prompt context management adopted across Google’s AI projects, such as Gemini.

Two types of context: instructions and data

Customizing an LLM application, such as a chatbot, often means giving it specific instructions about how to act. This might mean describing a certain type of personality type, situation, or role, or even a specific historical or fictional person. For example, when asking for help with a moral dilemma, you can ask the model to answer in the style of someone specific, which will very much influence the type of answer you get. You’ll find that details of a described situation can matter too. For example, trying a few variations of the following prompt will reveal that LLMs can craft their answers to a variety of hypothetical assumptions:

Simulate a panel discussion with Immanuel Kant, Che Guevara, and Lao Tzu.

Each one should give me advice individually, then each should get a chance to comment on the others' responses and conclude. 

Let's suppose that all three discussants are very hungry at the time.

My question: The pizza place gave me an extra pie, should I tell them or keep it?

To experiment with more developed character personas, check out Character AI.

However, effective prompt engineering isn’t just about crafting the right instructions about how to answer; it also involves creating a specific, customized data context for an LLM interaction, in which the right facts are available. This means loading in specific data, such as personal data for the user, real-time information, or hard-to-access data that the LLM wouldn’t have or needs to be fact-checked rather than relying solely on its training data. This approach–called retrieval assisted generation (RAG)–allows the AI’s output to be far more accurate and relevant to the user’s specific situation than would be possible for an uninformed generic model.

Data can be loaded in manually, just by typing it into ChatGPT, for example: If you ask for advice about how to install some software, you have to tell it about your hardware. If you ask for help crafting the perfect resume, you have to tell it your skills and work history first. However, while this is great for personal use, it has obvious limitations. Manually crafting data-rich prompts can be arduous and error-prone.

How do we supply this data efficiently and accurately to the model? This is where prompt templating comes into play, offering a streamlined solution to embed complex data within prompts.

Crafting contexts with prompt templating

Prompt Poet is a powerful tool designed to address the challenges of prompt engineering by simplifying and automating the creation of dynamic, data-rich prompts. Leveraging YAML and Jinja2, Prompt Poet allows users to create flexible and dynamic prompts, greatly enhancing the power of interactions with AI models. While there are other prompt templating solutions available, Prompt Poet offers a solution with several advantages:

  • Low Code Approach: Simplifies prompt design for both developers and non-technical users.
  • Template Flexibility: Uses YAML and Jinja2 to allow for complex prompt structures.
  • Context Management: Easily integrates external data and manages context dynamically.
  • Efficiency: Reduces the time spent on engineering string manipulations, letting users focus on crafting optimal prompts.

Example: daily planner

To illustrate the power of Prompt Poet, let’s work through a simple example. Suppose we want to put together an daily planning assistant to remind the user of upcoming events and provide contextual information to help prepare for their day, based on real-time data.

For example, you might want it to give you something like this:

Good morning! It looks like you have some virtual meetings in the morning and an afternoon hike planned. Don't forget to bring some water and sunscreen for your hike since it's sunny outside.

 Here are your schedule and current conditions for today:

- **09:00 AM:** Virtual meeting with the marketing team
- **11:00 AM:** Virtual one-on-one with the project manager
- **07:00 PM:** Afternoon hike at Discovery Park with friends

It's currently 65°F and sunny, and the AI is 17, so it should be good conditions for your afternoon hike at Discovery Park.

Please be aware that there's a bridge closure on I-90, so you might experience some delays if you need to travel through that area.

Is there anything else you would like to know?

To do that, we’ll need to provide three different pieces of context to the model, 1) customized instructions about the task, and 2) the required data to define the factual context of the user interaction, and 3) a small training data-set to support ‘few-shot learning’, in the form of some examples of the kind of thing we want it to say.

Prompt Poet gives us some powerful tools for handling this context. We’ll start by creating a template to hold the general form of the instructions, and filling it in with specific data at the time when we want to run the query. For the above example, we might use the following:


raw_template = """
- name: system instructions
  role: system
  content: |
    You are a helpful daily planning assistant. Use the following information about the user's schedule and conditions in their area to provide a detailed summary of the day. Remind them of upcoming events and bring any warnings or unusual conditions to their attention, including weather, traffic, or air quality warnings. Ask if they have any follow-up questions.

- name: realtime data
  role: system
  content: |   
    Weather in {{ user_city }}, {{ user_country }}:
    - Temperature: {{ user_temperature }}°C
    - Description: {{ user_description }}
    
    Traffic in {{ user_city }}:
    - Status: {{ traffic_status }}
    
    Air Quality in {{ user_city }}:
    - AQI: {{ aqi }}
    - Main Pollutant: {{ main_pollutant }}
    
    Upcoming Events:
    {% for event in events %}
    - {{ event.start }}: {{ event.summary }}
    {% endfor %}
    
"""

template_data = {
    "user_city": user_city,
    "user_country": user_country,
    "user_temperature": user_weather_info["temperature"],
    "user_description": user_weather_info["description"],
    "traffic_status": traffic_info,
    "aqi": aqi_info["aqi"],
    "main_pollutant": aqi_info["main_pollutant"],
    "events": events_info
}

Note that the template uses the {name:,role:,content} format of OpenAI, which has become something of a defacto standard recognized by most LLMs. Also note that Prompt Poet templating provides the ability to loop through an array of items, in this case, events, which we can suppose will be loaded in from a calendar app.

Take a few shots

Another kind of context we can add is a simulated history of the kind of responses we want the model to give. This is called “few shot learning”, with the ‘shots’ referring to the examples. It is a way of saying, “be this kind of model; keep doing this kind of thing”.

In our case, this would mean adding a few messages that look like they came from the LLM itself. Hence, in the following example, we label those examples with name: response and role: assistant, as ‘assistant’ is how LLMs self-refer, following ChatGPT’s convention.

- name: realtime data
  role: system
  content: |
    Weather in Seattle, US:
    - Temperature: 25°C
    - Description: Sunny

    Traffic in Seattle:
    - Status: Bridge closure on I-90, expect delays

    Air Quality in Seattle:
    - AQI: 45
    - Main Pollutant: PM2.5

    Upcoming Events:
    - 2024-08-06T09:00:00: Virtual meeting with the marketing team
    - 2024-08-06T11:00:00: Virtual one-on-one with the project manager
    - 2024-08-06T15:00:00: Afternoon hike at Discovery Park

- name: response
  role: assistant
  content: |
    Good morning! Here's your schedule and current conditions for today:

    **Weather in Seattle, US:**
    - It's currently 25°C and sunny. A great day to be outside!

    **Traffic in Seattle:**
    - Please be aware that there's a bridge closure on I-90, so you might experience some delays if you need to travel through that area.

    **Air Quality in Seattle:**
    - The AQI is 45, which is considered good. The main pollutant is PM2.5, but it's currently at a safe level.

    **Upcoming Events:**
    - **09:00 AM:** Virtual meeting with the marketing team
    - **11:00 AM:** Virtual one-on-one with the project manager
    - **03:00 PM:** Afternoon hike at Discovery Park

    It looks like you have some virtual meetings in the morning and an afternoon hike planned. Don't forget to bring some water and sunscreen for your hike since it's sunny outside.

    Is there anything else you would like to know?

- name: realtime data
  role: system
  content: |
    Weather in New York, US:
    - Temperature: 18°C
    - Description: Rainy

    Traffic in New York:
    - Status: Subway delays on Line 2, expect longer commute times

    Air Quality in New York:
    - AQI: 55
    - Main Pollutant: O3

    Upcoming Events:
    - 2024-08-06T10:00:00: Team brainstorming session
    - 2024-08-06T13:00:00: Lunch with Sarah
    - 2024-08-06T16:00:00: Dentist appointment

- name: response
  role: assistant
  content: |
    Good morning! It looks like you have a busy day ahead with a team session, lunch meeting, and a dentist appointment. Make sure to leave a bit earlier for your appointments to account for subway delays, and don't forget your umbrella. Is there anything else you would like to know or any follow-up questions you have?

    **Weather in New York, US:**
    - It's currently 18°C and rainy.

    **Traffic in New York:**
    - There are subway delays on Line 2, so expect a longer commute time if you're using the subway.

    **Air Quality in New York:**
    - The AQI is 55, which is moderate. The main pollutant is O3. It's generally safe, but those with respiratory conditions should be cautious.

    **Upcoming Events:**
    - **10:00 AM:** Team brainstorming session
    - **01:00 PM:** Lunch with Sarah
    - **04:00 PM:** Dentist appointment

- name: realtime data
  role: system
  content: |
    Weather in San Francisco, US:
    - Temperature: 20°C
    - Description: Cloudy

    Traffic in San Francisco:
    - Status: No major traffic alerts

    Air Quality in San Francisco:
    - AQI: 30
    - Main Pollutant: PM10

    Upcoming Events:
    - 2024-08-06T09:30:00: Morning jog at Golden Gate Park
    - 2024-08-06T12:00:00: Client call with XYZ Corp
    - 2024-08-06T17:00:00: Dinner with Alex

- name: response
  role: assistant
  content: |
    Good morning! Your day starts with a refreshing jog, followed by a client call at noon and dinner with Alex in the evening.
    Is there anything else you would like to know or any follow-up questions you have? Here's your schedule and current conditions for today:

    **Weather in San Francisco, US:**
    - It's currently 20°C and cloudy. Perfect weather for a jog!

    **Traffic in San Francisco:**
    - There are no major traffic alerts, so your commute should be smooth.

    **Air Quality in San Francisco:**
    - The AQI is 30, which is good. The main pollutant is PM10, and the air quality is safe for all activities.

    **Upcoming Events:**
    - **09:30 AM:** Morning jog at Golden Gate Park
    - **12:00 PM:** Client call with XYZ Corp
    - **05:00 PM:** Dinner with Alex

Bringing it all together

The code below uses Prompt Poet’s Prompt class to combined multiple data sources into a single, coherent prompt.

This allows us to invoke an daily planning assistant to provides personalized, context-aware responses. By pulling in weather data, traffic updates, AQI information, and calendar events, the model can offer detailed summaries and reminders, enhancing the user experience.

You can clone and experiment with the full working code example here.


# Get user's location and data
user_weather_info = get_weather_info(user_weather)
traffic_info = get_traffic_info(user_traffic)
aqi_info = get_aqi_info(user_aqi)
events_info = get_events_info(calendar_events)

# Template data
template_data = {
    "user_city": user_city,
    "user_country": user_country,
    "user_temperature": user_weather_info["temperature"],
    "user_description": user_weather_info["description"],
    "traffic_status": traffic_info,
    "aqi": aqi_info["aqi"],
    "main_pollutant": aqi_info["main_pollutant"],
    "events": events_info
}

# Create the prompt using Prompt Poet
prompt = Prompt(
    raw_template=yaml.dump(raw_template_yaml),
    template_data=template_data
)

# Get response from OpenAI
model = openai.ChatCompletion.create(
  model="gpt-4",
  messages=prompt.messages
)

Conclusion

Mastering the fundamentals of prompt engineering, particularly the roles of instructions and data, is crucial for maximizing the potential of large language models (LLMs). Prompt Poet stands out as a powerful tool in this field, offering a streamlined approach to creating dynamic, data-rich prompts.

Prompt Poet’s low-code, flexible template system makes prompt design accessible and efficient. By integrating external data sources that would not be available to an LLM’s training, data-filled prompt templates can better ensure AI responses are accurate and relevant to the user, significantly enhancing how useful they can be.

The simple example of an daily planning assistant illustrates how Prompt Poet can combine real-time data like weather, traffic, and calendar events to provide tailored, actionable insights. This demonstrates the importance of context in prompt engineering and how external data integration can transform AI interactions.

In conclusion, leveraging context and external data with tools like Prompt Poet will elevate your prompt engineering skills, enabling you to craft more effective and nuanced prompts. As AI evolves, staying proficient in prompt engineering will be key to developing innovative applications that meet diverse user needs with precision. Prompt Poet equips you to create intelligent, context-aware AI solutions, enhancing their relevance and impact.

Bio

Michael Trestman, PhD is a researcher and writer based in Portland, OR.