Introduction
Large Language Models (LLMs) have transformed how we interact with machines by making conversations feel intuitive, responsive, and increasingly intelligent. They now power everything from basic chat interfaces to complex AI agents that can plan, reason, and take action across tasks.
What enables this intelligence isn’t just the model’s parameters. It’s how we structure the interaction. To unlock the full potential of LLMs, especially in multi-turn or tool-augmented setups, the model must understand who is speaking, what role they are playing, and what has already happened in the conversation.
This is where roles come in, such as system, user, and assistant, which define the context and intent behind every message. In more advanced agentic systems, additional roles like tool_use, tool_result, and planner help organize reasoning and decision-making. These roles guide the model’s behavior, ensure context is preserved, and enable actions beyond simple text generation.
Whether you’re building a friendly chatbot or a fully autonomous agent, understanding and using role-based formatting is key to building reliable and effective LLM applications.
Understanding the Roles in LLM Conversations
When working with LLMs in chat-based apps or agent systems, roles help structure the conversation. Each message has a role that tells the model who is speaking and what kind of message it is. This helps the model decide how to respond and keep track of the conversation.
The basic roles are system, user, and assistant. These cover most everyday use cases. In more advanced setups, like when building AI agents, extra roles are added to handle things like tools, reasoning steps, or function calls. Now let’s take a look at how each role fits into the overall flow, from simple conversations to agent-level capabilities.
1. System Role: Set the Behavior
The system role gives the model general instructions before the conversation begins. It sets the context for how the model should act throughout the chat.
Examples:
This message is usually sent once at the beginning and stays active for the whole conversation. It’s useful for defining tone, personality, or any specific rules you want the model to follow.
2. User Role: The Human Input
The user role is where the person types their message. These are the questions or commands that the model responds to.
Examples:
Every new message from the user goes into this role. It’s what drives the interaction forward.
3. Assistant Role: The Model’s Response
The assistant role is where the model replies. Based on the system prompt and the latest user message, the model generates a response in this role.
Examples:
-
“You might enjoy visiting Tokyo for its culture, Kyoto for its temples, and Okinawa for its beaches.”
-
“A neural network is a type of machine learning model inspired by the human brain…”
This is the part users see as the model’s output.
4. Extra Roles for Agents: Tools and Reasoning
In more advanced cases, especially when building agent-based systems, there are extra roles that help the model do more than just reply with text. These include calling tools, showing results, or working through a plan.
Examples:
-
OpenAI: Uses roles like
function_callto let the model call external tools -
Claude: Uses
tool_useandtool_resultto show when a tool is used and what it returned -
LLaMA 3: Uses special tags like
<|python_tag|>for running code
These extra roles help the model go beyond conversation. They allow it to get live data, make decisions step by step, and carry out tasks more like an agent.
Why These Roles Matter
The system, user, and assistant roles work together to form the complete message history that an LLM uses to understand and respond. If these roles aren’t used correctly, the conversation can quickly lose context, drift off-topic, or become unpredictable.
Using roles properly helps you build LLM applications that are consistent, clear, and capable of handling more complex tasks. Here’s why they matter:
- Context Tracking: Roles help the model understand who said what and in what order. This lets the conversation flow naturally, allows the model to refer back to earlier messages, and keeps it from getting confused during longer chats.
- Controlling Behavior: The
systemrole sets the overall tone, rules, or personality for the model. This keeps the assistant aligned with your product’s voice and avoids responses that feel out of place. - Clear Task Execution: By separating system instructions, user prompts, and assistant replies, the model can better understand what’s being asked and how to respond. It removes ambiguity and improves the quality of answers.
These roles are also the base structure for more advanced features like tool use, planning steps, or multi-turn reasoning. If you’re building agents or tool-augmented systems, this structure is what makes those workflows possible.
Understanding the Roles in Agents
First, let’s understand what agents actually are. The term “agent” is often used loosely, and its definition can vary depending on the context. A helpful way to think about it comes from Anthropic’s post Building Effective Agents, which distinguishes between workflows and agents.
A workflow follows a fixed path of execution. An agent, by contrast, dynamically decides what to do next based on the current situation. This flexibility is what allows agents to operate in open-ended environments and handle tasks with many possible paths.
Core Components of Agents
Most modern agents are built around three essential components: memory, tools, and planning.
Memory
LLMs are stateless. They do not retain memory of past interactions unless that context is explicitly provided. In chat applications, this usually means managing and resending the full message history with each request.
Some platforms also support prompt caching, allowing frequently repeated inputs (such as long system messages) to be reused without reprocessing. This reduces latency and cost.
Tools
Tools allow agents to interact with external systems, for example, by calling APIs, searching the web, or running local code. These are often defined through schemas or function signatures.
Well-documented tools improve accuracy. A tool’s name, description, and input schema should be written as if the model were a developer using it. Clear documentation leads to better usage.
Planning
Agents need to reason about tasks and determine next steps. Planning can be as simple as using built-in chain-of-thought reasoning or as complex as maintaining explicit plans that update with new information.
Effective planning also includes the ability to recover from failed attempts and revise the approach when needed.
How Roles Work in Agent-Based Systems
As LLMs are integrated with memory, tools, and planning mechanisms, roles become a critical part of the architecture. They help structure the interaction and enable agents to reason, act, and track progress effectively.
Organizing Internal Steps
Agents often represent each internal action using a specific role. For example, a planning step might be expressed in the assistant role, a tool invocation in tool_use, and the output in tool_result. This helps maintain clarity over multi-step reasoning and tool execution.
Supporting Step-by-Step Reasoning
Techniques like Chain-of-Thought, ReAct, and Tree-of-Thoughts rely on assigning a role to each stage of reasoning. This makes the process interpretable, debuggable, and modular.
Handling Tool Use
When the agent calls a tool, it creates a tool_use message that includes the tool name and inputs. The response from the tool is captured in a tool_result message. This structure ensures tool use is clearly separated and easy to trace.
Planning and Feedback Loops
Many agents follow a loop of planning, acting, observing, and revising. Using roles to represent each phase helps manage these loops cleanly and makes it easier to extend or adjust the agent’s logic.
Tracking Memory and Context
Roles help manage both short-term memory (like previous messages and tool calls) and long-term memory (such as stored documents or knowledge). Labeling each message with a clear role ensures the agent can reference past steps effectively.
Multi-Agent Collaboration
In systems with multiple agents, roles can define each agent’s function — such as “Planner”, “Researcher”, or “Executor”. This helps avoid ambiguity and ensures coordination across components.
Roles in agent-based systems are more than just a formatting convention. They define how reasoning, tool use, memory management, and collaboration happen. Used well, they make agents more reliable, interpretable, and capable of handling complex tasks.
Examples of Using Roles in LLM and Agentic Systems
Let’s walk through some practical examples of implementing role-based prompt engineering. We’ll start with fundamental conversational roles using Clarifai’s OpenAI-compatible API, then extend to tool-calling capabilities, and finally explore how agentic frameworks like Google’s Agent Development Kit (ADK) streamline the development of advanced, role-driven agents.
1. Basic Conversational Roles: System and User
Even the simplest chatbot benefits from structured roles. The system role establishes the model’s persona or ground rules, while the user role delivers the human input. Below is an example of how we’ve used Clarifai’s OpenAI-compatible API to define these roles in the message history and guide the model’s behavior.
Code Example: Setting Persona and User Input
In this example, the system role explicitly instructs the model to act as a “helpful travel assistant” and prioritize “sustainable travel options.” The user role then provides the specific query. This foundational use of roles ensures the model’s response is aligned with the desired behavior from the very first turn.
2. Advanced Roles: Enabling Tool Use for Agentic Behavior
Building on basic conversational roles, agentic systems introduce additional roles to support interactions with external tools. This allows LLMs to fetch real-time data, run calculations, or call APIs as needed. The model decides when to call a tool, and your application returns the tool’s output back to the model, helping it generate a complete and informed response.
Code Example: LLM Tool Calling and Result Handling
This example demonstrates a complete agentic loop:
-
The
userinitiates the interaction by asking about the weather. -
The LLM, guided by the
systemrole (which defines it as a “helpful assistant with access to a weather tool”) and thetoolsprovided, recognizes the need to use an external function. It responds in theassistantrole, but instead of text, it provides atool_callsobject, indicating its intention to invoke theget_weatherfunction. -
Your application intercepts this
tool_callfrom theassistant‘s response. It then executes themock_get_weather_apifunction (which returns predefined, simulated weather data for demonstration purposes), retrieving thetool_output. -
The
tool_outputis then appended to the message history with therole: "tool"(ortool_resultin some API implementations), explicitly indicating that this message contains the result of a tool execution. This message is also linked back to the originaltool_call_id. -
Finally, the updated message history (including the initial
systemandusermessages, theassistant‘stool_call, and thetool‘stool_output) is sent back to the LLM. With the tool’s result now available in the conversation context, the LLM can generate a direct, informed answer for the user, presented once again in theassistantrole. This multi-turn interaction, driven by these specific and distinct roles, is the essence of agentic behavior.
3. Agent Development Kits (ADKs): Streamlining Agent Construction with Google ADK
While direct API calls give you granular control, Agent Development Kits and Frameworks provide higher-level abstractions to simplify building and managing complex agents. They often encapsulate the multi-step reasoning, tool orchestration, and memory management into a more intuitive framework. Google’s ADK, for instance, allows you to define agents with clear instructions and integrated tools, handling the underlying role-based messaging automatically.
Code Example: Building an Agent with Google ADK and Clarifai LLM
The above Google ADK example demonstrates how a framework simplifies agent development:
-
LiteLlm: This class allows ADK to seamlessly integrate with Clarifai’s OpenAI-compatible endpoint, making your agents flexible across different LLM providers. -
AgentDefinition: TheAgentclass itself is where you define the agent’s core identity. Theinstructionparameter serves as the primary system-level prompt, guiding the agent’s behavior and purpose. Thetoolsparameter registers your Python functions as callable tools for the LLM. -
RunnerandSessionService: ADK’sRunnerorchestrates the interaction, managing the conversation flow, calling tools when needed, and handling the back-and-forth messaging with the LLM (including role-based formatting). TheInMemorySessionServicemanages the conversation history (memory), ensuring the agent has context across turns. -
Simplified Interaction: From the user’s perspective (and your application’s logic), you simply send a
usermessage to therunner, and the ADK handles all the complex role management, tool invocation, and result processing behind the scenes, ultimately returning a final response. This highlights how frameworks abstract away the lower-level prompt engineering details, allowing you to focus on the agent’s overall logic and capabilities.
Conclusion
Roles are a fundamental part of working effectively with LLMs. They help the model stay grounded, maintain context, and respond reliably, especially when tools or multi-step reasoning are involved.
We started with the core roles: system for instructions, user for input, and assistant for responses. Using Clarifai’s OpenAI-compatible API, we showed how clearly defining these roles keeps interactions stable and purposeful.
We also covered how agent frameworks and tool use work together, from the model deciding when to call a tool, to your code executing it, returning the result via the tool role, and the model using that output to respond. Kits like Google ADK handle much of this automatically, managing roles and orchestration behind the scenes.
If you’re looking to build AI agents, we have a full walkthrough to help you get started, including how to build a blog-writing agent using CrewAI. Checkout the tutorial here.
To explore other agentic frameworks like Google ADK, OpenAI, and CrewAI in more depth, along with complete code examples and documentation, check out our full library here.
