Orchestrating AI Agents: A Deep Dive into AG2's Multi-Agent Architecture for absolute Beginners
The evolving landscape of agentic systems and methods are taking over the world by storm. vertical agents are poised to become the cornerstone of AI Engineerng , revolutionizing how we approach automation in offices and other environments. These agents having their specialized domain expertise and linked with their specific domain methods will be set to automate routine tasks, streamline workflows, and enhance productivity across industries. Agent Orchestration is one of the most important research right now trying to solve how multiple agents collaborate with Autonoumous syncing or with Human Input across different environments and usecases. As organizations begin to harness the power of these interconnected systems, we're witnessing the emergence of a new paradigm in business efficiency and innovation, one where the synergy between specialized agents creates outcomes greater than the sum of their parts.
Brief Overview of AG2
Agentic Systems are a research field currently and growing as fast it can with multiple frameworks , methodologies popping here and there . Recently a framework named ag2(formerly autogen) came out in the market and is gaining popularity in building production grade multi agents systems. Designed to streamline the development and orchestration of multi-agent systems. It stands out by offering a flexible architecture for constructing specialized agents that can seamlessly collaborate to solve complex tasks and many out of the box agents. Here's what AG2 offers:
Flexible Agent Construction and Orchestration: AG2 allows developers to create specialized agents such as Assistant agents for problem-solving, Executor agents for action-taking, and Critic agents for validation. These agents can be easily configured and orchestrated to work together through intuitive code.
Built-in Conversation Patterns: AG2 simplifies agent coordination by managing message routing, state management, and conversation flow. It supports various conversation patterns, including two-agent dialogues, group chats with dynamic speaker selection, and nested conversations for modularity.
Seamless Human-AI Collaboration: The framework facilitates human-AI interaction with configurable input modes, flexible intervention points, and optional human approval workflows. This ensures that human oversight and collaboration are seamlessly integrated into the agentic processes.
AG2 VS Microsoft Autogen
Recent developments have led to a significant split in the AutoGen community, resulting in two distinct paths: AG2 and Microsoft's AutoGen. This divergence stems from the original creators of AutoGen moving away from the official Microsoft repository to establish AG2, now at version 0.5 (7/12/2024). This move aims to maintain a community-driven approach, ensuring backward compatibility and preserving the familiar architecture that developers have come to rely on.
Meanwhile, Microsoft is charting a new course with AutoGen, maintaining version 0.2.X while working on a comprehensive rewrite for version 0.4+. This new version is expected to introduce a fresh architecture, potentially integrating with other Microsoft frameworks like Semantic Kernel, which could offer enhanced capabilities but may require significant migration efforts for existing users.
For developers, this split presents a choice between two paths. AG2 offers a stable, community-focused environment with backward compatibility, making it an attractive option for those who wish to continue using the existing framework without disruption. On the other hand, Microsoft's AutoGen promises innovation and integration with broader Microsoft ecosystems, which could be beneficial for those looking to leverage new features and capabilities.
In terms of package management, it's important to note that the existing packages pyautogen, autogen, and ag2 are all aligned with AG2, reflecting the original creator’s vision. Microsoft, however, plans to adopt a new naming convention with autogen-xxx for its future releases, signaling a clear distinction between the two paths.
Let’s Dive Deep in AG2 Framework
Auotgen provides many features and most common one used is generic multi-agent conversion framework where user can simply interact with multiple AI Agents and make complex workflows between them so they can solve tasks according to their specialized actions which are assigned to them. It is highly flexible, developers can customize the conversion pattern according to their needs by adding LLMs, Tools. This includes both autonomous and human-in-the-loop workflows, where agents can operate independently or with human oversight.
Agent Types
AutoGen has different kinds of agents, each with its own job:
AssistantAgent: This is like the main helper. It uses big language models to answer questions, plan things, and even run code. It's pretty flexible and can handle lots of different tasks.
UserProxyAgent: Think of this as the human stand-in. It acts like a user, running code and talking to other agents. It's important for tasks where human judgment is needed.
GroupChatManager: This agent is like the conversation boss. It makes sure all the agents talk in turn and share information properly, keeping everything running smoothly.
These agents can be tweaked to fit different needs, making them a solid base for building complex AI setups.
Conversation Management
AutoGen takes care of how agents talk to each other. It handles sending messages, keeping track of what's going on, and making sure conversations flow right. This means agents can focus on their jobs without worrying about how they communicate. It supports different chat styles, like two-agent talks or group chats, and keeps the conversation context so agents don't lose track of what's happening.
Built-in Tools and Integrations
AutoGen comes with handy tools and connections:
Code Execution: Agents can write and run code to solve problems, which helps them adapt to new tasks.
LLM Integration: It works with big language models like GPT-4, letting agents use the latest AI tech to come up with answers and think through tough problems.
API and Tool Use: Agents can connect to outside APIs and tools, giving them access to more data and features, so they can do even more tasks.
Swarm Orchestration with AG2
AG2 lets you set up a Swarm Chat, kinda like OpenAI’s Swarm. This setup has two main tricks:
1. Handoffs: Agents can pass tasks to each other using function calls, making the workflow smooth.
Context Variables: Agents can update shared info through function calls, keeping everything adaptable.
Instead of giving a task to just one agent, you can throw it to a bunch of them. Each agent can decide if it should pass the task to another agent. The chat ends when the last agent just gives a plain answer without suggesting more actions.
Key Parts
Create a SwarmAgent
All agents in the swarm chat should be SwarmAgent instances. They’re like AssistantAgents but with extra features for function registration and handoffs. When you make a SwarmAgent, you can give it a list of functions. These functions are turned into schemas for the LLMs, so you don’t have to worry about registering them for execution. You can also return a SwarmResult class, which lets you return a value, the next agent to call, and update context variables all at once.
Function Calls
Input Arguments: You gotta define the type of each argument, or it won’t register (e.g., arg_name: str).
Context Variables: If your function needs to access or change context variables, include context_variables: dict as an argument. This won’t be visible to the LLM but will be passed in by the swarm chat.
Docstring: This is used as the prompt, so make it clear.
Function Name: This becomes the tool name.
Registering Handoffs
You can create a function to decide the next agent, or use ON_CONDITION for quick handoff registration.
agent_1 = SwarmAgent(...)
agent_2 = SwarmAgent(...)
agent_3 = SwarmAgent(...)
# Register the handoff
agent_1.handoff(hand_to=[ON_CONDITION(agent_2, "condition_1"), ON_CONDITION(agent_3, "condition_2")])Nested Chats
You can also trigger a nested chat, which is handy for sub-tasks. Nested chats are a series of chats defined like this:
nested_chats = [
{
"recipient": my_first_agent,
"summary_method": "reflection_with_llm",
"summary_prompt": "Summarize the conversation into bullet points.",
},
{
"recipient": poetry_agent,
"message": "Write a poem about the context.",
"max_turns": 1,
"summary_method": "last_msg",
},
]AFTER_WORK
When the last agent doesn’t suggest a tool call or handoff, the chat ends by default. But you can set an AFTER_WORK handoff to define a fallback agent if you don’t want the chat to stop there.
agent_1.handoff(hand_to=[
ON_CONDITION(...),
AFTER_WORK(agent_4) # Fallback to agent_4 if no handoff is suggested
])Start a Swarm Chat
Once you’ve got your agents, start a swarm chat with initiate_swarm_chat.
chat_history, context_variables, last_active_agent = initiate_swarm_chat(
initial_agent=agent_1,
agents=[agent_1, agent_2, agent_3],
messages=[{"role": "user", "content": "Hello"}],
context_variables={"key": "value"}
)Handling Messages
Single Message: If it has a name, it’s from that agent. If not, it’s from the user agent or a temporary one.
Resume Chat: Names in messages must match the agents you passed in.
Demo with User Agent
You can include a user agent to accept user inputs. Here’s a quick setup:
from autogen import UserProxyAgent
user_agent = UserProxyAgent(name="User", code_execution_config=False)
agent_6 = SwarmAgent(
name="Agent_6",
system_message="You are Agent 6. Your job is to tell jokes.",
llm_config=llm_config,
)
agent_6.register_hand_off(
[
ON_CONDITION(agent_7, "Transfer to Agent 7"),
AFTER_WORK(AfterWorkOption.REVERT_TO_USER),
]
)
chat_result, _, _ = initiate_swarm_chat(
initial_agent=agent_6,
agents=[agent_6, agent_7],
user_agent=user_agent,
messages="start",
)This setup lets agents pass tasks around, update shared info, and even handle user inputs smoothly.
Ending Remarks:
Ag2(Autogen) have released their version 0.5 as of writing and introduced more types of agents like Reasoning Agent 2.0 but that I will write more on that in another post.
Ag2 will continue to develop rapidly as more OS Contributors are rising up in the Multi Agents field, it's not just about creating intelligent agents—it's about creating intelligent systems that work together to achieve more than any single agent could on its own.