What Is an AI Agent? LLMs, Tools, Memory, Planning and MCP Explained
Updated 2026-10-15
An AI agent is a program that uses a large language model (LLM) to decide what to do next, takes actions through tools, looks at the results, and repeats until a goal is met. A chatbot answers; an agent acts.
The five core concepts
1. The LLM (the “brain”)
The model reads the conversation so far plus a list of available tools and returns either a final answer or a request to call a tool. Models from OpenAI, Anthropic, Google and open-weight models (via Ollama, vLLM, etc.) all support this “tool calling” pattern.
2. Tools (the “hands”)
A tool is a function you expose to the model with a name, a description and a JSON Schema for its arguments: send_email, query_database, search_web. The model never runs code itself. It asks your code to run the function and you send back the result.
3. Memory
- Short-term memory is the message history you resend on each call. It is limited by the model’s context window, so long conversations are trimmed or summarized.
- Long-term memory is anything you store outside the model (a database, a vector store) and retrieve when relevant.
4. Planning (the loop)
The simplest planner is a loop: call the model, run any requested tools, append the results, call the model again. Stop when the model returns a plain answer or when you hit a maximum number of steps. Frameworks like LangGraph, CrewAI and the OpenAI Agents SDK add structure (graphs, multiple agents, handoffs) on top of this same loop.
5. MCP (Model Context Protocol)
MCP is an open protocol, introduced by Anthropic in November 2024, that standardizes how agents discover and call tools. Instead of writing a custom integration for every app in every framework, a developer writes one MCP server (for example, for Postgres or Jira) and any MCP client (Claude Desktop, Cursor, VS Code, your own agent) can use it. Servers run locally over stdio or remotely over HTTP.
The agent loop in 15 lines
messages = [{"role": "user", "content": goal}]
for step in range(10): # hard cap on steps
reply = llm(messages, tools=TOOLS)
if not reply.tool_calls:
print(reply.content) # final answer
break
messages.append(reply)
for call in reply.tool_calls:
result = run_tool(call.name, call.arguments)
messages.append({"role": "tool", "tool_call_id": call.id, "content": result})Every agent framework, however fancy, is a variation of this loop.
When you should not use an agent
If the steps are fixed and known in advance (“every morning, fetch X and email it”), a plain script or workflow is cheaper, faster and more reliable. Use an agent when the next step genuinely depends on what the model finds.