How to Use Claude Computer Use: Setup, Limits & Safe Workflows

Short version: Claude computer use lets the model see a desktop through screenshots and drive it with mouse and keyboard actions, but Claude never touches the machine itself. You add one entry, {"type": "computer_toolset_20260801"}, to a Messages API request, Claude answers with actions like left_click and type, and your code runs them inside a sandbox you control. Get the sandbox, the screenshot scaling and the agent loop right, and it's a genuinely useful way to automate apps that have no API. Get them wrong and you've handed an AI a logged-in computer with no guardrails.

This guide walks through the setup from the official Docker demo to your own loop, the 17 actions Claude can take, how to size screenshots so clicks land, what it costs in tokens, the real limits, and a safety checklist. Every spec here comes from Anthropic's computer use documentation and its quickstart repository, checked on 21 October 2026. We haven't run our own benchmark of computer use, so there are no "we measured" numbers below. If you're still choosing a model for agent work, our Claude Opus 5.5 overview covers where it fits.

By the end you'll know how to run Anthropic's reference container in a few minutes, how the request and response shapes work so you can build your own agent loop, and which settings decide whether Claude clicks the right button. You'll also have two copy-ready prompts: a system prompt that makes Claude verify each step, and a task prompt template for small, well-scoped jobs.

Computer use suits jobs that are slow and repetitive for a person but don't need split-second speed: filling forms in a legacy web app, testing a desktop UI end to end, gathering information spread across several windows. Anthropic's own limitations section says latency may be too slow for interactive, human-paced work and points to background information gathering and automated software testing as better fits.

What Claude Computer Use Is (and Isn't)

The computer use tool is what Anthropic calls a client toolset. One entry in your tools array gives Claude 17 member tools, and your application runs every one of them. Claude sees only the screenshots you send back. It has no direct connection to the desktop, the network or your files.

Three things it is not:

• Not a hosted computer. Anthropic doesn't provide the machine. You run a virtual machine or container, and your code turns each action into a real click or keystroke.
• Not the browser use tool. Anthropic has a separate browser use tool that reads and acts on webpages directly. The docs recommend it when the task stays inside webpages, and computer use when you need a whole desktop. You can declare both in one request; they keep separate coordinate frames and are told apart by toolset_name.
• Not available in Claude Managed Agents. The docs state the toolset isn't currently offered there, so this is a Messages API feature you wire up yourself.

Which models and platforms support it

The current toolset is generally available on the Claude API and Google Cloud, and needs no beta header. The supported models listed for computer_toolset_20260801 are:

Two catches. On the Claude API and Google Cloud, the 5.5 models support computer use only through the toolset and return an error for the older computer_20251124 and computer_20250124 versions. And Amazon Bedrock, Microsoft Foundry and Claude Platform on AWS currently offer only the earlier beta versions, which need a beta header (computer-use-2025-11-24) on every request. Older models such as Claude Opus 4.7, Opus 4.6 and Sonnet 4.6 also stay on computer_20251124. For how Opus 5.5 stacks up against Google's flagship on agent work, see our Gemini 4 Argon vs Claude Opus 5.5 comparison.

How Claude Computer Use Works

Everything runs as a loop. You send a task plus the toolset. Claude decides it needs to act and replies with one or more tool_use blocks, each naming a member such as screenshot or left_click and carrying "toolset_name": "computer". The response's stop_reason is tool_use. Your code runs the actions, sends back one tool_result per block, and Claude looks at the new screenshot to decide what's next. Anthropic calls the repetition of those steps without user input the agent loop.

Claude often plans a short sequence, such as click a search box, type a query, then take a screenshot, and returns it in one response. That's a batch action. It looks like parallel tool use, with one big difference: you must run the blocks in order, because the typing depends on the click that came before it.

Pro tip: If one action in a batch fails, stop there. Return its error with is_error: true, and answer every later block with is_error: true and the exact text "Not executed: an earlier computer action in this turn failed." Leaving any block unanswered gets the whole request rejected with an invalid_request_error.

Step 1: Run Anthropic's Reference Container

The fastest way to see computer use working is the computer-use-demo in anthropic-quickstarts. It packages everything the docs say you need: a virtual X11 display (Xvfb), a lightweight desktop with the Mutter window manager and Tint2 panel, preinstalled apps such as Firefox and LibreOffice, the tool implementations, and the agent loop. All of it runs inside one Docker container.

With Docker installed and your API key exported, the README's command is:

export ANTHROPIC_API_KEY=your_api_key
docker run \
    -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
    -v $HOME/.anthropic:/home/computeruse/.anthropic \
    -p 5900:5900 \
    -p 8501:8501 \
    -p 6080:6080 \
    -p 8080:8080 \
    -it ghcr.io/anthropics/anthropic-quickstarts:computer-use-demo-latest

Then open http://localhost:8080 for the combined chat and desktop view. The README also lists the Streamlit chat alone on port 8501, the desktop alone at http://localhost:6080/vnc.html, and a direct VNC connection on vnc://localhost:5900. Set the WIDTH and HEIGHT environment variables to change the screen size, though the README advises against sending screenshots above XGA/WXGA resolution.

Give it a harmless first task, such as "Open Firefox and search for today's weather in Delhi," and watch the desktop pane. You'll see the loop in action: a screenshot, a click, typing, another screenshot. That five-minute run teaches you more about pacing and failure modes than any diagram.

Step 2: Make Your First API Request

Once you want your own app, you call the Messages API directly. The docs' quick start declares the computer toolset next to the bash and text editor tools, because Claude typically uses them together:

curl https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-5-5",
    "max_tokens": 1024,
    "tools": [
      {"type": "computer_toolset_20260801"},
      {"type": "text_editor_20250728", "name": "str_replace_based_edit_tool"},
      {"type": "bash_20250124", "name": "bash"}
    ],
    "messages": [
      {"role": "user", "content": "Save a picture of a cat to my desktop."}
    ]
  }'

Notice what's missing. The toolset entry has no name, no display width or height, and no beta header. The docs say the entry rejects name, display_width_px, display_height_px, display_number and enable_zoom with an invalid_request_error, because coordinates always live in the pixel space of the screenshots you return. You also can't declare the toolset in the same request as a computer_20251124 entry or any other tool named computer.

The entry accepts only four fields:

• type: always computer_toolset_20260801.
• configs: per-member settings, such as {"zoom": {"enabled": false}} to withhold zoom if your environment can't produce zoom images.
• cache_control: a prompt caching breakpoint on the toolset definition.
• allowed_callers: ["direct"] only.

If your loop can only run one action per round trip, set disable_parallel_tool_use to true in tool_choice and Claude returns at most one action per turn.

Step 3: Learn the 17 Actions Claude Can Take

Each action is its own member tool. Claude names it in the tool_use block, and the input holds only that action's parameters, with no action field. Here's the full set, grouped the way you'll implement them:

Only screenshot and zoom need an image in their result. For every other action a short text such as OK is enough. Every result must echo "toolset_name": "computer"; a result that omits it is rejected. All 17 members are on by default, including zoom, so if you don't implement one, turn it off in configs rather than returning errors.

Step 4: Build the Agent Loop

The docs' own loop is short. It calls the API, appends Claude's reply, runs the requested actions, and sends the results back until Claude stops asking for tools or an iteration cap is hit:

def sampling_loop(model, messages, max_iterations=10):
    for _ in range(max_iterations):
        response = client.messages.create(
            model=model,
            max_tokens=4096,
            messages=messages,
            tools=[{"type": "computer_toolset_20260801"}],
        )
        messages.append({"role": "assistant", "content": response.content})

        tool_results = process_tool_calls(response)   # run blocks in order
        if not tool_results:
            return messages                           # task complete

        messages.append({"role": "user", "content": tool_results})
    return messages

The real work is process_tool_calls. Three rules from the docs keep it correct:

1. Dispatch on the pair (toolset_name, name), not on name alone. A custom tool in the same request can share a member's name, and later toolset versions can add members.
2. Run blocks sequentially and stop at the first failure, answering the skipped ones with the halt text from earlier.
3. Put any human-approval check before each block runs, not per turn. A batch can finish a multi-step action, like filling and submitting a form, inside one response.

Keep the iteration cap. It's your brake against a loop that keeps clicking the same wrong element. The docs' sample uses 10, which is a sensible starting point for small tasks; raise it only once you trust the task.

Pro tip: Claude usually ends a batch with a screenshot. When it doesn't, the docs suggest attaching a fresh screenshot to the last result yourself. That saves a round trip and means Claude always sees the current state of the screen.

If you'd rather not write the loop at all, the docs point to SDK toolset classes in the Python and TypeScript SDKs (in beta) that run the loop and an approval callback you pass in, plus partner integrations you can start from. If you'd rather have Claude draft the dispatcher for you, our Claude Opus 5.5 coding prompts are a good starting point.

Step 5: Size Screenshots So Clicks Land

This is where most broken setups fail. The toolset takes no display dimensions and the API doesn't downscale for you, so every screenshot must already fit the model's image limits or the request is rejected. Claude then returns coordinates in the pixel space of the image it saw, so if you shrank the screenshot, you scale its coordinates back up before clicking.

The limits from the docs:

• Every toolset model (Claude Opus 4.7 and later): up to 2,576 pixels on the long edge and 4,784 visual tokens, roughly 3.75 megapixels.
• Earlier models: up to 1,568 pixels on the long edge and about 1.15 megapixels.
• Many images in one request: once a request carries more than 20 images, each is held to a stricter per-side limit, so keep screenshots at 2,000 px or less per side in long loops.

When clicks miss, the docs' troubleshooting table maps symptoms to causes:

Step 6: Prompt Claude to Check Its Own Work

Anthropic is candid that Claude sometimes assumes an action worked without looking. The fix is a system prompt that forces verification. This one combines the docs' recommended wording with their advice to end batches on a screenshot and prefer the keyboard for tricky controls:

Then keep each task small and explicit. The docs' first prompting tip is to specify simple, well-defined tasks with explicit instructions for each step, and to include example screenshots for repeatable jobs. A template that follows that advice:

One more ordering tip from the docs: when you build a user turn that contains both instructions and a screenshot, put the text before the image. Describing the target first improves click accuracy.

Safe Workflows: The Four-Layer Setup

Computer use carries risks ordinary API calls don't, and Anthropic says those risks grow once the agent can reach the internet. Its documentation lists four precautions, and they're the minimum we'd put in place before pointing an agent at anything real.

1. Sandbox VM. Use a dedicated virtual machine or container with minimal privileges, so an accident or attack can't reach your real system. The reference demo already does this with Docker.
2. Domain allowlist. Limit internet access to the domains the task needs, which shrinks the amount of hostile content Claude can see.
3. No secrets. Don't give the model sensitive data such as account logins. If a task truly needs a login, the docs suggest passing it inside XML tags like <robot_credentials>, and warn that apps requiring login raise the risk of prompt injection. A throwaway test account beats your real one every time.
4. Human confirm. Ask a person to approve anything with meaningful real-world consequences or anything that needs consent: accepting cookies, financial transactions, agreeing to terms of service.

Prompt injection is the risk that matters most

The docs are direct about it: in some circumstances Claude will follow commands found in on-screen content even when they conflict with your instructions, including text inside webpages or images. Anthropic has trained the model to resist this and runs classifiers over what the tools return, screenshots included. When a classifier flags a likely injection, it steers Claude to check whether the instruction really came from you. You can ask support to turn that off for use cases with no human in the loop, but the precautions above still apply either way. Anthropic also asks that you tell end users about the risks and get their consent before enabling computer use in your own product.

On data, computer use is a client-side tool. Screenshots, mouse actions, keystrokes and files stay in your environment rather than being stored by Anthropic, and the docs list it as eligible for zero data retention. That makes your own logs the audit trail, so keep them.

What Computer Use Costs

There's no separate fee. Computer use follows standard tool use pricing, so you pay your model's normal input and output rates on top of a few fixed overheads the docs spell out:

Screenshots are the cost that grows. A loop that keeps every screenshot gets more expensive each turn, which is why prompt caching matters here. The docs suggest one cache breakpoint after the system prompt and tools, plus up to three on the latest tool results, advanced each turn. Our guide to Claude prompt caching explains how breakpoints work, and for the model rates themselves see our Claude API pricing breakdown.

For pruning old screenshots, the docs' general advice is to prune in batches, not every turn, for example keeping the last three and pruning every 25 turns, so the cached prefix stays identical between prunes. But on Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5 they say to avoid pruning on the client, because removing an earlier screenshot invalidates later thinking blocks. On those models, keep screenshots at 2,000 px or less per side and use server-side tool result clearing instead.

Limits You Should Plan Around

Anthropic's limitations list is worth reading in full before you promise anyone an automation. The ones that shape real projects:

• Latency. It can be too slow for human-paced interaction. Plan for background jobs.
• Coordinate mistakes. Claude can misjudge where to click. Turning on summarized thinking (display: "summarized") shows its reasoning, since toolset models omit thinking text by default.
• Tool selection. Reliability can drop in niche apps or when juggling several apps at once.
• Scrolling. Where scroll has no effect, keyboard alternatives such as Page Down help.
• Spreadsheets. Selecting cells works best with left_mouse_down, left_mouse_up and modifier keys, and complex operations may take several tries.
• Social platforms. Its ability to create accounts or post content on social and communication sites is limited by design.

The docs close that list with a plain instruction: always review and verify Claude's actions and logs, and don't use it for tasks that need perfect precision or sensitive user information without human oversight. If you're comparing this against an agent that works through code and files instead of pixels, our Claude Code vs Google Antigravity comparison covers that side of the field.

Migrating From computer_20251124

If you already run the older tool, moving to the toolset is mostly a shape change, and on the Claude API it's required for the 5.5 models. Per the docs:

• Replace the tools entry. Drop name and the display_* fields, and drop the computer-use-2025-11-24 beta header.
• The action moves from input.action to the block's name, and every block gains "toolset_name": "computer".
• Zoom is on by default now. To match an old setup that never enabled zoom, add "configs": {"zoom": {"enabled": false}}.
• Return toolset_name on every result and handle batches in order.

Common Pitfalls (and the Fixes)

Your Computer Use Tool Stack

If your agent mostly needs to read web content rather than click through it, scraping the page into clean text is usually cheaper and more reliable than sending screenshots, and our context engineering guide explains why cleaner input makes agents behave better.

The Verdict

My take: Claude computer use is the right tool when there's no API, no clean HTML and no way around a real desktop app, and it's the wrong tool for anything a script or the browser use tool can already do. The setup is less about the model and more about your harness. Run it in a sandbox, size screenshots to the limits, answer every block in order, approve consequential steps one at a time, and cache aggressively. Start with the reference container and a five-step task, then grow from there. For scheduled background jobs that don't need a desktop at all, Claude Routines may be the simpler fit.

Frequently Asked Questions

Keep Reading

• How to use Google Antigravity with Gemini 4 Argon
• Gemini 4 Argon explained: long decode, thinking levels and specs
• Gemini implicit caching vs Claude explicit caching