Windows Computer Use for DSH
Lets an AI agent see and use Windows apps and web pages: click, type and read the screen
by Yu-tao-Li
agent-desktop
by Vercel LabsCLI toolGet organised
Lets your AI agent read and click through the desktop apps on your computer
About agent-desktop
A command-line tool that lets an AI agent use desktop apps on macOS, Windows and Linux. It reads each app's buttons and fields through accessibility features, then clicks or types and checks the result.
For AI agent usersPeople automating desktop appsDevelopers
Made by Vercel Labs
Vercel Labs publishes new products and experiments from Vercel, the company behind Next.js.
- Stars
- ★ 2.1k
- License
- Apache-2.0
- Cost
- Free
- Platforms
- Mac · Win · Linux
- Level
- Intermediate
Overview
agent-desktop is a command-line tool from Vercel Labs that gives an AI agent computer use on the desktop. It reads each app's real interface through the operating system's accessibility features and returns a list of buttons, fields and menus the agent can act on, so the agent can click, type, open apps and check the result without guessing from screenshots. It runs on macOS, Windows and Linux. It suits people who want their agent to finish tasks inside desktop apps.
It works from each app's own interface structure instead of screen pixels, so every step can be checked and retried.
Try these first
Start with the first one: it is a small task you can check easily.
- Use agent-desktop to list the buttons and folders in the open Finder window. Don't click anything.First try · read-only
- Open TextEdit, write a note titled [title], save it in Documents and confirm the file exists.edits files
- List the notifications on screen and summarise them for me.
- Take a screenshot of the [app name] window.
Example prompts written by Agent Flows Hub.
Features
From the creator's documentation.
| Feature | What it does | Runs |
|---|---|---|
| Snapshots | Lists an app's buttons, fields and menus, each with a reference the agent can act on | Your computer |
| Actions | Click, type, select, scroll and toggle on those elements | Your computer |
| Apps and windows | Launch, close, move, resize and switch apps and windows | Your computer |
| Clipboard and notifications | Read and set the clipboard, list and dismiss notifications | Your computer |
| Screenshots | Capture a window; on macOS this needs Screen Recording permission | Your computer |
| Built-in guides | Prints usage guides for your agent, matched to the installed version | Your computer |
Good to know
You want your agent to do a small job inside a desktop app, such as saving a note in TextEdit or checking a folder in Finder. agent-desktop lets it read the app's buttons and fields, act on one, then read the app again to confirm what changed.
Limits
- On macOS you grant Accessibility permission, plus Screen Recording for screenshots; Linux support targets Ubuntu 24.04 or newer with GNOME.
- Snapshots, screenshots and clipboard reads can contain private content from your screen. Keep them out of shared logs and uploads.
Notes by Agent Flows Hub (original project name: agent-desktop), written from the creator's documentation, not from running it ourselves. agent-desktop README · Installing agent-desktop
Stuck setting it up?
Tell us what happened. We take a look first; if it needs paid work, you get an estimate before anything starts.
More to get organised
Other cli tools
- ccusage Shows how many tokens your coding agents used and what that would cost, day by day
- Claude Code Usage Monitor Shows your Claude Code usage live in the terminal and warns you before you hit a limit
- Claude Token Optimizer Shrinks the notes Claude Code rereads at every session start, so each uses fewer tokens
- CodeBurn Shows what your AI coding tools cost, split by project, model and task
- Dev Browser Lets your coding agent drive a real Chrome browser to open pages, click and check results
- Fabric Runs ready-made AI prompts on any text, such as summarising an article into key points