Documentation

Rosply is an AI desktop agent for Windows, macOS, and Linux. You describe a task in natural language and Rosply executes it by seeing your screen and controlling the mouse and keyboard.

Under the hood Rosply takes a screenshot, sends it along with your task to a vision AI model, receives a list of actions, and executes them. This loop repeats until the task is done or you press Ctrl+H.

Rosply does not require any browser extension, accessibility APIs, or app-specific plugins. It works with any application by treating the screen as its only input.

• Open apps, navigate websites, search the web
• Read content from the screen and save it to files
• Control any application by clicking, typing, scrolling, and dragging
• Generate complete code projects in VS Code via the built-in rosply-code extension
• Listen for a wake word ("Hey Rosply") and respond to voice commands
• Remember values across steps (e.g. read a price on one page, use it in another)
• Emergency stop at any moment with Ctrl+H

1. Get an API key

Create a free account at openrouter.ai and copy your API key. The free tier gives access to several vision-capable models including Gemini Flash.

2. Download and extract

Download Rosply from Gumroad and extract the folder to a location of your choice. Then open a terminal inside the rosply folder.

3. Run setup

The setup script checks your Python version, creates a virtual environment, installs all Python and Node.js dependencies, sets up the Electron UI, and installs the VS Code extension automatically. It takes 2–5 minutes on first run.

Windows

Double-click windows-setup\setup.bat or run it from a terminal.

macOS

bash mac-linux-setup/mac.sh

Linux

bash mac-linux-setup/linux.sh

4. Add your API key

The setup script opens .env for editing automatically. Replace the placeholder with your key:

OPENROUTER_API_KEY=sk-or-...your-key-here...

5. Launch

Windows

windows-setup\start.bat

macOS / Linux

bash mac-linux-setup/start.sh

All settings live in .env. The only required field is OPENROUTER_API_KEY.

Recommended vision models

Type your task in the input box and press Enter or click the send button.

Example tasks

Open Chrome and search for the current Bitcoin price

Open Notepad and write a short poem about space

Go to youtube.com and play the first video about GSAP animations

Create a portfolio website with dark theme and GSAP animations in VS Code

Open the Downloads folder and tell me what files are there

Voice input

• Click the microphone button or press Space to start listening
• Say "Hey Rosply" at any time to wake the agent without touching the keyboard

Emergency stop

Press Ctrl+H at any time to immediately stop the current task.

Rosply also ships as a standalone Windows app. It connects to the agent already running on your machine and gives you a full chat window, a system tray icon, and auto-start at boot, so you do not need to keep a terminal or the main agent window open.

Once paired, you can also send prompts from ai.rosply.com on your phone or any other browser, and the agent will pick them up and run them on your PC.

What it looks like

The app opens to a chat view: a sidebar on the left lists your past prompts, and the center panel shows a status orb above a message box, the same kind of interface you would expect from a chat app. Typing a prompt and pressing Enter sends it straight to your agent. While a task runs you see a live "Thinking" then "Working" indicator, and each step the agent takes is logged under a collapsible "Step" toggle so the chat stays readable. A small badge in the top bar shows whether the agent is currently online.

The app also has a system tray icon so it can sit in the background between tasks, an account panel for managing devices and signing out, and a light and dark theme that follows your system by default.

Setting up the Windows app for the first time

1. 1Make sure the Rosply agent is already installed and has run at least once on this PC.
2. 2Install Rosply Server from the Microsoft Store, or get the installer from the Windows App section of the homepage.
3. 3Click Connect to Google and sign in with the same account you want to use for Rosply.
4. 4The app checks for the agent on this machine automatically. If it finds it, you are done, the chat window opens right away.
5. 5If it does not find the agent, or you prefer to pair from another device, open ai.rosply.com and sign in with the same Google account. A 6 digit code appears there.
6. 6Type that 6 digit code into the Windows app. Once accepted, the app is paired and ready to use.

Pairing an extra device

Once you have one device paired (the Windows app or a browser), adding more is the same process every time.

1. 1On the new device, open ai.rosply.com or the Windows app and sign in with the same Google account.
2. 2A 6 digit code appears on that new device.
3. 3On a device that is already paired, a prompt to approve a new device appears automatically. You can also open it manually from Account, then Approve a new device.
4. 4Type the 6 digit code shown on the new device into that prompt.
5. 5The new device is now paired and can send prompts to your agent.

Managing your devices

In the Windows app, open Account, then Devices to see every device paired to your account: what it is, and when it was last active. From there you can sign a device out, which removes its access immediately, or ban a device, which signs it out and also blocks it from ever pairing to your account again. Banned devices are listed separately and can be unbanned at any time.

When you ask Rosply to create a project or write code, it uses the rosply-code VS Code extension.

1. 1The extension receives a description of what to build from the agent
2. 2It sends the description to a coding-focused AI model (DeepSeek V3 via OpenRouter)
3. 3The model generates all project files at once
4. 4The extension writes them directly into the current VS Code workspace (or creates a new folder on the Desktop)

To update the extension after pulling new code, run windows-setup\install_extension.bat (Windows) or bash mac-linux-setup/install_extension.sh (macOS/Linux), then restart VS Code completely.

Rosply can be registered as a global MCP server so you can trigger PC automation directly from within Claude Code. After setup, you can type a task inside Claude Code and Rosply will execute it on your desktop.

Step 1 - Register the MCP server

The setup script verifies that Claude Code is installed natively, then registers Rosply as a global MCP server using the claude CLI.

Windows

windows-setup\claude-setup.bat

macOS / Linux

bash mac-linux-setup/claude-setup.sh

The script registers the MCP server at user scope so it is available in every Claude Code project.

Step 2 - Install the plugin

After restarting Claude Code, run these two commands inside it:

/plugin marketplace add harkixsha/rosply-agent-plugin
/plugin install rosply-agent@harkixsha-plugins

Step 3 - Give a command

Type any task and Claude Code sends it to Rosply via the MCP tool. Rosply executes the task on your desktop and reports back.

use the rosply skill to open Chrome and navigate to github.com

How it works

The setup script calls claude mcp add to register agent/mcp_server.py as a global MCP server. The Python server exposes a run_task tool that receives a task string and runs the Rosply agent loop on your desktop.

rosply/
  windows-setup/
    setup.bat              First-time setup (Windows)
    start.bat              Launch script (Windows)
    claude-setup.bat       Claude Code MCP registration
    install_extension.bat  VS Code extension installer
  mac-linux-setup/
    mac.sh                 First-time setup (macOS)
    linux.sh               First-time setup (Linux)
    start.sh               Launch script (macOS / Linux)
    claude-setup.sh        Claude Code MCP registration
    install_extension.sh   VS Code extension installer
    _patch_platform.py     Platform compatibility patches
  agent/
    brain.py               AI reasoning loop
    actions.py             Action implementations (click, type, scroll...)
    memory.py              Persistent key-value memory
    security.py            Blocks dangerous input and system paths
  inputs/
    vision.py              Screen capture and encoding
    stt.py                 Speech-to-text (faster-whisper)
    tts.py                 Text-to-speech (edge-tts, fallback pyttsx3)
    wake_word.py           Always-on wake word detection
  ui/
    main_window.py         Main pywebview window
    bubble.py              Floating overlay during tasks
    vignette.py            Fullscreen dark border during capture
    chats/                 Saved chat history (JSON)
  extensions/
    rosply-code/           VS Code extension for code generation
  config.py                Loads .env, exposes all settings
  main.py                  Entry point
  requirements.txt         Python dependencies
  logs/
    rosply.log             Runtime log (auto-rotated at 2 MB)