Language: English | 简体中文 | 日本語
OpenGUI lets AI operate real Android phones.
You can use the same repository in four practical ways:
skills/open-gui-bootstrap/SKILL.md, describe the goal in plain language, and let it handle setup, build, install, and local debugging.Plan Supervisor maintains task state and continuation, Executor Graph runs screenshot, vision, action, and call-user loops on top of live device state, and Summarizer closes the run with a structured result.OpenGUI is built as a mobile operator system with explicit orchestration layers.
The source code currently exposes these pieces:
server/apps/backend/src/modules/graph-agent/graph/mobile-agent.graph.ts for the main graphserver/apps/backend/src/modules/graph-agent/graph/executor.graph.ts for the device-side execution loopserver/apps/backend/src/common/ws/standby.gateway.ts for standby device dispatchclient/core_network/.../StandbySocketManager.kt for persistent device standby connectionsclient/core_accessibility/.../GestureService.kt for Android-side action execution| Dimension | Typical phone-agent demo | OpenGUI |
|---|---|---|
| Execution model | Short interactive loop | Main graph plus executor subgraph |
| Task state | Usually local and session-bound | Task state managed in the backend graph |
| Device path | Often laptop-driven control | Android client with standby and execution sockets |
| Model usage | One model does most of the work | Planning and VLM paths can be split across providers |
| Remote operation | Optional add-on | Feishu, Telegram, REST API, and standby dispatch are built into the backend |
Start with skills/open-gui-bootstrap/SKILL.md.
The intended flow is simple:
It should only stop for:
Recommended profiles:
Use the latest Claude Opus model family across planning, supervision, review, and vision when you want the strongest overall quality.
This is the easiest way to get the best execution quality, and it is the most expensive path.
Use Qwen 3.6 Plus for text-side roles such as Planner and Supervisor, and use Doubao Pro for the VLM side.
This usually preserves the overall system shape while lowering model cost by roughly 10x to 15x compared with an all-Opus setup, depending on task length, screenshot volume, and token mix.
Recommended prompts:
Read ./skills/open-gui-bootstrap/SKILL.md and help me run OpenGUI. Only ask me for phone-side actions.
Read ./skills/open-gui-bootstrap/SKILL.md and bootstrap OpenGUI with the latest Claude Opus model family for planning, supervision, review, and vision.
Read ./skills/open-gui-bootstrap/SKILL.md and set up OpenGUI with Qwen 3.6 Plus for Planner and Supervisor, and Doubao Pro for VLM execution.
Read ./skills/open-gui-bootstrap/SKILL.md and use my existing model APIs to get OpenGUI working.
Use the repository scripts directly:
cd server
./start.sh
cd client
./start.sh
Reference docs:
flowchart LR
U["User or IM command"] --> BS["Bootstrap Skill / API / IM entry"]
BS --> SP["Plan Supervisor"]
SP --> EX["Executor Graph"]
EX --> AC["Android Client"]
AC --> GX["AccessibilityService + screenshots + actions"]
EX --> RV["Execution review and retry"]
RV --> SP
SP --> SM["Summarizer"]
SM --> SR["Structured Results"]
RD["Feishu / Telegram / REST API"] --> ST["Standby Gateway"]
ST --> AC
SP --> MR["Model Routing"]
MR --> MA["Claude / GPT / Gemini / Kimi / MiniMax / compatible"]
EX --> MR
server/apps/backend/src/modules/graph-agent/graph/server/apps/backend/src/modules/task/task.controller.tsserver/apps/backend/src/common/ws/standby.gateway.tsclient/core_network/src/main/java/com/coremate/opengui/network/websocket/StandbySocketManager.ktclient/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.ktIf OpenGUI is useful to you, the most helpful ways to support it are:
OpenGUI is source-available under the Business Source License 1.1 (BUSL-1.1).
You may copy, modify, distribute, and use the source for non-production purposes. Production use, commercial use, hosted services, and integration into commercial products require a separate commercial license from Core-Mate.
For this version:
This is public source, but it is not OSI-approved open source until the Change Date.
See LICENSE.