Three ways into one Android phone

Aster is the Android companion that lets an AI agent see and operate a phone. The Aster case study covers what it does, and Giving AI an Android phone covers why the permission boundary matters. This piece covers the three transports a client can use to reach the phone, and what stays the same underneath.

One handler map, three ways in

Inside the app, every capability is a CommandHandler. A handler declares the action names it supports and returns a CommandResult with a success flag, optional JSON data and an optional error. At start-up the app builds one map from action name to handler. Each transport is a ConnectionMode that receives a request, looks the action up in that map and sends the result back in its own format.

That shared map is why a fix to tap or observe lands everywhere at once. It is also where the safety checks attach. Every transport also runs parameters through the same normaliser first, so a handler finds snapshot_id whether the caller wrote it in snake_case or camelCase.

Remote WebSocket: the phone dials out

The default mode keeps the phone as a client. The Android app opens a WebSocket to the aster Node.js server on port 5987. An MCP client such as Claude talks to that server over HTTP at :5988/mcp, and a web dashboard runs on :5989. The server turns MCP tool calls into commands on the socket and waits for the matching response.

Dialling out means the phone needs no open port, and one server can broker several devices. The trust model is a status rather than a secret. A new device connects, reports a hashed Android ID, its name and model, and lands as pending. Every command for a device that is not approved is refused. The owner approves or rejects it from the dashboard or the CLI, and a rejected device is disconnected with close code 4003.

With no shared secret, anything on the network that knows the port can register as pending, and the README says so. The Node ws server also does not terminate TLS; for remote use, TLS comes from Tailscale Serve or a reverse proxy.

Local MCP: a server inside the app

Some people do not want a second machine at all. In Local MCP mode the app hosts its own MCP server: embedded Ktor on the CIO engine, with the MCP Kotlin SDK’s Streamable HTTP transport, listening on port 8080 by default. The routes are small: POST /mcp for requests and DELETE /mcp to end a session. Responses come back as plain JSON rather than an SSE stream.

The tool list is generated from the same handler map. Each action gets a description and a JSON schema for its parameters. By default every declared parameter is required, with explicit exceptions where that would be wrong. tap accepts either an element ref or an x and y pair, so it marks nothing as required and the handler checks the combination at runtime.

This path has no device-approval gate and no token. It is meant for a trusted network or a private one such as Tailscale.

Binder IPC: two apps on one phone

The third path never touches the network. Another app on the same device, in practice OpenAlly, binds to Aster’s IPC service and calls it through an AIDL interface. Binder reports the caller’s Linux UID with every call, and that UID is the identity Aster tracks after authentication.

Approval and the token

The calling app first opens an approval screen inside Aster that shows the caller’s name and icon. If the owner approves, Aster generates a 32-character hex token with SecureRandom and returns it as the activity result. The client presents that token to authenticate(), which compares it in constant time and records the calling UID.

The comment beside one guard in that method explains a subtle case. The service is exported without an Android permission, and before any session starts the stored token is an empty string. Comparing an empty token with an empty token would succeed. authenticate() now refuses outright until a real token has been issued.

An interface that only grows

Aster and OpenAlly are installed and updated separately, so they can run different versions. AIDL transaction codes are positional: a method’s number comes from where it is declared. Inserting a method in the middle would quietly retarget every later call across a version mismatch, so new methods only ever go at the end.

interface IAsterService {
    String authenticate(String token);
    String executeCommand(String action, String paramsJson);
    ParcelFileDescriptor readLargeResult(String resultId);
    void registerCallback(IAsterCallback callback);
    // later additions are appended here, never inserted above
}

That excerpt is simplified; the real interface has more methods.

Large results take a pipe

Binder transactions have a limited buffer, and a screenshot or a long file listing can exceed it. When a serialised result is larger than about 500 KB, Aster keeps it and returns a small {"_largeResult": "<id>"} placeholder. The client then calls readLargeResult, which streams the bytes through a ParcelFileDescriptor pipe and clears the stored copy.

Observation is a snapshot

All three paths reach the same screen model. observe walks the accessibility tree window by window and returns a flat list of elements. Each has a short ref such as e12, a role, text, bounds, state, and any actions not already implied by that state. The whole result carries a snapshot_id.

Two details came from watching agents fail. A clickable list row often has no text of its own; its label lives in child views that the filter drops. Aster now gathers descendant text into a label field so the model can name the row. A large app window also used to exhaust the element budget before the navigation bar was read, so the agent never saw the tab it needed. The budget is now split into one bucket for application windows and another for system windows.

The app keeps the last three snapshots. It never stores live AccessibilityNodeInfo objects, which Android recycles, only a descriptor per ref with the view ID, text, content description, window and bounds. When the snapshot holding a ref has been evicted, an action on that ref fails with stale_ref and a hint to observe again.

Acting, then reporting what changed

When an agent taps a ref, Aster re-resolves it to a fresh node: by view ID, then by text and role within the same window, then by the node nearest the recorded bounds. A blind tap at the old coordinates is off unless the caller opts in, because it could report success on a screen that had moved.

The result describes the outcome rather than the attempt. It includes ok, the strategy that resolved the ref, changed, the foreground package afterwards and whether the screen settled. changed is true when the acted window’s content changed, when the set of windows changed, or when the foreground app changed. If none of those moved, it reports false.

Guards below every transport

Safety checks sit under the transports. Every handler is wrapped once, when the map is built, by a decorator that runs the package policy before the real handler. The policy is fail-closed: if the foreground app cannot be identified, control actions are refused, and a bundled banking and payments denylist blocks control of those apps by default. Reads such as observe and screenshots are still allowed. launch_intent is checked against the app it is about to open, not whatever happens to be in front.

The kill switch covers every mode too. While an agent controls the screen, a persistent notification offers STOP. In IPC mode, pressing it broadcasts a stop event, ends authenticated sessions and rejects control actions until the client reconnects with the current token.

Different catalogues on purpose

The transports do not expose identical tool lists. Through the Node server, an MCP client sees 49 tools with an aster_ prefix. The on-device paths read the handler map directly and expose a different, larger set of unprefixed actions; the README counts 78, including the ref-based verbs such as observe, tap and scroll. One server tool, aster_list_devices, has no on-device equivalent because it brokers between phones.

I document the catalogues separately rather than forcing them into one list, so the person setting up a client can see which path, and which trust model, they chose.