Your robot.
Within reach.

A web and native interface for controlling and visualizing ROS 2 robots.

Robo-Boy combines camera streaming, configurable control pads, 3D visualization, behavior-tree editing, TF inspection, and custom themes in one touch-friendly workspace. The web, desktop, and mobile apps share the same React codebase; the native packages are thin shells and do not bundle ROS.

Two routes: a native app, or the browser stack.

The native app is the shorter route. It needs only the ROS services, so there is no certificate to create and no proxy to run. Choose the browser stack when Robo-Boy should open directly from a browser.

Native app and ROS stack

1. 1

   Install an app

   Linux.deb

   Linux.rpm

   macOS.dmg

   Windows.exe

   Android.apk

   iOS.ipa

   These stable links advance with each Robo-Boy application release. The macOS image is the Electron build for Apple silicon, the Linux packages are x86-64, and the iOS package is unsigned and must be signed before installation.

2. 2

   Start the ROS services only

   git clone git@github.com:tessel-la/robo-boy.git
   cd robo-boy
   docker compose up -d --build ros-stack

   This one service supplies rosbridge, rosapi, and web_video_server. The web frontend, proxy, and certificate are not started, because the native app talks to these ports directly.

   If the robot publishes message, service, or action types of its own, mount that workspace as well, or those topics arrive without a definition to read them with. See robot-specific interfaces.

3. 3

   Open the app and connect

   Use Quick Connect when ROS runs on the same computer. When it runs elsewhere, open the advanced options, choose Host or IP, and enter that computer's hostname, LAN address, or VPN name.

   docker compose logs -f ros-stack
   docker compose down

Robo-Boy never starts, stops, or bundles ROS or the media gateway. Run those services on the robot or simulator computer instead of locally and only step 2 changes.

Browser, with the full Docker stack

Serving Robo-Boy to a browser adds two requirements the native apps do not have: a browser needs HTTPS before it will grant a page a camera and other secure-context features, and the app and the robot services have to appear under one origin. Caddy supplies both, and a locally trusted certificate is what makes the first one work without warnings. You need Docker with Compose and mkcert.

1. 1

   Clone and create a trusted local certificate

   git clone git@github.com:tessel-la/robo-boy.git
   cd robo-boy
   mkcert -install
   mkdir -p infra/caddy/certs
   mkcert \
     -key-file infra/caddy/certs/local-key.pem \
     -cert-file infra/caddy/certs/local-cert.pem \
     localhost 127.0.0.1 ::1 YOUR_HOST_IP

   Include the computer's LAN or VPN address as YOUR_HOST_IP when another device will open Robo-Boy.

2. 2

   Start the stack

   docker compose up -d --build

   Open https://localhost, or https://YOUR_HOST_IP from a device on the same network. Every ROS and DDS setting has a default, so no environment file is needed for this configuration; add one only to select a robot workspace overlay.

3. 3

   Stop or inspect the stack

   docker compose logs -f app
   docker compose logs -f caddy
   docker compose logs -f ros-stack
   docker compose down

npm ci
npm run dev
# http://localhost:5173

docker compose up -d --build ros-stack
npm ci
npm run desktop:dev

Robo-Boy talks to ROS through rosbridge.

The ROS stack supplies rosapi, rosbridge, and web_video_server. Robo-Boy discovers topics and interfaces after the shared ROSLIB WebSocket connects.

Default service contract

On the web, Quick Connect normally resolves these through same-origin /websocket, /video_stream, /mesh_resources, and /ollama routes. Advanced Ports override the first three service ports. Tauri connects directly.

DDS and robot-specific interfaces

The ros-stack service uses host networking for DDS discovery. Every container that must discover the same graph needs compatible ROS_DOMAIN_ID, ROS_LOCALHOST_ONLY, and RMW_IMPLEMENTATION settings.

Robo-Boy reads a topic through the interface definition that describes it, and it only has the definitions present inside ros-stack. A robot with message, service, or action types of its own therefore needs its built workspace mounted there, or those topics appear in the graph and cannot be decoded. Mount the workspace's install directory under a unique path in a Compose override:

services:
  ros-stack:
    volumes:
      - /path/to/robot_ws/install:/overlay_ws/robot:ro

COMPOSE_FILE=docker-compose.yml:infra/compose/robot.yml docker compose up -d --build

Overrides for the bundled Aerostack and manipulator workspaces already exist under infra/compose/. A workspace built with colcon --symlink-install also needs its build and source directories mounted, since its installed files point back at them.

ROS_DOMAIN_ID=0
ROS_AUTOMATIC_DISCOVERY_RANGE=SUBNET
ROS_STATIC_PEERS=
RMW_IMPLEMENTATION=rmw_fastrtps_cpp
FASTDDS_BUILTIN_TRANSPORTS=UDPv4

A robot workspace overlay is selected by pointing Compose at one of the bundled override files, either in .env or on the command line. After changing overlays, recreate the service:

COMPOSE_FILE=docker-compose.yml:infra/compose/aerostack.yml
# or infra/compose/manipulator.yml

docker compose up -d --build --force-recreate ros-stack

A frontend-first system with a narrow ROS boundary.

Robo-Boy has no application server or database. The browser or Tauri webview owns the interface, ROSLIB owns the active bridge connection, and browser storage holds serializable local state.

The workspace is a set of tiles backed by stable panel IDs.

Built-in and compatible external panels are merged into one catalog. The workspace stores panel instances and layout locally; an external tile retains its reverse-domain panel ID so Robo-Boy can show an actionable unavailable state if the deployment later removes that panel.

Everyday workflows

Independent code. Small contract. Trusted boundary.

Panel SDK 2.2.0 is a type-only, additive update to the Panel API 2.0 contract. A panel exports one framework-neutral definition, runs in its own opaque-origin iframe, owns everything below the supplied DOM element, and receives only declared host services over a private channel. The release adds the webxr capability, which delegates XR spatial tracking and fullscreen for an immersive session, on top of the dedicated WHEP, discovery, and HLS host endpoints and binary arrayBuffer() responses introduced in 2.1.0; Robo-Boy does not scan node_modules or import panel source into its application build.

The host owns tile creation, sizing, ordering, saved workspace layouts, and replacement. The panel owns only DOM below its mount element; v2 exposes no parent-page or layout-mutation API. Store panel configuration through context.storage, and keep framework/runtime dependencies inside the panel's own bundle.

Capability declarations

Permission declarations

{
  "capabilities": ["ros", "network"],
  "permissions": {
    "ros": {
      "selectTopic": true,
      "publish": ["/cmd_vel"],
      "services": ["/robot/reset"]
    },
    "network": {
      "origins": ["https://telemetry.example.com"],
      "hostEndpoints": ["webrtcWhep", "webrtcDiscovery", "webrtcHls"]
    }
  }
}

ROS and network capabilities require matching permission blocks. Static topic/service patterns are checked for every operation. selectTopic opens a trusted Robo-Boy picker and grants only the chosen topic to that tile; panels never receive the unfiltered graph. Brokered network requests omit ambient browser credentials, revalidate redirects, and can return text, JSON, or binary bytes. videoStream remains available for older panels; new stream panels should request the dedicated gateway endpoints.

One imported module can create many isolated tile instances.

setActive(isActive) is optional; mount and unmount are mandatory. Effective active state combines workspace activity, tile intersection, and document visibility. Removing the iframe terminates the panel realm even if cleanup fails.

The complete runtime surface fits in a few interfaces.

interface RoboBoyPanelDefinition {
  apiVersion: string;
  id: string;
  activate(
    context: RoboBoyPanelContext
  ): RoboBoyPanelInstance | Promise<RoboBoyPanelInstance>;
}

interface RoboBoyPanelInstance {
  mount(container: HTMLElement): void | Promise<void>;
  setActive?(isActive: boolean): void | Promise<void>;
  unmount(): void | Promise<void>;
}

interface RoboBoyPanelContext {
  readonly panelId: string;
  readonly instanceId: string;
  readonly capabilities: readonly RoboBoyPanelCapability[];
  readonly ros: RoboBoyPanelRos | null;
  readonly storage: RoboBoyPanelStorage | null;
  readonly network: RoboBoyPanelNetwork | null;
  readonly runtime: RoboBoyPanelRuntime;
  readonly connection: RoboBoyPanelConnection;
  readonly viewport: RoboBoyPanelViewport;
  readonly theme: RoboBoyPanelTheme;
  readonly logger: RoboBoyPanelLogger;
}

interface RoboBoyPanelRuntime {
  readonly target: 'web' | 'desktop';
}

interface RoboBoyPanelConnectionSnapshot {
  readonly status: 'disconnected' | 'connecting' | 'connected';
  readonly generation: number;
}

interface RoboBoyPanelConnection {
  getSnapshot(): RoboBoyPanelConnectionSnapshot;
  subscribe(
    listener: (snapshot: RoboBoyPanelConnectionSnapshot) => void
  ): () => void;
}

interface RoboBoyPanelViewportSnapshot {
  readonly width: number;
  readonly height: number;
  readonly isIntersecting: boolean;
  readonly isDocumentVisible: boolean;
  readonly isActive: boolean;
}

interface RoboBoyPanelViewport {
  getSnapshot(): RoboBoyPanelViewportSnapshot;
  subscribe(
    listener: (snapshot: RoboBoyPanelViewportSnapshot) => void
  ): () => void;
  requestFullscreen(): Promise<void>;
}

interface RoboBoyPanelLogger {
  debug(message: string, ...details: unknown[]): void;
  info(message: string, ...details: unknown[]): void;
  warn(message: string, ...details: unknown[]): void;
  error(message: string, ...details: unknown[]): void;
}

interface RoboBoyPanelRos {
  getTopics(): Promise<RoboBoyRosTopic[]>;
  selectTopic(options?: { currentTopic?: string }): Promise<RoboBoyRosTopic>;
  subscribe(
    options: RoboBoyRosSubscriptionOptions,
    listener: (message: RoboBoyJsonObject) => void
  ): Promise<RoboBoyRosSubscription>;
  publish(options: RoboBoyRosPublishOptions): Promise<void>;
  callService(options: RoboBoyRosServiceOptions): Promise<RoboBoyJsonObject>;
}

type RoboBoyHostEndpoint =
  | 'videoStream'
  | 'webrtcWhep'
  | 'webrtcDiscovery'
  | 'webrtcHls';

interface RoboBoyPanelNetworkResponse {
  readonly ok: boolean;
  readonly status: number;
  readonly statusText: string;
  readonly headers: { get(name: string): string | null };
  text(): Promise<string>;
  json<T extends RoboBoyJsonValue = RoboBoyJsonValue>(): Promise<T>;
  arrayBuffer(): Promise<ArrayBuffer>;
}

interface RoboBoyPanelNetwork {
  readonly endpoints: Partial<Record<RoboBoyHostEndpoint, string>>;
  fetch(
    url: string,
    request?: RoboBoyPanelNetworkRequest
  ): Promise<RoboBoyPanelNetworkResponse>;
}

interface RoboBoyPanelThemeSnapshot {
  readonly colorScheme: 'light' | 'dark';
  readonly tokens: Readonly<
    Partial<Record<RoboBoyPanelThemeToken, string>>
  >;
}

interface RoboBoyPanelTheme {
  getSnapshot(): RoboBoyPanelThemeSnapshot;
  subscribe(
    listener: (snapshot: RoboBoyPanelThemeSnapshot) => void
  ): () => void;
}

interface RoboBoyPanelStorage {
  readonly schemaVersion: 1;
  readonly quotaBytes: number;
  get<T extends RoboBoyJsonValue>(key: string, fallback: T): T;
  set(key: string, value: RoboBoyJsonValue): void;
  remove(key: string): void;
  sizeBytes(): number;
}

Build a persistent greeting panel from scratch.

This minimal standalone panel uses the v2 sandbox lifecycle and brokered storage API. It has no runtime framework dependency; TypeScript emits a browser-ready ESM file.

1. 1

   Create a sibling repository

   mkdir my-roboboy-panel
   cd my-roboboy-panel
   mkdir -p src scripts

   For local staging, keep it next to robo-boy/. A local desired-state source points at the repository explicitly; it does not scan sibling projects.

2. 2

   Add package.json

   {
     "name": "my-roboboy-panel",
     "version": "0.1.0",
     "type": "module",
     "scripts": {
       "build": "tsc -p tsconfig.json",
       "typecheck": "tsc -p tsconfig.json --noEmit",
       "integrity": "node scripts/print-integrity.mjs"
     },
     "devDependencies": {
       "@tessel-la/roboboy-panel-sdk": "https://github.com/tessel-la/robo-boy/releases/download/panel-sdk-v2.2.0/tessel-la-roboboy-panel-sdk-2.2.0.tgz",
       "typescript": "^5.2.2"
     }
   }

   npm install

   The SDK is distributed as a release tarball, not through the npm registry. Pin it and commit the lockfile.

3. 3

   Configure TypeScript

   {
     "compilerOptions": {
       "target": "ES2020",
       "module": "ES2020",
       "moduleResolution": "bundler",
       "lib": ["ES2020", "DOM"],
       "strict": true,
       "declaration": false,
       "outDir": "dist",
       "skipLibCheck": true
     },
     "include": ["src"]
   }

4. 4

   Implement src/index.ts

   import type {
     RoboBoyPanelContext,
     RoboBoyPanelDefinition,
   } from '@tessel-la/roboboy-panel-sdk';
   
   const activate = (context: RoboBoyPanelContext) => {
     let root: HTMLElement | null = null;
     let button: HTMLButtonElement | null = null;
     let count = context.storage?.get('greetings', 0) ?? 0;
   
     const renderCount = () => {
       if (button) button.textContent = `Send greeting (${count})`;
     };
   
     const sendGreeting = () => {
       count += 1;
       context.storage?.set('greetings', count);
       context.logger.info(`Greeting ${count} sent.`);
       renderCount();
     };
   
     return {
       mount(container: HTMLElement) {
         root = document.createElement('section');
         root.style.cssText = `
           height: 100%; display: grid; place-items: center;
           padding: 24px; box-sizing: border-box;
           color: var(--text-color, #f5f7fb);
           background: var(--background-secondary, #252b35);
         `;
   
         const content = document.createElement('div');
         content.style.textAlign = 'center';
   
         const heading = document.createElement('h2');
         heading.textContent = 'Hello from outside Robo-Boy';
   
         button = document.createElement('button');
         button.type = 'button';
         button.addEventListener('click', sendGreeting);
         renderCount();
   
         content.append(heading, button);
         root.append(content);
         container.replaceChildren(root);
       },
   
       setActive(isActive: boolean) {
         root?.toggleAttribute('data-inactive', !isActive);
       },
   
       unmount() {
         button?.removeEventListener('click', sendGreeting);
         root?.remove();
         button = null;
         root = null;
       },
     };
   };
   
   const definition: RoboBoyPanelDefinition = {
     apiVersion: '2.0.0',
     id: 'com.example.greeting',
     activate,
   };
   
   export default definition;

5. 5

   Add roboboy.panel.json

   {
     "schemaVersion": 1,
     "id": "com.example.greeting",
     "name": "Greeting",
     "description": "A minimal persistent greeting panel.",
     "version": "0.1.0",
     "entryPoint": "./dist/index.js",
     "integrity": "sha256-REPLACE_WITH_THE_GENERATED_BASE64_DIGEST=",
     "compatibility": {
       "panelApi": "^2.0.0",
       "roboboy": ">=0.3.0-0 <1.0.0"
     },
     "capabilities": ["storage"],
     "author": { "name": "Your team" },
     "repository": "https://github.com/YOUR_ORG/my-roboboy-panel",
     "tags": ["example"]
   }

   Use a stable lowercase reverse-domain ID you control. The definition, manifest, desired-state selection, and published catalog entry must agree on identity and compatibility.

6. 6

   Build and calculate integrity

   Create scripts/print-integrity.mjs:

   import { createHash } from 'node:crypto';
   import { readFile } from 'node:fs/promises';
   
   const bundle = await readFile(new URL('../dist/index.js', import.meta.url));
   console.log(`sha256-${createHash('sha256')
     .update(bundle)
     .digest('base64')}`);

   npm run typecheck
   npm run build
   npm run integrity

   Replace the placeholder in the manifest with the complete sha256-... output after every bundle change.

7. 7

   Create the desired-state configuration

   Create an ignored config/panel-sources.json inside robo-boy/. Schema v2 uses typed sources and explicit selection; local development does not require an inventory entry.

   {
     "schemaVersion": 2,
     "sources": [
       {
         "type": "local",
         "name": "development",
         "root": "../..",
         "rootEnv": "ROBOBOY_PANEL_WORKSPACE",
         "repositories": ["my-roboboy-panel"]
       }
     ],
     "selection": {
       "mode": "include",
       "panelIds": ["com.example.greeting"]
     }
   }

8. 8

   Stage, load, and use it

   cd ../robo-boy
   npm run panels:stage-local -- \
     --config config/panel-sources.json
   
   npm run dev:panels -- \
     --config config/panel-sources.json

   Open the workspace, choose Add panel, and select Greeting. Click the button, remove and re-add the tile, and confirm the instance-owned count returns. The normal npm run dev command intentionally keeps the tracked empty panel registry.

Patterns for real robot panels.

Panel API v2 brokers ROS as JSON operations. Panels do not receive ROSLIB, the rosbridge URL, or the complete graph; every request is checked against the manifest permissions.

import type { RoboBoyRosSubscription } from
  '@tessel-la/roboboy-panel-sdk';

let subscription: RoboBoyRosSubscription | null = null;

if (context.ros) {
  const selected = await context.ros.selectTopic({
    currentTopic: '/battery_state',
  });
  subscription = await context.ros.subscribe({
    topic: selected.name,
    messageType: selected.messageType,
    queueLength: 1,
    throttleMs: 100,
    compression: 'none',
  }, (message) => {
    renderBattery(message);
  });
}

// In async unmount():
await subscription?.unsubscribe();
subscription = null;

await context.ros?.publish({
  topic: '/cmd_vel',
  messageType: 'geometry_msgs/msg/Twist',
  message: {
    linear: { x: 0.2, y: 0, z: 0 },
    angular: { x: 0, y: 0, z: 0 },
  },
});

import type { RoboBoyJsonObject } from
  '@tessel-la/roboboy-panel-sdk';

type Settings = { topic: string; compact: boolean };

const fallback: Settings = {
  topic: '/battery_state',
  compact: false,
};

const stored = context.storage?.get(
  'settings',
  fallback as unknown as RoboBoyJsonObject,
);
let settings = (stored as unknown as Settings) ?? fallback;

function save(next: Settings) {
  settings = next;
  try {
    context.storage?.set(
      'settings',
      next as unknown as RoboBoyJsonObject,
    );
  } catch (error) {
    context.logger.warn('Unable to persist settings.', error);
  }
}

let frame: number | null = null;
let active = context.viewport.getSnapshot().isActive;

const draw = () => {
  if (!active) return;
  renderLatestSample();
  frame = requestAnimationFrame(draw);
};

const stop = context.viewport.subscribe((snapshot) => {
  active = snapshot.isActive;
  if (active && frame === null) frame = requestAnimationFrame(draw);
  if (!active && frame !== null) cancelAnimationFrame(frame);
  if (!active) frame = null;
});

// In unmount():
stop();
if (frame !== null) cancelAnimationFrame(frame);

let device: BluetoothDevice | null = null;

async function connectFromUserGesture() {
  device = await navigator.bluetooth.requestDevice({
    filters: [{ services: ['YOUR_STANDARD_OR_VENDOR_SERVICE_UUID'] }],
  });
  const server = await device.gatt?.connect();
  // Resolve your documented GATT service and characteristics.
}

// In unmount():
device?.gatt?.disconnect();
device = null;

Test the artifact the host will actually import.

Local repository layout

workspace/
  robo-boy/                 # host application
  my-roboboy-panel/        # independent source + dist + manifest

The local source lists repository directories explicitly, then verifies bundle, manifest, permissions, compatibility, and integrity before generating an ignored .panel-stage/public/ tree. A separate inventory entry is needed only when publishing.

Robo-Boy project checks

npm run lint
npm run test:run
npm run build
npm run e2e

For Docker development with the known examples, add infra/compose/panels.yml. For a private sibling panel, mount its repository through an uncommitted deployment-local Compose override and select it in the schema-v2 source configuration.

Publish immutable ESM; install into the deployment.

1. Bundle every runtime dependency. The release entry must be self-contained browser ESM with no unresolved bare imports or relative dynamic chunks. Frameworks such as React, Vue, and Three.js belong to the panel bundle; ROS access comes from the v2 broker.
2. Generate and record SHA-256 SRI. The exact value must agree across bundle, panel manifest, and inventory.
3. Publish a versioned release. Include the manifest and bundle. Treat assets as append-only.
4. Register metadata separately. An inventory points at immutable release artifacts; it is not installed state and does not host source.
5. Install for a deployment. The panel manager resolves explicit desired state, downloads selected releases, verifies identity and hashes, then atomically replaces installed.json. The browser transfers verified same-origin bytes into isolated panel sandboxes.

# Official remote source overlay
export ROBOBOY_PANEL_MANAGER_TOKEN='use-a-long-random-deployment-secret'

docker compose \
  -f docker-compose.yml \
  -f infra/compose/panels.remote.yml \
  build app panel-manager

docker compose \
  -f docker-compose.yml \
  -f infra/compose/panels.remote.yml \
  up -d

The Compose manager exposes authenticated preview/apply installation controls through Manage installations…. Private sources can use HTTPS release storage and authorization supplied through an approved named environment variable. Credentials stay on the manager side, redirects are revalidated, and duplicate IDs across configured sources are rejected.

Build against the contract, not the host implementation.

Start at the boundary that is failing.