Post

Debugging the Ghost in the Browser: Resolving a BrowserOS MCP Sidecar Crash & Migration Desync

Debugging the Ghost in the Browser: Resolving a BrowserOS MCP Sidecar Crash & Migration Desync

When working with autonomous AI coding agents, one of the most powerful workflows is delegating web tasks—authentication flows, dashboard verifications, and scraping—to a dedicated agent browser. In my setup on Arch/CachyOS Linux, I rely on BrowserOS Neo connected to Google’s Antigravity CLI (agy) via the Model Context Protocol (MCP).

Everything was running smoothly until a sudden failure stopped the browser tool dead in its tracks:

1
2
✗ browseros-neo error: calling "initialize": sending "initialize": rejected by transport: 
  Post "http://127.0.0.1:9010/mcp": dial tcp 127.0.0.1:9010: connect: connection refused

BrowserOS MCP Connection Refused in Antigravity CLI

Port 9010 was dead. No matter how many times the browser was launched, agy couldn’t reach the MCP endpoint. What looked like a simple connection timeout turned out to be a fascinating journey through Chromium IPC architecture, Rust SeaORM migration rollbacks, SQLite column collision bugs, and an overly aggressive pre-flight package script.

Here is the deep-dive analysis of what went wrong and how we systematically diagnosed, repaired, and permanently resolved it through methodical pair-debugging with AI.


1. Architecture: How BrowserOS MCP Actually Works

To debug why 127.0.0.1:9010 was refusing connections, we first had to trace the plumbing of BrowserOS Neo:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
┌─────────────────────────┐          HTTP POST /mcp          ┌──────────────────────────────────┐
│   Antigravity CLI (agy) │ ───────────────────────────────> │  BrowserOS Host (browserclaw)    │
│      (MCP Client)       │                                  │  Proxy Port: 0.0.0.0:9010        │
└─────────────────────────┘                                  └──────────────────────────────────┘
                                                                               │
                                                                   Internal IPC / Proxy
                                                                               ▼
┌─────────────────────────┐          CDP ws://:9114          ┌──────────────────────────────────┐
│  Puppeteer / DevTools   │ ───────────────────────────────> │  Rust Sidecar Server             │
│                         │                                  │  (browseros-claw-server)         │
│                         │                                  │  Port: 127.0.0.1:9210            │
└─────────────────────────┘                                  └──────────────────────────────────┘
                                                                               │
                                                                         SeaORM / SQLite
                                                                               ▼
                                                             ~/.browserclaw/browserclaw.sqlite
  1. browserclaw (Chromium application): Acts as the parent process and opens port 9010 as a public HTTP proxy.
  2. browseros-claw-server (Rust sidecar): A background daemon spawned by Chromium that binds to 127.0.0.1:9210, managing sessions, state, and tool dispatch.
  3. browserclaw.sqlite: Stores persistent sessions, tab metadata, and tracks schema migrations using SeaORM.

If the Rust sidecar fails to start or crashes, Chromium attempts 5 connection retries before giving up:

1
[ERROR:browseros_server_manager.cc:1058] browseros: Bundled server unavailable; stopping sidecar

When the sidecar stops, Chromium immediately tears down the port 9010 proxy, leaving the MCP client with connection refused.


2. Digging Into the Logs: The Crash Sequence

Checking journalctl --user and ~/.browserclaw/logs/claw-server.log revealed the initial smoking gun:

1
2
3
4
5
6
7
2026-09-29T07:15:23Z INFO sea_orm_migration::migrator: Applying all pending migrations
2026-09-29T07:15:23Z INFO sea_orm_migration::migrator: Applying migration 'm0017_add_run_error_budget'
2026-09-29T07:15:23Z INFO sea_orm_migration::migrator: Migration 'm0017_add_run_error_budget' has been applied
2026-09-29T07:15:23Z INFO sea_orm_migration::migrator: Applying migration 'm0018_add_feedback_invite'
2026-09-29T07:15:23Z INFO sea_orm_migration::migrator: Migration 'm0018_add_feedback_invite' has been applied
2026-09-29T07:15:23Z INFO sea_orm_migration::migrator: Applying migration 'm0019_add_feedback_invite_dismissal'
Error: Execution Error: error returned from database: (code: 1) duplicate column name: dismissed_at_ms

Immediately followed 5 seconds later by:

1
2
3
4
5
2026-09-29T07:15:28Z INFO sea_orm_migration::migrator: Applying all pending migrations
Error: Custom Error: Migration file of version 'm0017_add_run_error_budget' is missing, 
this migration has been applied but its file is missing
Migration file of version 'm0018_add_feedback_invite' is missing, 
this migration has been applied but its file is missing

Two different binaries were conflicting over the exact same SQLite database.


3. Root Cause Analysis: The Two-Binary Trap

Trapped Between Two Versions

By inspecting _EXE metadata across journal entries, we found that BrowserOS maintains two different server binaries:

  1. The System Bundled Server (v0.0.57): /usr/lib/browserclaw/BrowserClawServer/default/resources/bin/browseros-claw-server Compiled with migrations only up to m0016_add_task_summary.
  2. The Auto-Updated Server (v0.0.59): ~/.config/browser-claw/.browseros/BrowserClawServer/versions/0.0.59/resources/bin/browseros-claw-server Compiled with newer migrations m0017, m0018, and m0019.

The Fatal Migration Loop

  1. When BrowserOS launched, it downloaded the updated server 0.0.59.
  2. On startup/shutdown, 0.0.59 ran pending SeaORM migrations:
    • m0017_add_run_error_budget: Created table run_error_budget (Success).
    • m0018_add_feedback_invite: Created table feedback_invite (Success).
    • m0019_add_feedback_invite_dismissal: Attempted ALTER TABLE feedback_invite ADD COLUMN dismissed_at_ms.
  3. The Column Collision: The feedback_invite table on disk already had the dismissed_at_ms column from an earlier partial migration attempt. In SQLite, adding an existing column triggers a hard fatal error: duplicate column name: dismissed_at_ms.
  4. Because migration m0019 aborted, 0.0.59 failed to boot. BrowserOS detected the failure and rolled back to the bundled fallback server (v0.0.57).
  5. SeaORM Integrity Panic: When the fallback server booted, it queried SELECT version FROM seaql_migrations;. It saw m0017 and m0018 recorded in the database, but since those migrations were written after v0.0.57 was compiled, SeaORM threw: Migration file of version 'm0017_add_run_error_budget' is missing and aborted immediately.

The Rogue Doctor Script

To make matters worse, the AUR package for browseros-neo-bin had bundled a wrapper script (/usr/lib/browserclaw/browseros-neo-doctor.sh) designed to run before every browser launch:

1
2
3
4
5
6
7
8
9
# From browseros-neo-doctor.sh
rm -rf "$LOCK_DIR/versions" 2>/dev/null || true

sqlite3 "$DB" "SELECT version FROM seaql_migrations;" | while read -r v; do
  strings "$BIN" | grep -aFq "$v" || {
    echo "removing unknown migration $v"
    sqlite3 "$DB" "DELETE FROM seaql_migrations WHERE version='$v';"
  }
done

The script:

  • Forcefully wiped versions/ on every boot, deleting v0.0.59.
  • Checked strings only on the old bundled binary, deleting valid migrations from the database.
  • And checked the wrong directory for server.lock (.../BrowserClawServer/server.lock instead of .../.browseros/server.lock), leaving stale 0-byte lock files behind.

4. The Clean, Permanent Fix

Instead of band-aid workarounds, we engineered a durable, surgical fix in three steps:

Step 1: Repair the Database Schema

Since feedback_invite was empty (0 rows), we dropped it completely and cleared the dangling migration entries so the migrator could rebuild it cleanly:

1
2
3
4
5
6
7
8
sqlite3 ~/.browserclaw/browserclaw.sqlite "
  DROP TABLE IF EXISTS feedback_invite;
  DELETE FROM seaql_migrations WHERE version IN (
    'm0017_add_run_error_budget',
    'm0018_add_feedback_invite',
    'm0019_add_feedback_invite_dismissal'
  );
"

Step 2: Remove Stale Lock Files

1
rm -f ~/.config/browser-claw/.browseros/server.lock

Step 3: Implement an Intelligent User-Space Wrapper

We created ~/.local/bin/browserclaw (which has priority in PATH over /usr/bin):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
#!/usr/bin/env bash
# BrowserOS / BrowserClaw wrapper with robust pre-flight repair.
set -euo pipefail

DB="${HOME}/.browserclaw/browserclaw.sqlite"
LOCK_DIR="${HOME}/.config/browser-claw/.browseros"
FALLBACK_LOCK_DIR="${LOCK_DIR}/BrowserClawServer"

# 1. Clean up stale server.lock files if process is dead or lock is empty
for lfile in "${LOCK_DIR}/server.lock" "${FALLBACK_LOCK_DIR}/server.lock"; do
  if [[ -f "$lfile" ]]; then
    _pid=$(tr -d '[:space:]' < "$lfile" 2>/dev/null || true)
    if [[ -z "$_pid" ]] || ! kill -0 "$_pid" 2>/dev/null; then
      rm -f "$lfile" 2>/dev/null || true
    fi
  fi
done

# 2. Prevent m0019 duplicate column collision from breaking startup
if [[ -f "$DB" ]]; then
  _has_m0019=$(sqlite3 "$DB" "SELECT count(*) FROM seaql_migrations WHERE version='m0019_add_feedback_invite_dismissal';" 2>/dev/null || echo "0")
  if [[ "$_has_m0019" == "0" ]]; then
    _fi_exists=$(sqlite3 "$DB" "SELECT count(*) FROM sqlite_master WHERE type='table' AND name='feedback_invite';" 2>/dev/null || echo "0")
    if [[ "$_fi_exists" != "0" ]]; then
      sqlite3 "$DB" "DROP TABLE IF EXISTS feedback_invite;" 2>/dev/null || true
      sqlite3 "$DB" "DELETE FROM seaql_migrations WHERE version IN ('m0018_add_feedback_invite');" 2>/dev/null || true
    fi
  fi
fi

# 3. Preserve auto-updated versions and execute real browser
export LD_LIBRARY_PATH="/usr/lib/browserclaw${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
exec /usr/lib/browserclaw/browserclaw "$@"

We also created ~/.local/share/applications/browserclaw.desktop pointing to ~/.local/bin/browserclaw so that launching BrowserOS from the KDE Plasma application menu uses the wrapper seamlessly.


5. Verification & Live Confirmation

Upon launching BrowserOS with the new wrapper, the sidecar logs immediately confirmed success:

1
2
3
4
INFO sea_orm_migration::migrator: Migration 'm0017_add_run_error_budget' has been applied
INFO sea_orm_migration::migrator: Migration 'm0018_add_feedback_invite' has been applied
INFO sea_orm_migration::migrator: Migration 'm0019_add_feedback_invite_dismissal' has been applied
INFO browseros_claw_server_rs: claw-server-rust listening bound=127.0.0.1:9210

All three listening ports came alive:

  • Port 9114: Chromium CDP
  • Port 9210: Rust Sidecar (browseros-claw-server)
  • Port 9010: Chromium MCP Reverse Proxy

Testing the Protocol Handshake

Testing the JSON-RPC handshake against http://127.0.0.1:9010/mcp:

1
2
3
4
curl -X POST http://127.0.0.1:9010/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test-client","version":"1.0"}},"id":1}'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "protocolVersion": "2024-11-05",
    "capabilities": {
      "tools": { "listChanged": true }
    },
    "serverInfo": {
      "name": "browseros-neo",
      "title": "BrowserOS neo",
      "version": "0.0.59"
    }
  }
}

HTTP 200 OK. serverInfo: 0.0.59.

Antigravity CLI Verification

Back in Antigravity CLI, browseros-neo reconnected instantly with green checkmarks, loading all browser action tools:

BrowserOS MCP Resolved in Antigravity CLI

BrowserOS Cockpit Verification

Checking the BrowserOS Neo Cockpit MCP management page confirms both connected agents are paired and active:

BrowserOS Cockpit Connected Agents


Takeaways

  1. Auto-updates and Embedded Migrations: When a binary auto-updates independently of its parent package manager, schema migrations must be strictly backward-compatible and idempotent. Never assume an ALTER TABLE ADD COLUMN is safe without checking table definition state.
  2. Be Careful with Package Doctor Scripts: Automated cleanup scripts that grep binary strings or delete cache folders (versions/) can inadvertently sabotage native update mechanisms.
  3. Trace the Entire Protocol Chain: In modern agentic tools, a single “connection refused” error often spans multiple application layers (CLI client → proxy → sidecar daemon → SQLite database). Methodical logging, process inspection, and AI pair-debugging will always uncover the root cause.
This post is licensed under CC BY 4.0 by the author.