---
name: update-fleet
description: Push code, skills, or config updates to running agent servers via update-agent.yml / deploy-skills.yml, with all pre-flight and restart gotchas
disable-model-invocation: true
---

# Update the Agent Fleet

Push updates to existing agents. Playbooks live in `infra/ansible/playbooks/`, inventory in `infra/ansible/inventory/hosts.ini` (gitignored — IPs never versioned).

## Pre-flight (skipping these ships stale code silently)

1. **Rebuild livia-cli dist** — `infra/livia-cli/dist/` is gitignored but it is what Ansible rsyncs (excludes `src/`) and Docker bind-mounts:
   ```bash
   pnpm --dir infra/livia-cli build
   ```
2. **Rebuild shared** if registry/instructions changed: `pnpm --filter @livia/shared build`, and `pnpm gen:agents-vars` if the connector catalogue moved.
3. **Check client-vars.yml completeness** (read-only SSH): `/etc/livia/client-vars.yml` must have `client_name`, `openclaw_gateway_token`, `reporter_secret`. Missing → run `playbooks/migrate-client-vars.yml` first (idempotent).

## Commands

```bash
# Full agent update (livia-cli + livia-mail binaries, templates, image rebuild, gateway restart)
ansible-playbook -i infra/ansible/inventory/hosts.ini \
  infra/ansible/playbooks/update-agent.yml \
  --vault-password-file infra/ansible/.vault_pass \
  --limit <server-name>

# Skills only
ansible-playbook -i infra/ansible/inventory/hosts.ini \
  infra/ansible/playbooks/deploy-skills.yml \
  --vault-password-file infra/ansible/.vault_pass \
  --limit <server-name>
```

## Hard rules

- **Never `delete: true`** in deploy-skills — clients create custom skills on their agents.
- **Never re-render `openclaw.json`** — only `setup.yml` writes it (channels, credentials, allowlist, crons mutated at runtime). `update-agent.yml` already respects this.
- **`connectors` / `connector_env_vars` extra-vars = full desired state.** Passing an empty list wipes all connectors. Omit them entirely for a plain update.
- Reconfiguring connectors on a prod agent needs explicit user authorization (classified as production deploy).

## Gateway restart

Always **`docker compose stop` → `docker compose up -d --force-recreate`**, never `restart` or bare `up -d` (port-rebind race + rm-of-running-container, Sentry LIVIA-PROVISIONER-1/2). Health checks: `retries: 15, delay: 5` minimum — prod cold start is 32–47s.

**Orphan recovery**: a botched recreate leaves `<hex>_openclaw-openclaw-gateway-1` containers that compose no longer tracks (`docker compose ps` empty while the container serves). Only fix: `docker stop <hex>_…` + `docker rm <hex>_…` by exact name, then `docker compose up -d openclaw-gateway`, then poll `/health` until `{"ok":true,"status":"live"}`.

## Post-deploy verification (SSH per agent)

```bash
ssh -i ~/.ssh/livia_deployer root@<IP>
```

- Skills landed: grep the new content in `~/.openclaw/skills/<skill>/SKILL.md` (NOT `workspace/skills/`)
- CLI updated: grep the new subcommand in `~/livia-cli/dist/index.js`
- AGENTS.md re-rendered with the expected block
- Container is the canonical `openclaw-openclaw-gateway-1` (no hex prefix); port varies per agent (`docker ps --format '{{.Ports}}'`)
- Warmup chat request: 401 is acceptable post-restart (health check allows it); the first real message pays the prep tax

Memory sqlite (`~/.openclaw/memory/main.sqlite`) and sessions persist through updates via the `~/.openclaw` bind-mount — nothing to back up.
