# Pi → Dropbox sync (rclone)

**Platform:** Raspberry Pi 5 / Bookworm only for day-to-day sync. All four Pis run
the same scripts from `~/EtherGold/4-Board Test Fixture/sync/`. Dropbox has no native ARM client — **rclone**
uploads (and optional shared pulls) over the network.

**Dropbox folder (2026-06-15):** captures upload to the **Cursor OHD workspace** tree:

```text
Dyzine Tek, LLC/Engineering/Digital Signage/Overhead Bank Signage/DyzineFrame/Cursor OHD Project/EtherGold/4-Board Test Fixture/N/
```

(`N` = 1–4 per Pi.) Legacy Hardware Development path is **auto-migrated** by
`install_sync.sh` when found in `rclone.env`. See **§ Workspace migration** below.

Your PC enters the picture **once**: paste the Dropbox OAuth token during
`rclone config` (step 2). After that, captures and syncs run on the Pis.

Typical flow:

1. Capture on Pi → `~/EtherGold/2026-06-08 Capture Only RAW/data/` + `logs/`
2. `./sync_to_dropbox.sh` → files land in Dropbox under `4-Board Test Fixture/N/`
   (only `DROPBOX_DEST` differs per physical position; local paths are identical)

---

## 1. Install rclone on each Pi (once)

```bash
cd ~/EtherGold/4-Board Test Fixture/sync
bash install.sh
./setup_rclone.sh
```

Uses the official installer (newer than `apt install rclone` on Bookworm).

---

## 2. Authorize Dropbox (headless Pi)

The Pi has no browser, so you authorize on your **Windows PC** and paste a token
into the Pi.

### On Windows (one time for auth helper)

Install rclone:

```powershell
winget install Rclone.Rclone
```

Open a **new** PowerShell window, then:

```powershell
rclone authorize "dropbox"
```

- Browser opens → sign in to Dropbox → Allow.
- PowerShell prints a long `token` JSON blob. **Copy the entire line.**

### On the Pi

```bash
rclone config
```

| Prompt | Answer |
|--------|--------|
| `n` / `s` / … | `n` (new remote) |
| name | `dyzine-dropbox` (or any name — use the same in `rclone.env`) |
| Storage | `dropbox` (type number or name from list) |
| client_id | Enter (blank = rclone shared app) |
| client_secret | Enter (blank) |
| Edit advanced config? | `n` |
| Use auto config? | **`n`** (headless) |
| token | **Paste** the token from Windows |
| Configure as team drive? | `n` (unless you use Dropbox Business team folders) |
| Yes this is OK | `y` |
| Quit | `q` |

### Verify

```bash
rclone listremotes
# should show: dyzine-dropbox:

rclone lsd dyzine-dropbox:
# lists top-level Dropbox folders — use this to pick DROPBOX_DEST
```

---

## 3. Point sync at the right Dropbox folder

```bash
cd ~/EtherGold/4-Board Test Fixture/sync
cp rclone.env.example rclone.env
nano rclone.env
```

**Location:** `rclone.env` belongs in **`~/EtherGold/4-Board Test Fixture/sync/`** on each Pi.
The sync script always reads `${ROOT}/rclone.env` from the `sync` folder.

Set **per Pi** — local paths are **identical on all four boards**; **`install_sync.sh`**
sets `DROPBOX_DEST` from hostname (`topbox1` → `.../1`, etc.):

- `RCLONE_REMOTE` — same on every Pi (`dyzine-dropbox`, **no colon** in `rclone.env`)
- `DROPBOX_FIXTURE_ROOT` — same on every Pi (fixture parent, no `/1` suffix)
- `SYNC_SOURCE` — **same on every Pi** (parent of `data/` and `logs/`)
- `DROPBOX_DEST` — **auto-set by `install_sync.sh`** to `.../4-Board Test Fixture/N`

Run after moving `sync/` or on every pull (last step):

```bash
cd ~/EtherGold/4-Board\ Test\ Fixture/sync
./install_sync.sh
# topbox3 → DROPBOX_DEST=.../3
```

Override: `FIXTURE_POS=2` in `rclone.env`, or `FIXTURE_POS=2 ./install_sync.sh`.

On the command line, rclone uses a colon after the remote name (`dyzine-dropbox:`). The sync
script builds that for you from `RCLONE_REMOTE` + `DROPBOX_DEST`.

**Shared settings** (same on every Pi except auto-set `DROPBOX_DEST`):

```bash
RCLONE_REMOTE=dyzine-dropbox
SYNC_SOURCE="4-Board Test Fixture/fixture-timing"
PI_LABEL=
ETHERGOLD_ROOT=/home/topbox/EtherGold
PULL_SCOPE=timing
```

`install_sync.sh` sets `DROPBOX_DEST=.../N` from hostname. **Pi 1 example** after install:

```bash
DROPBOX_FIXTURE_ROOT="Dyzine Tek, LLC/Engineering/Digital Signage/Overhead Bank Signage/DyzineFrame/Cursor OHD Project/EtherGold/4-Board Test Fixture"
DROPBOX_DEST="${DROPBOX_FIXTURE_ROOT}/1"
```

**Pi 2 / 3 / 4** — same `DROPBOX_FIXTURE_ROOT`; `install_sync` writes `/2`, `/3`, or `/4` into `DROPBOX_DEST`.

For **RAW capture** instead of fixture-timing, use:
`SYNC_SOURCE="2026-06-08 Capture Only RAW"`.

Local capture (every Pi, fixture-timing):

```text
~/EtherGold/4-Board Test Fixture/fixture-timing/data/events.jsonl
~/EtherGold/4-Board Test Fixture/fixture-timing/data/packets.raw
~/EtherGold/4-Board Test Fixture/fixture-timing/logs/fixture_listen.log
```

(RAW kit: `~/EtherGold/2026-06-08 Capture Only RAW/data/` + `logs/`.)

Dropbox after upload (position from `DROPBOX_DEST`; flat landing — no `topboxN/` subfolder):

```text
.../Cursor OHD Project/EtherGold/4-Board Test Fixture/1/data/events.jsonl    ← Pi 1
.../Cursor OHD Project/EtherGold/4-Board Test Fixture/4/logs/fixture_bridge.log  ← Pi 4
```

`install_sync.sh` sets `PI_LABEL=` (empty) so uploads overwrite `N/data/` on PC.
Legacy captures under `N/topboxN/data/` are consolidated by `normalize_pc_landing.ps1`
before archive.

Do **not** put `/1` `/2` etc. in `SYNC_SOURCE` unless you actually create those
folders on the Pi (you do not — the Pis are identical).

Optional **fixture-timing** (same `SYNC_SOURCE` pattern, second sync):

```bash
SYNC_SOURCE="4-Board Test Fixture/fixture-timing" ./sync_to_dropbox.sh
```

(Adjust if your timing kit lives elsewhere under `~/EtherGold/`.)

---

## 4. Upload after a capture

**Preview first (recommended):**

```bash
cd ~/EtherGold/4-Board Test Fixture/sync
DRY_RUN=1 ./sync_to_dropbox.sh
```

**Upload** (only `SYNC_SOURCE` from `rclone.env`):

```bash
./sync_to_dropbox.sh
```

**One-off override:**

```bash
DROPBOX_DEST=".../4-Board Test Fixture/4" SYNC_SOURCE="2026-06-08 Capture Only RAW" ./sync_to_dropbox.sh
```

Uploaded by default:

| Pattern | File |
|---------|------|
| `**/data/packets.raw` | RAW hex capture |
| `**/data/raw_stats.json` | RAW stats |
| `**/data/*.csv` | Gameplay decode CSV |
| `**/data/*.json` | Other capture JSON |
| `**/logs/raw_capture.log` | Debug log |

Uses `rclone copy` with `--include "/data/**"` and `--include "/logs/**"` (upload only — does **not** delete remote files).

### Pi housekeeping (optional)

You do **not** need to move `data/` and `logs/` away from the kit folders.
Keep scripts + captures together on the Pi while developing. Just set a narrow
`SYNC_SOURCE` so only the active fixture folder uploads.

After a successful sync, old test runs can be deleted or moved locally:

```bash
mkdir -p ~/EtherGold/_archive
mv ~/EtherGold/2026-06-05\ Capture\ Only ~/EtherGold/_archive/
```

Do not sync `_archive/` — it is outside `SYNC_SOURCE`.

---

## 5. Optional: auto-sync on a timer

```bash
crontab -e
```

Upload every 15 minutes while testing:

```cron
*/15 * * * * /home/topbox/EtherGold/4-Board Test Fixture/sync/sync_to_dropbox.sh >> /home/topbox/EtherGold/4-Board Test Fixture/sync/sync.log 2>&1
```

---

## 6. Pull shared fixture content on a Pi (excludes 1–4)

Use **`pull_fixture_shared_from_dropbox.sh`** on a **Pi** when you need to pull
shared Dropbox content (e.g. `fixture-timing/`) **without** downloading the large
per-position capture trees in `1/`–`4/`.

Same `rclone.env` remote as upload. Set `DROPBOX_FIXTURE_ROOT` (or derive from
`DROPBOX_DEST`). Optional `LOCAL_FIXTURE_ROOT` — default on Pi:

```bash
/home/topbox/EtherGold/4-Board Test Fixture
```

### Pull scope (~file counts on Dropbox today)

| `PULL_SCOPE` | What you get | Approx. files |
|--------------|--------------|---------------|
| **`timing`** (default) | `fixture-timing/**` + `sync/**` (excludes POG source) | ~60 |
| **`shared`** | Fixture root except `1–4/`, `sessions/`, and POG source | ~50 |
| **`full`** | Old behavior: includes `sessions/` archives (still excludes POG source) | ~900+ |

**Never pulled to Pis:** `fixture-timing/Legacy Code/` (POG 3.50 firmware source),
`Legacy/`, `v18r Code/`, and stale dev stacks (`jun12-replay/`, `v136-stack/`).
Those stay PC/Dropbox reference only. If a Pi already has `Legacy Code/`, delete it:

```bash
rm -rf ~/EtherGold/4-Board\ Test\ Fixture/fixture-timing/Legacy\ Code
rm -rf ~/EtherGold/4-Board\ Test\ Fixture/fixture-timing/Legacy
rm -rf ~/EtherGold/4-Board\ Test\ Fixture/fixture-timing/v18r\ Code
```

The old default effectively pulled **`sessions/`** (~855 archived files). That is
why a dry run could list hundreds or thousands of **unchanged** lines — especially
with `-v`, which logs every file rclone compares.

**Preview (lists each file that would change):**

```bash
cd ~/EtherGold/4-Board\ Test\ Fixture/sync
DRY_RUN=1 bash pull_fixture_shared_from_dropbox.sh
```

**Download fixture-timing + driver fork (default):**

```bash
bash pull_fixture_shared_from_dropbox.sh
```

**Only if you need session archives on the Pi:**

```bash
PULL_SCOPE=full DRY_RUN=1 bash pull_fixture_shared_from_dropbox.sh
PULL_SCOPE=full bash pull_fixture_shared_from_dropbox.sh
```

**Quiet mode (summary stats only — useful for `PULL_SCOPE=full`):**

```bash
QUIET=1 bash pull_fixture_shared_from_dropbox.sh
# or: VERBOSE=0 bash pull_fixture_shared_from_dropbox.sh
```

Filters: `fixture_timing.rclone-filter`, `fixture_shared.rclone-filter`,
`fixture_full.rclone-filter`.

`sync/` lives **under** `4-Board Test Fixture/`, so **`PULL_SCOPE=timing` updates
these rclone scripts automatically** on each pull (no separate bootstrap).

**Migrate from old `~/EtherGold/sync/` on each Pi (once):**

```bash
mkdir -p ~/EtherGold/4-Board\ Test\ Fixture/sync
[ -f ~/EtherGold/sync/rclone.env ] && cp ~/EtherGold/sync/rclone.env ~/EtherGold/4-Board\ Test\ Fixture/sync/
rm -rf ~/EtherGold/sync
cd ~/EtherGold/4-Board\ Test\ Fixture/sync && bash install.sh
# install.sh runs install_sync.sh → sets DROPBOX_DEST from hostname (topboxN → /N)
```

**Remove `sessions/` on Pis** (not needed for fixture operation):

```bash
rm -rf ~/EtherGold/4-Board\ Test\ Fixture/sessions
```

Add `PULL_SCOPE=timing` to `rclone.env` so Pis never pull session archives.

If you see **900+ checks**, the Pi is still on an **old pull script** or
`PULL_SCOPE=full`. Pull from the new `sync/` path and confirm
`PULL_SCOPE=timing` in `rclone.env`.

If you see **3000+** listed paths, also verify `DROPBOX_FIXTURE_ROOT` in
`rclone.env` — it must be the **4-Board Test Fixture** folder, not all of
`EtherGold/`.

Your PC sees new files via **Dropbox desktop sync** — you do not run rclone on
Windows for capture or shared pulls.

---

## Workspace migration (2026-06-15)

The PC canonical folder moved from **Hardware Development** to **Cursor OHD Project**
(see `../README.md`). Each Pi must repoint `rclone.env` once.

### On each Pi (recommended)

**Option A — pull new scripts, then migrate** (if Pis can still read old Dropbox path):

```bash
cd ~/EtherGold/4-Board\ Test\ Fixture/sync
# If sync/ is stale, one-time manual copy of migrate_dropbox_paths.sh + dropbox_paths.sh from PC/USB
bash migrate_dropbox_paths.sh
```

**Option B — USB / manual:** copy updated `sync/` from PC, then:

```bash
cd ~/EtherGold/4-Board\ Test\ Fixture/sync
bash install.sh
# install.sh → install_sync.sh auto-replaces legacy path in rclone.env
```

**Option C — edit by hand:** set in `rclone.env`:

```bash
DROPBOX_FIXTURE_ROOT="Dyzine Tek, LLC/Engineering/Digital Signage/Overhead Bank Signage/DyzineFrame/Cursor OHD Project/EtherGold/4-Board Test Fixture"
```

Then `bash install_sync.sh` (writes `DROPBOX_DEST=.../N`).

### Verify

```bash
grep DROPBOX_ rclone.env
DRY_RUN=1 bash sync_to_dropbox.sh
rclone lsd "dyzine-dropbox:${DROPBOX_DEST}"
```

Canonical paths live in `dropbox_paths.sh` (single source — update there if the PC folder moves again).

### On PC

Ensure Dropbox has synced. Live landing after Pi upload:

```text
Cursor OHD Project/EtherGold/logs/N/data/events.jsonl
```

Archive (writes to workspace `sessions/`, sibling of `EtherGold/`):

```powershell
cd "EtherGold/sync"
.\archive_session.ps1 -SessionLabel "short-description"
```

**Verify remote landing (Windows, rclone):**

```powershell
cd "EtherGold/sync"
powershell -NoProfile -File .\verify_dropbox_landing.ps1
powershell -NoProfile -File .\verify_dropbox_landing.ps1 -CheckEvents
```

Requires `rclone` on PATH (`winget install Rclone.Rclone`) and remote `dyzine-dropbox` (same name as Pis).

---

## Wi-Fi profile prune (stale SSIDs)

Pis accumulate saved Wi-Fi profiles (`POG-CC`, the Raspberry Pi Imager
`preconfigured` profile, phone hotspots). Every one with `autoconnect=yes` is a
radio NetworkManager will race on boot, and a headless box has no secrets agent,
so a stale profile ends in `no secrets: No agents were available` and a failed
activation loop.

`prune_wifi_profiles.sh` sets `connection.autoconnect=no` on every saved Wi-Fi
profile except the keep list. Nothing is deleted — re-enable with
`nmcli connection modify "<name>" connection.autoconnect yes`. Secrets are never
read or printed.

**Lockout guard:** the profile carrying the current SSH session is always kept,
even if not named with `-Keep`. On a dual-radio box the radio you are talking
over survives and the other is pruned.

```powershell
.\apply_wifi_prune.ps1 -DryRun -IncludeBench    # preview (always do this first)
.\apply_wifi_prune.ps1 -IncludeBench            # chain hops + bench pos 1-5
.\apply_wifi_prune.ps1 -OctetList "45" -Down    # one host, also drop it now
```

`-Down` deactivates a pruned profile immediately; without it the change takes
effect at next reconnect or boot, which is the safe default for remote work.

Ops-only — does not touch the bridge, EG tip, or uartd, so no `EG_VERSION` bump.

---

## Troubleshooting

| Problem | Fix |
|---------|-----|
| Pi keeps asking for Wi-Fi credentials / `no secrets: No agents were available` | Stale saved profile with `autoconnect=yes` — run `apply_wifi_prune.ps1` (see above) |
| `events.jsonl` missing but `packets.raw` synced | Old `sync_to_dropbox.sh` only had `--include "/data/*.json"` — **`.jsonl` is not `.json`**. Update `sync/` from Dropbox and re-run sync |
| `No capture files under ...` | Captures are probably in `fixture-timing/data/` — set `SYNC_SOURCE="4-Board Test Fixture/fixture-timing"` in `rclone.env` |
| `SYNC_SOURCE is not a directory` | Run `ls ~/EtherGold/4-Board\ Test\ Fixture/fixture-timing/data` and fix path |
| Script exits immediately, 0 files | `DRY_RUN=1 bash sync_to_dropbox.sh` — check "Local: N matching capture file(s)" |
| `Permission denied` on `./sync_to_dropbox.sh` | `bash install.sh` or `bash sync_to_dropbox.sh` (no `./` needed) |
| `Failed to get token` | Re-run `rclone authorize "dropbox"` on Windows; paste fresh token |
| `directory not found` | Check `DROPBOX_DEST` with `rclone lsd dyzine-dropbox:` |
| `Forbidden` / quota | Dropbox account permissions or full quota |
| Stale apt rclone | Use `./setup_rclone.sh` (official build) |
| `$'\r': command not found` | CRLF in `rclone.env` — `sed -i 's/\r$//' ~/EtherGold/4-Board Test Fixture/sync/rclone.env` |
| Windows line endings on Pi | `sed -i 's/\r$//' ~/EtherGold/4-Board Test Fixture/sync/*.sh rclone.env` after USB copy |

### Test without uploading

```bash
rclone copy --dry-run -v SOURCE dyzine-dropbox:DEST
```

### Own Dropbox app (optional)

For heavy use, create an app at [dropbox.com/developers/apps](https://www.dropbox.com/developers/apps)
(Full Dropbox, redirect URI `http://localhost:53682/`) and enter your
`client_id` / `client_secret` during `rclone config`. See [rclone dropbox docs](https://rclone.org/dropbox/).

---

## Security

- `rclone.env` holds paths only (no secrets).
- OAuth token lives in `~/.config/rclone/rclone.conf` on the Pi — treat like a password.
- Do not copy `rclone.conf` into Dropbox.