Zum Inhalt

tvOS T2a — Editor-Remote (iPhone / iPad)

Date: 2026-06-16 Status: Approved (design), ready for planning Scope: new apple/PhysarumApp/Sources/Remote/* (shared + per-platform), small touches to AmbientController, RootView (iOS/iPad), TVRootView, project.yml (Info.plist/Bonjour). Engine/WGSL untouched.

Motivation

tvOS T1 is autonomous (Siri Remote, lean-back). The natural companion is using the full editor app on iPhone/iPad as a remote that drives the ATV over the local network — the documented T2. Johannes has a real Apple TV 4K (Wohnzimmer) now, so this is buildable + device-verifiable.

Goal (T2a)

The iPhone/iPad full editor discovers an Apple TV running Physarum and mirrors its editing session to it: preset selection, live param edits, and prev/next/cycle commands flow to the ATV in real time. The phone keeps showing its own local preview (WYSIWYG); the ATV mirrors it. On disconnect the ATV resumes autonomous T1 cycling.

Non-Goals (explicit follow-ups)

  • macOS as remote — DeviceDiscoveryUI is iOS/iPadOS/tvOS only; macOS needs an NWBrowser-based discovery. Discovery is abstracted behind a protocol now so macOS is a small fast-follow.
  • T2b — phone-mic → audio-reactive (real-time band streaming): separate increment.
  • No Engine/Renderer/WGSL changes; no change to the standalone iOS/macOS editor behavior when not connected.

Architecture

Key idea (max reuse): the remote is just another source for the same AmbientController sink the Siri Remote already feeds (per the original Inc-12 design). No parallel "driven mode". The only thing beyond T1: the remote also streams live param edits, applied via the existing setParams path. The ATV's crossfade-to-black + cycle logic are reused unchanged.

iPhone/iPad editor (AppState) ──observe──▶ RemoteController ──RemoteMessage──▶ NWConnection
                                                                                    │
ATV: NWListener (Bonjour) ──accept──▶ AmbientHost ──apply──▶ AmbientController.apply(intent)
                                                          └──▶ AppState.edited = params

Components (small, focused, mostly shared)

  • Remote/RemoteMessage.swift (shared, host-testable): the Codable wire protocol. 
    enum RemoteMessage: Codable, Equatable {
    case hello(RemoteState)     // full sync on connect
    case intent(ControlIntent) // next / previous / jumpTo / togglePlaylist
    case params(PresetParams)  // live edited params (throttled)
}
struct RemoteState: Codable, Equatable { let selectedID: String; let params: PresetParams; let cycling: Bool }
    Requires adding Codable to ControlIntent (currently Equatable; synthesizable). PresetParams is already Codable.
  • Remote/RemoteLink.swift (shared): wraps an NWConnection; sends/receives length-prefixed (UInt32 + JSON) RemoteMessages; a receive callback + a send(_:); runs on a dedicated queue, hops to main for delivery. Connection-state callbacks (ready/failed/cancelled).
  • Remote/RemoteDiscovery.swift (protocol + iOS impl): protocol RemoteDiscovering → surfaces a chosen NWConnection/endpoint. iOS/iPad impl wraps DeviceDiscoveryUI (DDDevicePickerViewController) filtered to our Bonjour service. (macOS NWBrowser impl later.)
  • Remote/AmbientHost.swift (tvOS): an NWListener advertising _physarum._tcp; on an incoming connection builds a RemoteLink, decodes messages and applies them — intent → AmbientController.apply, params → appState.edited, hello → all three. On connect: adopt the remote's state (its cycle state — typically off, so the ATV holds). On disconnect/cancel: resume autonomous cycling (T1).
  • Remote/RemoteController.swift (iOS/iPad): owns the discovery + RemoteLink; observes the local AppState (selectedID, edited, cycle) and sends messages — selection/intents immediately, params throttled to ~20 Hz (LAN-smooth, avoids flooding). Exposes a connection state for the UI.
  • iOS/iPad UI (RootView touch): a "Connect to Apple TV" affordance (toolbar/menu) that presents the DeviceDiscoveryUI picker, plus a small connection-status indicator. The editor itself is unchanged; when connected, its existing edits flow through RemoteController.

Data flow & lifecycle

  1. ATV launches → AmbientHost advertises the Bonjour service, keeps running T1 autonomously.
  2. User taps "Connect to Apple TV" on the phone → DeviceDiscoveryUI picker → picks the ATV → NWConnection.
  3. Phone sends .hello(currentState); thereafter .intent/.params on local changes (params throttled).
  4. ATV applies: selection → crossfade-to-black transition (reused), params → live setParams, cycle → toggle.
  5. Disconnect (user, or network loss) → ATV resumes T1 auto-cycle from the current preset; phone shows disconnected + can reconnect.

Concurrency

NWConnection/NWListener run on a dedicated dispatch queue; all AppState/AmbientController mutations hop to main (the model is main-only, like the existing tick path). Param decode + apply on main; throttle on the send side.

Entitlements / Info.plist (verify exactly during impl)

  • ATV + phone: NSLocalNetworkUsageDescription, NSBonjourServices = _physarum._tcp.
  • DeviceDiscoveryUI: declare the browsed service; the system picker handles user consent (no separate local-network prompt for the browse). Exact keys confirmed at implementation against current SDK.
  • project.yml: per-SDK Info.plist additions (tvOS advertises; iOS browses).

Testing & Verification

  • Host-unit (macOS): RemoteMessage Codable round-trip (each case, incl. ControlIntent cases + a PresetParams); AmbientHost's apply mapping (message → AmbientController/AppState effect) with an injected link; throttle logic pure-tested. ControlIntent Codable round-trip.
  • Device-empirical (sign-off, real ATV + phone, same network — no simulator): discover + pair via DeviceDiscoveryUI; selection/edits/prev-next/cycle mirror live to the ATV; disconnect → ATV resumes cycling; reconnect works. This is the real validation; the transport + DeviceDiscoveryUI can't be simulated.

Risks

  • DeviceDiscoveryUI specifics (entitlement/Info.plist/picker API) are the main unknown — pin down against the current SDK during impl; fall back to raw NWBrowser if it fights us (same NWConnection core).
  • Device-only validation: most of the value is verifiable only on hardware — host tests cover the protocol + apply logic; the network path is Johannes' device sign-off.
  • Param flood: throttle (~20 Hz) + send only on actual change; selection is a discrete event.
  • Security: local-network only, no auth in T2a (home LAN). A pairing/auth step is a later concern if needed.