Native Audio-Reactivity Implementation Plan¶

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Port the web audio-reactivity feature to native (microphone-only, iOS + macOS) with full 10-target routing parity, driving the live animation.

Architecture: The native engine is already a driftfree port — the Metal shaders (agentStep/deposit/composite) read+apply all 10 audio factors (UniformInput.Audio 6 agent slots, PostUniformInput 4 post slots), currently fixed at 1.0. We add: a pure AudioAnalyzer (mic PCM → 3 bands via vDSP), a pure evaluateAudioFactors (bands × AudioReactive routes → 10 factors, port of computeAudioFactors), an AudioInput (AVAudioEngine mic + permission + lock-free publish), engine seams that make the factors settable, per-frame wiring in RenderCoordinator, and an editor section. No shader changes. Audio modulates only the live view; capture/export stays neutral/deterministic.

Tech Stack: Swift 5 / SwiftUI, AVFoundation (AVAudioEngine), Accelerate (vDSP.FFT), Metal (PhysarumEngine), XcodeGen, XCTest.

Spec: docs/superpowers/specs/2026-06-04-native-audio-reactivity-design.md

Reference (web, for numeric parity): src/audio/bands.ts (band edges 20-250/250-4k/4k-16k, dB floor/ceil −90/−20), src/engine/Engine.ts computeAudioFactors (AUTO_GAIN_TARGET 0.7, EMA 0.005, PEAK_FLOOR 0.05, MAX 20; per-route factors[target] += level·depth·effGain).

File Structure¶

App (apple/PhysarumApp/): - Create Sources/Audio/AudioBands.swift — AudioBands value type. - Create Sources/Audio/AudioFactors.swift — AudioFactors (keyed by ModTarget), AutoGainState, evaluateAudioFactors, agent/post adapters. - Create Sources/Audio/AudioAnalyzer.swift — pure bandLevel + AudioAnalyzer (vDSP FFT pipeline + smoothing). - Create Sources/Audio/AudioInput.swift — @Observable AVAudioEngine mic capture, permission, lock-free band publish, meter levels. - Modify Sources/Player/RenderCoordinator.swift — own an AudioInput, per-frame evaluate + push factors, start/stop on enabled. - Modify Sources/Player/MetalView.swift + Sources/App/RootView.swift — thread the shared AudioInput. - Create Sources/Editor/AudioReactiveSection.swift — enable/autoGain/gain/meter + routes editor. - Modify Sources/Editor/EditorView.swift — add the section. - Modify project.yml — NSMicrophoneUsageDescription. - Tests: Tests/AudioFactorsTests.swift, Tests/AudioAnalyzerTests.swift.

Engine (apple/PhysarumEngine/): - Modify Sources/PhysarumEngine/Renderer/SimEngine.swift — settable audioFactors, used by writeParams. - Modify Sources/PhysarumEngine/Renderer/Renderer.swift — settable post audio factors, used by writePost.

Engine facts (verbatim, verified): ModTarget (Types.swift) = .moveSpeed,.turnSpeed,.depositAmount,.sensorDistance,.rhythmAmplitude,.chaosStrength,.exposure,.bloomIntensity,.saturation,.contrast. AudioBand = .bass,.mid,.high. AudioReactive{enabled:Bool, gain:Float, autoGain:Bool, routes:[AudioRoute]}, AudioRoute{source:AudioBand, target:ModTarget, depth:Float}. UniformInput.Audio{moveSpeedFactor,turnSpeedFactor,depositFactor,sensorDistanceFactor,rhythmAmplitudeFactor,chaosStrengthFactor} (all default 1). PostUniformInput{…, exposureFactor,bloomIntensityFactor,saturationFactor,contrastFactor} (all default 1).

PART A — Increment 4a: Analyzer + Evaluator (pure, tested)¶

Task 1: AudioBands + AudioFactors + evaluateAudioFactors¶

Files: - Create: apple/PhysarumApp/Sources/Audio/AudioBands.swift - Create: apple/PhysarumApp/Sources/Audio/AudioFactors.swift - Test: apple/PhysarumApp/Tests/AudioFactorsTests.swift

[ ] Step 1: Write the failing test

// apple/PhysarumApp/Tests/AudioFactorsTests.swift
import XCTest
@testable import Physarum
import PhysarumEngine

final class AudioFactorsTests: XCTestCase {
    private func reactive(enabled: Bool = true, gain: Float = 1, autoGain: Bool = false,
                          _ routes: [AudioRoute] = []) -> AudioReactive {
        AudioReactive(enabled: enabled, gain: gain, autoGain: autoGain, routes: routes)
    }

    func testDisabledYieldsAllNeutral() {
        var ag = AutoGainState()
        let f = evaluateAudioFactors(bands: AudioBands(bass: 1, mid: 1, high: 1),
                                     reactive: reactive(enabled: false), autoGain: &ag)
        for t in ModTarget.allCases { XCTAssertEqual(f[t], 1.0, accuracy: 1e-6) }
    }

    func testSingleRouteAdditiveOverOne() {
        var ag = AutoGainState()
        // bass=0.5, depth=2, gain=1, no autoGain → contribution 1.0 → factor 1+1 = 2.0
        let f = evaluateAudioFactors(
            bands: AudioBands(bass: 0.5, mid: 0, high: 0),
            reactive: reactive(gain: 1, [AudioRoute(source: .bass, target: .moveSpeed, depth: 2)]),
            autoGain: &ag)
        XCTAssertEqual(f[.moveSpeed], 2.0, accuracy: 1e-5)
        XCTAssertEqual(f[.turnSpeed], 1.0, accuracy: 1e-6)   // untouched target stays neutral
    }

    func testMultipleRoutesToSameTargetAccumulate() {
        var ag = AutoGainState()
        let f = evaluateAudioFactors(
            bands: AudioBands(bass: 0.5, mid: 0.5, high: 0),
            reactive: reactive([AudioRoute(source: .bass, target: .exposure, depth: 1),
                                AudioRoute(source: .mid, target: .exposure, depth: 1)]),
            autoGain: &ag)
        XCTAssertEqual(f[.exposure], 1.0 + 0.5 + 0.5, accuracy: 1e-5)
    }

    func testNegativeDepthSubtracts() {
        var ag = AutoGainState()
        let f = evaluateAudioFactors(
            bands: AudioBands(bass: 0.5, mid: 0, high: 0),
            reactive: reactive([AudioRoute(source: .bass, target: .contrast, depth: -1)]),
            autoGain: &ag)
        XCTAssertEqual(f[.contrast], 0.5, accuracy: 1e-5)
    }

    func testAutoGainBoostsQuietSignal() {
        var ag = AutoGainState()   // runningPeak starts 0
        let r = reactive(gain: 1, autoGain: true,
                         [AudioRoute(source: .bass, target: .moveSpeed, depth: 1)])
        // quiet bass 0.05; first frame runningPeak ≈ 0.005*0.05 ≈ 0.00025 → clamped to floor 0.05
        // autoGainMul = min(20, 0.7/0.05) = 14 ; contribution = 0.05*1*14 = 0.7 → factor 1.7
        let f = evaluateAudioFactors(bands: AudioBands(bass: 0.05, mid: 0, high: 0),
                                     reactive: r, autoGain: &ag)
        XCTAssertEqual(f[.moveSpeed], 1.7, accuracy: 1e-3)
    }

    func testAgentAndPostAdapters() {
        var f = AudioFactors()
        f[.moveSpeed] = 1.5; f[.bloomIntensity] = 2.0
        XCTAssertEqual(f.agentUniform.moveSpeedFactor, 1.5, accuracy: 1e-6)
        XCTAssertEqual(f.agentUniform.turnSpeedFactor, 1.0, accuracy: 1e-6)
        XCTAssertEqual(f.postBloomIntensity, 2.0, accuracy: 1e-6)
        XCTAssertEqual(f.postExposure, 1.0, accuracy: 1e-6)
    }
}

[ ] Step 2: Run to verify it fails

cd apple/PhysarumApp && xcodegen generate && xcodebuild -scheme Physarum -destination 'platform=macOS' test -only-testing:PhysarumAppTests/AudioFactorsTests 2>&1 | tail -25 Expected: FAIL to compile — AudioBands, AudioFactors, evaluateAudioFactors, AutoGainState undefined.

[ ] Step 3: Write AudioBands.swift

// apple/PhysarumApp/Sources/Audio/AudioBands.swift
import Foundation

/// Three normalized (0..1) frequency-band energies. Web parity: bass/mid/high.
struct AudioBands: Equatable {
    var bass: Float
    var mid: Float
    var high: Float
    static let zero = AudioBands(bass: 0, mid: 0, high: 0)
    var peak: Float { max(bass, max(mid, high)) }
}

[ ] Step 4: Write AudioFactors.swift

// apple/PhysarumApp/Sources/Audio/AudioFactors.swift
import Foundation
import PhysarumEngine

/// One multiplier per ModTarget (all default 1.0 = neutral). Port of the web
/// `AudioFactors = Record<ModTarget, number>` with FACTOR_ONES default.
struct AudioFactors: Equatable {
    private var values: [ModTarget: Float]
    init() { values = Dictionary(uniqueKeysWithValues: ModTarget.allCases.map { ($0, 1.0) }) }
    subscript(_ t: ModTarget) -> Float {
        get { values[t] ?? 1.0 }
        set { values[t] = newValue }
    }

    /// Agent-side 6 → the engine's UniformInput.Audio.
    var agentUniform: UniformInput.Audio {
        var a = UniformInput.Audio()
        a.moveSpeedFactor = self[.moveSpeed]
        a.turnSpeedFactor = self[.turnSpeed]
        a.depositFactor = self[.depositAmount]
        a.sensorDistanceFactor = self[.sensorDistance]
        a.rhythmAmplitudeFactor = self[.rhythmAmplitude]
        a.chaosStrengthFactor = self[.chaosStrength]
        return a
    }
    // Post-side 4 → set on the Renderer.
    var postExposure: Float { self[.exposure] }
    var postBloomIntensity: Float { self[.bloomIntensity] }
    var postSaturation: Float { self[.saturation] }
    var postContrast: Float { self[.contrast] }
}

/// Carries the auto-gain running peak across frames (EMA). Web parity.
struct AutoGainState { var runningPeak: Float = 0 }

private enum AudioGain {
    static let target: Float = 0.7
    static let ema: Float = 0.005
    static let peakFloor: Float = 0.05
    static let maxMul: Float = 20
}

/// Port of Engine.computeAudioFactors: advance the auto-gain EMA, then for each
/// route accumulate `level · depth · effGain` additively onto the target's factor
/// (which starts at 1.0). `hasAudio` mirrors the web's `this.audio !== null`.
func evaluateAudioFactors(bands: AudioBands, reactive: AudioReactive,
                          autoGain: inout AutoGainState, hasAudio: Bool = true) -> AudioFactors {
    if hasAudio {
        autoGain.runningPeak += AudioGain.ema * (bands.peak - autoGain.runningPeak)
    }
    let autoGainMul: Float = reactive.autoGain
        ? min(AudioGain.maxMul, AudioGain.target / max(autoGain.runningPeak, AudioGain.peakFloor))
        : 1
    let effGain = reactive.gain * autoGainMul

    var f = AudioFactors()   // all 1.0
    if reactive.enabled && hasAudio {
        for route in reactive.routes {
            let level: Float = route.source == .bass ? bands.bass
                             : route.source == .mid ? bands.mid : bands.high
            f[route.target] += level * route.depth * effGain
        }
    }
    return f
}

[ ] Step 5: Run to verify it passes

cd apple/PhysarumApp && xcodebuild -scheme Physarum -destination 'platform=macOS' test -only-testing:PhysarumAppTests/AudioFactorsTests 2>&1 | tail -25 Expected: TEST SUCCEEDED (6 tests). If testAutoGainBoostsQuietSignal is off, recompute by hand from the constants and adjust the expected value (the formula, not the impl) — the impl must match the web exactly.

[ ] Step 6: Commit

cd "$(git rev-parse --show-toplevel)"
git add apple/PhysarumApp/Sources/Audio/AudioBands.swift apple/PhysarumApp/Sources/Audio/AudioFactors.swift apple/PhysarumApp/Tests/AudioFactorsTests.swift
git commit -m "feat(app): audio band/factor types + route evaluator (web parity)"

(Do NOT git add the gitignored .xcodeproj.)

Task 2: AudioAnalyzer (vDSP FFT → bands)¶

Files: - Create: apple/PhysarumApp/Sources/Audio/AudioAnalyzer.swift - Test: apple/PhysarumApp/Tests/AudioAnalyzerTests.swift

[ ] Step 1: Write the failing test

// apple/PhysarumApp/Tests/AudioAnalyzerTests.swift
import XCTest
import Accelerate
@testable import Physarum

final class AudioAnalyzerTests: XCTestCase {
    /// Generate `n` samples of a sine at `freq` Hz, amplitude 1.0.
    private func sine(freq: Double, sampleRate: Double, n: Int) -> [Float] {
        (0..<n).map { Float(sin(2.0 * Double.pi * freq * Double($0) / sampleRate)) }
    }

    func testBandLevelMapsDbFloorCeilTo01() {
        // A flat spectrum at the dB ceiling (-20) over the bass range → ~1.0; at floor (-90) → 0.
        let fft = 1024, sr = 48000.0
        let ceil = [Float](repeating: -20, count: fft / 2)
        let floor = [Float](repeating: -90, count: fft / 2)
        XCTAssertEqual(bandLevel(db: ceil, sampleRate: sr, fftSize: fft, range: (20, 250)), 1.0, accuracy: 1e-4)
        XCTAssertEqual(bandLevel(db: floor, sampleRate: sr, fftSize: fft, range: (20, 250)), 0.0, accuracy: 1e-4)
    }

    func testSineEnergyLandsInExpectedBand() {
        let sr = 48000.0, n = 1024
        let analyzer = AudioAnalyzer(fftSize: n, smoothing: 0)   // no smoothing for a clean assertion
        let low = analyzer.analyze(samples: sine(freq: 120, sampleRate: sr, n: n), sampleRate: sr)
        XCTAssertGreaterThan(low.bass, low.mid)
        XCTAssertGreaterThan(low.bass, low.high)
        let high = analyzer.analyze(samples: sine(freq: 8000, sampleRate: sr, n: n), sampleRate: sr)
        XCTAssertGreaterThan(high.high, high.bass)
    }

    func testSilenceIsNearZeroAndClamped() {
        let sr = 48000.0, n = 1024
        let analyzer = AudioAnalyzer(fftSize: n, smoothing: 0)
        let b = analyzer.analyze(samples: [Float](repeating: 0, count: n), sampleRate: sr)
        for v in [b.bass, b.mid, b.high] {
            XCTAssertGreaterThanOrEqual(v, 0); XCTAssertLessThanOrEqual(v, 1)
            XCTAssertLessThan(v, 0.2, "silence should map near the dB floor")
        }
    }
}

[ ] Step 2: Run to verify it fails

cd apple/PhysarumApp && xcodegen generate && xcodebuild -scheme Physarum -destination 'platform=macOS' test -only-testing:PhysarumAppTests/AudioAnalyzerTests 2>&1 | tail -25 Expected: FAIL to compile — bandLevel, AudioAnalyzer undefined.

[ ] Step 3: Write AudioAnalyzer.swift

// apple/PhysarumApp/Sources/Audio/AudioAnalyzer.swift
import Foundation
import Accelerate

// Web parity (src/audio/bands.ts): band edges + dB floor/ceil.
private let BASS_HZ: (Double, Double) = (20, 250)
private let MID_HZ: (Double, Double) = (250, 4000)
private let HIGH_HZ: (Double, Double) = (4000, 16000)
private let DB_FLOOR: Float = -90
private let DB_CEIL: Float = -20

/// Average dB across the FFT-bin range for `range` Hz, mapped to 0..1.
/// Pure port of the web `bandLevel`. `db` is a half-spectrum (fftSize/2 bins).
func bandLevel(db: [Float], sampleRate: Double, fftSize: Int, range: (Double, Double)) -> Float {
    let binHz = sampleRate / Double(fftSize)
    let loBin = Int((range.0 / binHz).rounded(.down))
    let hiBin = Int((range.1 / binHz).rounded(.up))
    let lo = max(0, loBin), hi = min(db.count - 1, hiBin)
    guard hi >= lo else { return 0 }
    var sum: Float = 0
    for i in lo...hi { sum += db[i] }
    let avg = sum / Float(hi - lo + 1)
    let v = (avg - DB_FLOOR) / (DB_CEIL - DB_FLOOR)
    return min(1, max(0, v))
}

/// Stateful FFT analyzer: time-domain Float samples → 3 normalized bands.
/// Hann window → real FFT (vDSP) → magnitude → dB → bandLevel → per-band EMA smoothing.
final class AudioAnalyzer {
    private let fftSize: Int
    private let halfN: Int
    private let log2n: vDSP_Length
    private let fft: vDSP.FFT<DSPSplitComplex>
    private let window: [Float]
    private let smoothing: Float
    private var smoothed = AudioBands.zero

    init(fftSize: Int = 1024, smoothing: Float = 0.8) {
        self.fftSize = fftSize
        self.halfN = fftSize / 2
        self.log2n = vDSP_Length(log2(Float(fftSize)))
        self.fft = vDSP.FFT(log2n: log2n, radix: .radix2, ofType: DSPSplitComplex.self)!
        self.window = vDSP.window(ofType: Float.self, usingSequence: .hanningDenormalized,
                                  count: fftSize, isHalfWindow: false)
        self.smoothing = smoothing
    }

    /// Analyze one frame. Uses up to `fftSize` samples (zero-pads if fewer).
    func analyze(samples: [Float], sampleRate: Double) -> AudioBands {
        var input = [Float](repeating: 0, count: fftSize)
        let count = min(samples.count, fftSize)
        for i in 0..<count { input[i] = samples[i] }
        vDSP.multiply(input, window, result: &input)

        var realp = [Float](repeating: 0, count: halfN)
        var imagp = [Float](repeating: 0, count: halfN)
        var db = [Float](repeating: DB_FLOOR, count: halfN)
        realp.withUnsafeMutableBufferPointer { rp in
            imagp.withUnsafeMutableBufferPointer { ip in
                var split = DSPSplitComplex(realp: rp.baseAddress!, imagp: ip.baseAddress!)
                input.withUnsafeBytes {
                    vDSP.convert(interleavedComplexVector: Array($0.bindMemory(to: DSPComplex.self)),
                                 toSplitComplexVector: &split)
                }
                fft.forward(input: split, output: &split)
                var mags = [Float](repeating: 0, count: halfN)
                vDSP.absolute(split, result: &mags)
                // magnitude → dB (1e-9 floor to avoid log(0)); scale ≈ matches AnalyserNode range.
                let scale = 1.0 / Float(fftSize)
                for i in 0..<halfN {
                    let m = mags[i] * scale
                    db[i] = 20 * log10(max(m, 1e-9))
                }
            }
        }
        let raw = AudioBands(
            bass: bandLevel(db: db, sampleRate: sampleRate, fftSize: fftSize, range: BASS_HZ),
            mid: bandLevel(db: db, sampleRate: sampleRate, fftSize: fftSize, range: MID_HZ),
            high: bandLevel(db: db, sampleRate: sampleRate, fftSize: fftSize, range: HIGH_HZ))
        // Per-band EMA to mimic the web AnalyserNode smoothingTimeConstant feel.
        let s = smoothing
        smoothed = AudioBands(
            bass: s * smoothed.bass + (1 - s) * raw.bass,
            mid: s * smoothed.mid + (1 - s) * raw.mid,
            high: s * smoothed.high + (1 - s) * raw.high)
        return smoothed
    }
}

[ ] Step 4: Run to verify it passes

cd apple/PhysarumApp && xcodebuild -scheme Physarum -destination 'platform=macOS' test -only-testing:PhysarumAppTests/AudioAnalyzerTests 2>&1 | tail -25 Expected: TEST SUCCEEDED (3 tests). If the vDSP.convert(interleavedComplexVector:) call has a signature mismatch on this SDK, use the equivalent vDSP_ctoz with an explicit DSPComplex reinterpret of input; keep the math identical (window → split-complex → forward FFT → magnitude → dB). If the sine-band assertion is marginal, widen the assertion to the dominant-band check only (the dB-floor/ceil parity test is the strict one).

[ ] Step 5: Commit

cd "$(git rev-parse --show-toplevel)"
git add apple/PhysarumApp/Sources/Audio/AudioAnalyzer.swift apple/PhysarumApp/Tests/AudioAnalyzerTests.swift
git commit -m "feat(app): vDSP audio analyzer (PCM → bass/mid/high, web band parity)"

PART B — Increment 4b: Mic input + engine wiring¶

Task 3: Engine — settable audio factors (sim + post)¶

Files: - Modify: apple/PhysarumEngine/Sources/PhysarumEngine/Renderer/SimEngine.swift (writeParams 119-137) - Modify: apple/PhysarumEngine/Sources/PhysarumEngine/Renderer/Renderer.swift (writePost 154-168)

The shaders already consume UniformInput.Audio (6) and PostUniformInput factors (4); they're just always packed as 1.0. Make them settable.

[ ] Step 1: SimEngine — add a settable agent-factor struct used by writeParams

In SimEngine.swift, add a public stored property (near the other public vars):

/// Live audio-modulation factors for the 6 agent-side targets. Default neutral.
/// Set per frame by the render loop; consumed by `writeParams`.
public var audioFactors = UniformInput.Audio()

In writeParams() (after i.matrix = preset.matrixTuple, before let bytes = packParams(i)), add:

i.audio = audioFactors

[ ] Step 2: Renderer — settable post factors used by writePost

In Renderer.swift, add a public stored property:

/// Live audio-modulation factors for the 4 post-side targets. Default neutral.
public var audioPostFactors: (exposure: Float, bloomIntensity: Float, saturation: Float, contrast: Float) = (1, 1, 1, 1)

In writePost(_:) (after post.contrast = preset.contrast, before post.backgroundColor), add:

post.exposureFactor = audioPostFactors.exposure
post.bloomIntensityFactor = audioPostFactors.bloomIntensity
post.saturationFactor = audioPostFactors.saturation
post.contrastFactor = audioPostFactors.contrast

[ ] Step 3: Build the engine + run its tests

cd apple/PhysarumEngine && swift build 2>&1 | tail -4 && swift test 2>&1 | tail -4 Expected: build ok; 21 tests still pass (defaults are 1.0, so existing offscreen renders are byte-identical).

[ ] Step 4: Commit

cd "$(git rev-parse --show-toplevel)"
git add apple/PhysarumEngine/Sources/PhysarumEngine/Renderer/SimEngine.swift apple/PhysarumEngine/Sources/PhysarumEngine/Renderer/Renderer.swift
git commit -m "feat(engine): settable audio modulation factors (sim 6 + post 4)"

Task 4: AudioInput (AVAudioEngine mic + permission + publish)¶

Files: - Create: apple/PhysarumApp/Sources/Audio/AudioInput.swift - Modify: apple/PhysarumApp/project.yml (info.properties)

GPU/HW path — build-verified; live behavior verified by Johannes.

[ ] Step 1: Add the mic permission string to project.yml

Under targets.Physarum.info.properties (next to NSPhotoLibraryAddUsageDescription), add:

        NSMicrophoneUsageDescription: "Physarum uses the microphone so the animation can react to sound."

(No App-Sandbox entitlement is needed: the macOS build is ad-hoc/unsigned with no sandbox. If sandboxing is ever enabled, add a .entitlements file with com.apple.security.device.audio-input.)

[ ] Step 2: Write AudioInput.swift

// apple/PhysarumApp/Sources/Audio/AudioInput.swift
import Foundation
import AVFoundation
import os

/// Microphone capture + real-time band analysis. The tap runs on the audio
/// render thread (vDSP only, no allocation beyond the analyzer's reused buffers),
/// publishing bands into a lock-guarded slot the render loop pulls each frame.
/// `meterLevels` is a main-thread @Observable mirror for the UI meter.
@Observable
final class AudioInput {
    private let engine = AVAudioEngine()
    private let analyzer = AudioAnalyzer(fftSize: 1024, smoothing: 0.8)
    private let lock = OSAllocatedUnfairLock(initialState: AudioBands.zero)
    private(set) var running = false
    /// Throttled copy for the UI meter (updated on main).
    var meterLevels = AudioBands.zero
    private var meterTick = 0

    /// Lock-free-ish read for the render loop (single unfair-lock acquire).
    var latestBands: AudioBands { lock.withLock { $0 } }

    /// Idempotent. Requests permission on first enable; starts/stops the engine.
    func setEnabled(_ on: Bool) {
        if on { start() } else { stop() }
    }

    private func start() {
        guard !running else { return }
        requestPermission { [weak self] granted in
            guard let self, granted else { return }
            self.beginTap()
        }
    }

    private func beginTap() {
        guard !running else { return }
        #if os(iOS)
        let session = AVAudioSession.sharedInstance()
        try? session.setCategory(.record, mode: .measurement, options: [.mixWithOthers])
        try? session.setActive(true)
        #endif
        let input = engine.inputNode
        let format = input.inputFormat(forBus: 0)
        guard format.sampleRate > 0 else { return }
        let sr = format.sampleRate
        input.installTap(onBus: 0, bufferSize: 1024, format: format) { [weak self] buffer, _ in
            guard let self, let ch = buffer.floatChannelData else { return }
            let n = Int(buffer.frameLength)
            let samples = Array(UnsafeBufferPointer(start: ch[0], count: n))
            let bands = self.analyzer.analyze(samples: samples, sampleRate: sr)
            self.lock.withLock { $0 = bands }
            self.publishMeter(bands)
        }
        engine.prepare()
        do { try engine.start(); running = true } catch { running = false }
    }

    private func stop() {
        guard running else { return }
        engine.inputNode.removeTap(onBus: 0)
        engine.stop()
        running = false
        lock.withLock { $0 = .zero }
        Task { @MainActor in self.meterLevels = .zero }
    }

    private func publishMeter(_ bands: AudioBands) {
        meterTick &+= 1
        guard meterTick % 4 == 0 else { return }   // ~throttle the @Observable updates
        Task { @MainActor in self.meterLevels = bands }
    }

    private func requestPermission(_ done: @escaping (Bool) -> Void) {
        #if os(iOS)
        if #available(iOS 17.0, *) {
            AVAudioApplication.requestRecordPermission { ok in DispatchQueue.main.async { done(ok) } }
        } else {
            AVAudioSession.sharedInstance().requestRecordPermission { ok in DispatchQueue.main.async { done(ok) } }
        }
        #else
        switch AVCaptureDevice.authorizationStatus(for: .audio) {
        case .authorized: done(true)
        case .notDetermined:
            AVCaptureDevice.requestAccess(for: .audio) { ok in DispatchQueue.main.async { done(ok) } }
        default: done(false)
        }
        #endif
    }
}

[ ] Step 3: Regenerate + build (macOS + iOS sim)

cd apple/PhysarumApp && xcodegen generate \
  && xcodebuild -scheme Physarum -destination 'platform=macOS' build 2>&1 | tail -6 \
  && xcodebuild -scheme Physarum -destination 'generic/platform=iOS Simulator' build 2>&1 | tail -6

Expected: BUILD SUCCEEDED (both). If OSAllocatedUnfairLock is unavailable, use a small class wrapping os_unfair_lock or an NSLock; keep the single-acquire read semantics.

[ ] Step 4: Verify the plist key

cd apple/PhysarumApp && /usr/libexec/PlistBuddy -c "Print :NSMicrophoneUsageDescription" Resources/Info.plist Expected: prints the usage string.

[ ] Step 5: Commit

cd "$(git rev-parse --show-toplevel)"
git add apple/PhysarumApp/Sources/Audio/AudioInput.swift apple/PhysarumApp/project.yml
git check-ignore apple/PhysarumApp/Resources/Info.plist || git add apple/PhysarumApp/Resources/Info.plist
git commit -m "feat(app): AudioInput (AVAudioEngine mic + permission + band publish)"

Task 5: Wire audio into the render loop¶

Files: - Modify: apple/PhysarumApp/Sources/Player/RenderCoordinator.swift - Modify: apple/PhysarumApp/Sources/Player/MetalView.swift - Modify: apple/PhysarumApp/Sources/App/RootView.swift - Modify: apple/PhysarumApp/Sources/Model/AppState.swift (own the shared AudioInput)

[ ] Step 1: AppState owns the shared AudioInput

In AppState.swift, add (near the other stored props):

let audio = AudioInput()

[ ] Step 2: RenderCoordinator — hold the AudioInput + evaluate/push factors each frame

In RenderCoordinator.swift, add stored props:

var audioInput: AudioInput?
private var autoGain = AutoGainState()

Replace draw(in:) (124-130) with:

func draw(in view: MTKView) {
    let ds = view.drawableSize
    setField(targetFieldSize(viewWidth: Double(ds.width), viewHeight: Double(ds.height), format: format))
    applyAudio()
    sim.step()
    guard let desc = view.currentRenderPassDescriptor, let drawable = view.currentDrawable else { return }
    renderer.render(sim: sim, preset: params, into: desc, drawable: drawable, cover: cover)
}

private func applyAudio() {
    let ar = params.audioReactive
    audioInput?.setEnabled(ar.enabled)
    guard ar.enabled, let input = audioInput, input.running else {
        sim.audioFactors = UniformInput.Audio()                 // neutral
        renderer.audioPostFactors = (1, 1, 1, 1)
        return
    }
    let f = evaluateAudioFactors(bands: input.latestBands, reactive: ar, autoGain: &autoGain)
    sim.audioFactors = f.agentUniform
    renderer.audioPostFactors = (f.postExposure, f.postBloomIntensity, f.postSaturation, f.postContrast)
}

[ ] Step 3: Thread the AudioInput through MetalView

In MetalView.swift, add a property:

var audioInput: AudioInput? = nil

In sync(_:) (49-55), add (after c.format = format):

c.audioInput = audioInput

[ ] Step 4: Pass it from RootView

In RootView.swift, where MetalView(...) is constructed, add the argument:

MetalView(field: field, selectedPreset: state.selectedPreset, editedParams: state.edited,
          cover: cover, format: format, audioInput: state.audio)

[ ] Step 5: Regenerate + build (macOS + iOS sim) + app suite

cd apple/PhysarumApp && xcodegen generate \
  && xcodebuild -scheme Physarum -destination 'platform=macOS' build 2>&1 | tail -6 \
  && xcodebuild -scheme Physarum -destination 'generic/platform=iOS Simulator' build 2>&1 | tail -6 \
  && xcodebuild -scheme Physarum -destination 'platform=macOS' test 2>&1 | tail -10

Expected: BUILD SUCCEEDED (both); TEST SUCCEEDED (58 + AudioFactorsTests + AudioAnalyzerTests).

[ ] Step 6: Commit

cd "$(git rev-parse --show-toplevel)"
git add apple/PhysarumApp/Sources/Player/RenderCoordinator.swift apple/PhysarumApp/Sources/Player/MetalView.swift apple/PhysarumApp/Sources/App/RootView.swift apple/PhysarumApp/Sources/Model/AppState.swift
git commit -m "feat(app): drive live audio modulation from the render loop"

PART C — Increment 4c: Editor UI¶

Task 6: AudioReactiveSection — enable / autoGain / gain / meter¶

Files: - Create: apple/PhysarumApp/Sources/Editor/AudioReactiveSection.swift - Modify: apple/PhysarumApp/Sources/Editor/EditorView.swift

[ ] Step 1: Write AudioReactiveSection.swift (controls + meter; routes added in Task 7)

// apple/PhysarumApp/Sources/Editor/AudioReactiveSection.swift
import SwiftUI
import PhysarumEngine

struct AudioReactiveSection: View {
    @Bindable var state: AppState

    private var ar: AudioReactive { state.edited.audioReactive }

    var body: some View {
        VStack(alignment: .leading, spacing: Theme.Space.s3) {
            Toggle("Enabled", isOn: Binding(
                get: { state.edited.audioReactive.enabled },
                set: { state.pushUndo(); state.edited.audioReactive.enabled = $0 }))
                .font(Theme.Fonts.label)

            Toggle("Auto-gain", isOn: Binding(
                get: { state.edited.audioReactive.autoGain },
                set: { state.pushUndo(); state.edited.audioReactive.autoGain = $0 }))
                .font(Theme.Fonts.label)

            VStack(alignment: .leading, spacing: Theme.Space.s1) {
                HStack {
                    Text("Gain").font(Theme.Fonts.label)
                    Spacer()
                    Text(String(format: "%.2fx", ar.gain)).font(Theme.Fonts.value)
                        .foregroundStyle(Theme.Colors.fgSecondary)
                }
                Slider(value: Binding(
                    get: { Double(state.edited.audioReactive.gain) },
                    set: { state.edited.audioReactive.gain = Float($0) }),
                    in: 0.1...10,
                    onEditingChanged: { editing in if editing { state.pushUndo() } })
                    .tint(Theme.Colors.accent).controlSize(.small)
            }

            meter
        }
    }

    private var meter: some View {
        VStack(alignment: .leading, spacing: Theme.Space.s1) {
            meterBar("Bass", state.audio.meterLevels.bass)
            meterBar("Mid", state.audio.meterLevels.mid)
            meterBar("High", state.audio.meterLevels.high)
        }
    }

    private func meterBar(_ label: String, _ level: Float) -> some View {
        HStack(spacing: Theme.Space.s2) {
            Text(label).font(Theme.Fonts.eyebrow).foregroundStyle(Theme.Colors.fgTertiary)
                .frame(width: 32, alignment: .leading)
            GeometryReader { geo in
                ZStack(alignment: .leading) {
                    Capsule().fill(Theme.Colors.overlay)
                    Capsule().fill(Theme.Colors.accent)
                        .frame(width: geo.size.width * CGFloat(min(1, max(0, level))))
                }
            }
            .frame(height: 6)
        }
    }
}

[ ] Step 2: Add the section to EditorView

In EditorView.swift, after the PhysSection("Effects") { ... } line, add:

PhysSection("Audio Reactive") { AudioReactiveSection(state: state) }

[ ] Step 3: Regenerate + build (macOS + iOS sim)

cd apple/PhysarumApp && xcodegen generate \
  && xcodebuild -scheme Physarum -destination 'platform=macOS' build 2>&1 | tail -5 \
  && xcodebuild -scheme Physarum -destination 'generic/platform=iOS Simulator' build 2>&1 | tail -5

Expected: BUILD SUCCEEDED (both).

[ ] Step 4: Commit

cd "$(git rev-parse --show-toplevel)"
git add apple/PhysarumApp/Sources/Editor/AudioReactiveSection.swift apple/PhysarumApp/Sources/Editor/EditorView.swift
git commit -m "feat(app): audio editor section (enable/auto-gain/gain + live meter)"

Task 7: Routes editor (add/remove, source/target/depth)¶

Files: - Modify: apple/PhysarumApp/Sources/Editor/AudioReactiveSection.swift

[ ] Step 1: Add the routes UI

Append a routes editor to AudioReactiveSection.body (after meter):



odelineno-28-1">            routesEditor odelineno-29-1">    private var routesEditor: some View { VStack(alignment: .leading, spacing: Theme.Space.s2) { HStack { Text("Routes").font(Theme.Fonts.label) Spacer() Button { state.pushUndo() if state.edited.audioReactive.routes.count < 16 { state.edited.audioReactive.routes.append( AudioRoute(source: .bass, target: .moveSpeed, depth: 1)) } } label: { Image(systemName: "plus.circle") } .buttonStyle(.plain).foregroundStyle(Theme.Colors.accent) .disabled(state.edited.audioReactive.routes.count >= 16) } if state.edited.audioReactive.routes.isEmpty { Text("No routes — audio has no effect until you add one.") .font(Theme.Fonts.eyebrow).foregroundStyle(Theme.Colors.fgTertiary) } ForEach(Array(state.edited.audioReactive.routes.enumerated()), id: \.offset) { idx, _ in routeRow(idx) } } } private func routeRow(_ idx: Int) -> some View { VStack(spacing: Theme.Space.s1) { HStack { Picker("", selection: Binding( get: { state.edited.audioReactive.routes[idx].source }, set: { state.pushUndo(); state.edited.audioReactive.routes[idx].source = $0 })) { ForEach(AudioBand.allCases, id: \.self) { Text($0.rawValue.capitalized).tag($0) } } Picker("", selection: Binding( get: { state.edited.audioReactive.routes[idx].target }, set: { state.pushUndo(); state.edited.audioReactive.routes[idx].target = $0 })) { Section("Agent") { ForEach(agentTargets, id: \.self) { Text(label($0)).tag($0) } } Section("Post") { ForEach(postTargets, id: \.self) { Text(label($0)).tag($0) } } } Button { state.pushUndo() state.edited.audioReactive.routes.remove(at: idx) } label: { Image(systemName: "minus.circle") } .buttonStyle(.plain).foregroundStyle(Theme.Colors.fgTertiary) } Slider(value: Binding( get: { Double(state.edited.audioReactive.routes[idx].depth) }, set: { state.edited.audioReactive.routes[idx].depth = Float($0) }), in: -2...2, onEditingChanged: { editing in if editing { state.pushUndo() } }) .tint(Theme.Colors.accent).controlSize(.small) } .padding(.vertical, Theme.Space.s1) } private var agentTargets: [ModTarget] { [.moveSpeed, .turnSpeed, .depositAmount, .sensorDistance, .rhythmAmplitude, .chaosStrength] } private var postTargets: [ModTarget] { [.exposure, .bloomIntensity, .saturation, .contrast] } private func label(_ t: ModTarget) -> String { t.rawValue }

[ ] Step 2: Regenerate + build (macOS + iOS sim) + app suite

cd apple/PhysarumApp && xcodegen generate \
  && xcodebuild -scheme Physarum -destination 'platform=macOS' build 2>&1 | tail -5 \
  && xcodebuild -scheme Physarum -destination 'generic/platform=iOS Simulator' build 2>&1 | tail -5 \
  && xcodebuild -scheme Physarum -destination 'platform=macOS' test 2>&1 | tail -10

Expected: BUILD SUCCEEDED (both); TEST SUCCEEDED (suite unchanged count + audio tests).

[ ] Step 3: Commit

cd "$(git rev-parse --show-toplevel)"
git add apple/PhysarumApp/Sources/Editor/AudioReactiveSection.swift
git commit -m "feat(app): audio routes editor (source/target/depth, add/remove, undo)"


Task 8: Gate — sim + device¶
Files: none (verification).

[ ] Step 1: Build/install/launch on the iPhone simulator + screenshot

xcrun simctl boot 'iPhone 17 Pro' 2>/dev/null; open -a Simulator
cd apple/PhysarumApp && xcodebuild -scheme Physarum -destination 'platform=iOS Simulator,name=iPhone 17 Pro' build 2>&1 | tail -3
APP=$(xcodebuild -scheme Physarum -destination 'platform=iOS Simulator,name=iPhone 17 Pro' -showBuildSettings 2>/dev/null | awk '/ BUILT_PRODUCTS_DIR =/{d=$3}/ FULL_PRODUCT_NAME =/{n=substr($0,index($0,"=")+2)}END{print d"/"n}')
xcrun simctl install booted "$APP" && xcrun simctl launch booted de.jkaindl.physarum
sleep 4 && xcrun simctl io booted screenshot /tmp/audio-3g-sim.png

Expected: app launches; the editor's "Audio Reactive" section is present (open the editor to confirm). (The sim has no real mic input; meter stays flat — that's expected.)

[ ] Step 2: Headless deploy to the iPhone

UDID=1D819185-3910-5389-A9BE-C377B8010156
cd apple/PhysarumApp && xcodebuild -scheme Physarum -destination "platform=iOS,id=$UDID" -allowProvisioningUpdates build 2>&1 | tail -3
APP=$(xcodebuild -scheme Physarum -destination "platform=iOS,id=$UDID" -showBuildSettings 2>/dev/null | awk '/ BUILT_PRODUCTS_DIR =/{d=$3}/ FULL_PRODUCT_NAME =/{n=substr($0,index($0,"=")+2)}END{print d"/"n}')
xcrun devicectl device install app --device "$UDID" "$APP"

Expected: install succeeds.

[ ] Step 3: Hand-off note for Johannes (mic verification)

Document that the live audio behavior needs a real device + sound: enable audio (mic permission prompt appears), play music aloud, add a route (e.g. bass→moveSpeed depth 1.5, high→bloomIntensity depth 1), confirm the animation reacts and the meter moves; toggle auto-gain; disable → animation returns to neutral; confirm capture/export remains audio-free.

Verification Gate (whole feature)¶

[ ] Engine swift test green (21).
[ ] App suite green (58 + AudioFactorsTests + AudioAnalyzerTests).
[ ] macOS + iOS-sim BUILD SUCCEEDED.
[ ] Sim screenshot shows the Audio Reactive editor section.
[ ] Deployed to iPhone JK.
[ ] Johannes mic sign-off: permission prompt; music drives agent + post targets; meter moves; auto-gain + gain work; disable → neutral; capture stays audio-free.


Self-Review (against the spec)¶
Spec coverage: §2 mic-only/both platforms (Task 4 permission split) · 10-target parity (Task 1 AudioFactors all 10; Task 3 sim 6 + post 4 — revision: post via the existing PostUniformInput factor slots, NOT CPU-side as the spec drafted — the composite shader already reads them (composite.metal:103/106/125/129), so the in-shader path is cleaner and exact-parity; no shader change) · agent via in-shader slots (Task 3) · vDSP analysis + band parity (Task 2) · route evaluator parity incl. auto-gain constants (Task 1) · live-only modulation, capture neutral (Task 5 leaves CaptureService untouched) · permissions (Task 4) · threading lock-free publish (Task 4) · editor UI enable/autoGain/gain/meter/routes (Tasks 6-7) · tests (Tasks 1-2) · gate (Task 8).
Placeholder scan: none — real code/commands throughout.
Type consistency: AudioBands/AudioFactors/AutoGainState/evaluateAudioFactors consistent across Tasks 1,5. audioFactors: UniformInput.Audio (SimEngine) + audioPostFactors tuple (Renderer) consistent Tasks 3,5. AudioInput.setEnabled/latestBands/meterLevels/running consistent Tasks 4,5,6. state.audio consistent Tasks 5,6,7. state.edited.audioReactive mutations + pushUndo() consistent Tasks 6,7. ModTarget/AudioBand/AudioRoute/AudioReactive are the real engine types (PhysarumEngine).