Establish canonical testing strategy with headless CI #2

New issue

Open

opened 2026-09-02 15:25:24 +02:00 by emidab · 0 comments

emidab commented

2026-09-02 15:25:24 +02:00

Owner

Summary

Establish the canonical testing strategy for Lullalink and prove it works in headless CI before building features. This is the priority: every future feature issue assumes this testing harness exists.

Also prepare the app and test infrastructure to be driven by an AI agent later — the AI must be able to take screenshots and navigate the app headlessly. This is about infrastructure readiness, not building an AI tool now.

What to do

Set up the unit/widget/integration test pyramid with consistent conventions
Adopt golden tests for widget/UI regression (widget tests, no physical device)
Add fake platform plugins so platform channels work in tests without hardware
Wire a headless CI pipeline (e.g. Forgejo Actions) that runs the full suite
Document the testing patterns in README / CONTRIBUTING so all future work follows them
AI agent readiness: make the app navigable and screenshot-able by an external agent without a physical device, using established automation/driver mechanisms (e.g. flutter drive/integration_test, semantic Keys on interactive widgets, stable widget structure, a documented way to capture screenshots headlessly). The goal is that a future AI tool can drive, observe, and capture the UI — no AI tool is implemented in this issue.

Acceptance criteria

flutter test runs the whole suite headlessly on CI
At least one golden test for a widget
At least one fake platform plugin demonstrated
CI job is green and blocks merge
Testing conventions documented and easy to copy for new features
Interactive widgets carry stable, documented Keys / ids usable by an automation driver
A headless screenshot capture path (e.g. integration_test / flutter drive) works on CI without a physical device
README/CONTRIBUTING documents how an external (AI) agent can navigate the app and capture screenshots

Notes

This is the stated first priority of the milestone — do it before feature work
Prefer established frameworks (golden_toolkit, mocktail, plugin_platform_interface, integration_test)
Everything must run in headless CI (no physical device, no GPU)
"AI readiness" = stable semantics + a screenshot/drive path, NOT an AI implementation

## Summary Establish the canonical testing strategy for Lullalink and prove it works in headless CI before building features. This is the priority: every future feature issue assumes this testing harness exists. Also prepare the app and test infrastructure to be driven by an AI agent later — the AI must be able to **take screenshots and navigate the app headlessly**. This is about infrastructure *readiness*, not building an AI tool now. ## What to do 1. Set up the unit/widget/integration test pyramid with consistent conventions 2. Adopt golden tests for widget/UI regression (widget tests, no physical device) 3. Add fake platform plugins so platform channels work in tests without hardware 4. Wire a headless CI pipeline (e.g. Forgejo Actions) that runs the full suite 5. Document the testing patterns in README / CONTRIBUTING so all future work follows them 6. **AI agent readiness**: make the app navigable and screenshot-able by an external agent without a physical device, using established automation/driver mechanisms (e.g. `flutter drive`/integration_test, semantic `Key`s on interactive widgets, stable widget structure, a documented way to capture screenshots headlessly). The goal is that a future AI tool can drive, observe, and capture the UI — no AI tool is implemented in this issue. ## Acceptance criteria - [ ] `flutter test` runs the whole suite headlessly on CI - [ ] At least one golden test for a widget - [ ] At least one fake platform plugin demonstrated - [ ] CI job is green and blocks merge - [ ] Testing conventions documented and easy to copy for new features - [ ] Interactive widgets carry stable, documented `Key`s / ids usable by an automation driver - [ ] A headless screenshot capture path (e.g. integration_test / flutter drive) works on CI without a physical device - [ ] README/CONTRIBUTING documents how an external (AI) agent can navigate the app and capture screenshots ## Notes - This is the stated first priority of the milestone — do it before feature work - Prefer established frameworks (golden_toolkit, mocktail, plugin_platform_interface, integration_test) - Everything must run in headless CI (no physical device, no GPU) - "AI readiness" = stable semantics + a screenshot/drive path, NOT an AI implementation