English-language Playwright tests cheap enough for every PR

What to build

A GitHub Actions / CI add-on that lets solo devs and small teams write end-to-end test cases in plain English, auto-compiles them into Playwright scripts, and runs them cheaply on every pull request.

Solo founders and small teams want E2E test coverage without hand-writing or maintaining brittle Playwright code, and without paying for enterprise QA platforms — a natural-language-to-Playwright compiler aimed at every-PR CI runs fills that gap.

Demand

Non-technical founders, PMs, and small dev teams shipping fast with AI want test coverage they can author and read themselves, and want it cheap enough to run on every PR rather than reserved for nightly runs.

• Hacker News (Show HN post)News

  Show HN: Sedum – Playwright tests in English, cheap enough to run on every PR — 3 points, 1 comment on HN.

Stack

• Playwright
• OpenAI API or Claude API (spec-to-code compilation)
• GitHub Actions
• Postgres (test spec storage/versioning)
• GitHub App API (PR status checks)

Solo + AI difficulty

The Playwright codegen and CI wiring is straightforward with AI assistance; the hard part is making the English-to-script compiler reliable enough that non-technical reviewers trust it (handling ambiguous specs, flaky selectors, and cost control on LLM calls per PR). A usable MVP (GitHub Action + basic spec compiler) is realistic in 2-4 weeks solo; production-grade reliability and cost tuning will take longer.

Entry threshold

A solo builder with AI assistance can prototype an English-to-Playwright test compiler and CI runner in a few weeks using existing LLM APIs and the open Playwright framework; main effort is reliability of natural-language-to-test translation and keeping per-run costs low enough for frequent CI use.

Window

6-12 months

Where to find first users

• Show HN / Hacker News launch
• Product Hunt launch
• r/webdev and r/SaaS
• GitHub Marketplace listing for the Action

Competitors

• QA Wolf
• Testim
• Mabl

Counter-signals & risks

• Natural-language-to-code compilation layers (e.g., LLM-based translation) can introduce non-determinism or misinterpretation of test intent, risking false positives/negatives in CI gating.

• Low per-run cost claims are unverified at scale; LLM inference or compilation costs could rise significantly with complex test suites or high PR volume, eroding the 'cheap enough for every PR' value proposition.

• The market for AI-assisted E2E testing tools is crowded (e.g., existing Playwright codegen, QA Wolf, Testim, Mabl), making differentiation and retention difficult for a solo/small-team-focused entrant.