feat(merge): require explicit e2e scope on merge_pull_request, enforce per-target policy #13

Closed
opened 2026-07-10 13:57:05 +00:00 by Phil · 1 comment
Owner

Problem

E2E behavior on deploy is driven by a free-form token in the merge commit message ([skip e2e] / [e2e: <pattern>]), parsed by the parse-selectors job in flow.raven deploy-{staging,prod}.yml. Both deploy workflows are opt-out: E2E runs by default unless [skip e2e] is present.

The recurring failure is behavioral: the agent forgets the token or emits the wrong scope. Consequences:

  • Unintended full/smoke runs occupy the staging-stack / prod-stack concurrency lock (cancel-in-progress: false) and stall the next deploy → routine manual cancels.
  • On the other side, a stray [skip e2e] on a master merge can silently skip the prod gate.

Flipping the workflow default (opt-out → opt-in) was considered and rejected: it only reroutes what "forgetting" defaults to, it does not force an intentional decision. The fix must force the decision where the agent actually makes it — the tool.

Fix — force the decision at the tool boundary

Per CLAUDE.md §7 (never commit directly to dev/master), every deploy-triggering push originates from a merge_pull_request call. That is the chokepoint. Add a required, no-default e2e parameter to merge_pull_request:

  • Omitting it → tool error, no merge. No silent fallback.
  • The MCP resolves the PR's base branch and enforces an allowed-value policy per target.
  • The MCP generates the exact commit-message token itself and injects it into the merge commit message — the agent never hand-writes the flag syntax (kills [e2e:auth] vs [e2e: auth] typos).

Per-target policy (by PR base branch)

Base allowed e2e values forbidden
dev (staging) skip · smoke · full · scoped:<pattern>
master (prod) full · scoped:<pattern> skip (tool rejects with a clear error)

Rationale: fast iteration on staging (skip allowed, smoke default), clean gate on prod (no skip possible; full or scoped only).

Coupled change in flow.raven (must land together)

The commit-message token vocabulary is a shared contract between this tool (emitter) and parse-selectors (parser). The parser + run-e2e composite action currently understand [skip e2e] and [e2e: <pattern>], and deploy-staging hardcodes suite: smoke. To support full on staging, parse-selectors needs to treat a reserved token (proposal: [e2e: full]suite=all) as a suite override. The implementer must read .forgejo/actions/run-e2e/action.yml + both deploy workflows first and pin the exact token↔behavior mapping before wiring, then open the matching flow.raven PR.

Proposed token mapping (verify against run-e2e semantics before committing):

  • e2e=skip → append [skip e2e] (dev only)
  • e2e=smoke → no token (staging default suite is smoke)
  • e2e=full → dev: [e2e: full] (reserved → suite=all); master: no token (prod default is already suite=all — do NOT emit [e2e: full], it would set tests=full and grep-match zero specs)
  • e2e=scoped:<pat>[e2e: <pat>]

Defense-in-depth (not the primary mechanism)

Keep the workflows opt-out as a backstop, and add a pre-push git hook in flow.raven that hard-rejects a raw git push to dev/master without an explicit E2E token — covers the one path this tool can't see (a forbidden-but-possible direct push). Track the hook in the flow.raven PR.

Acceptance criteria

  • merge_pull_request rejects the call when e2e is absent (no default).
  • e2e=skip on a master-based PR is rejected with an actionable error.
  • Tool resolves base branch correctly and injects the mapped token into the merge commit message.
  • Token vocabulary verified against run-e2e/action.yml + deploy-{staging,prod}.yml; documented in the tool's help text.
  • Companion flow.raven PR: parse-selectors understands the full-on-staging reserved token; pre-push hook backstop added.
  • End-to-end: a dev merge with e2e=smoke runs smoke; e2e=full runs the full suite on staging; a master merge with e2e=full runs the full prod suite; e2e=skip on master is impossible.

References

  • Failure background: flow.raven deploy-staging.yml / deploy-prod.yml parse-selectors + e2e jobs (both if: skip != 'true').
  • Depends on nothing — can ship independently of the Forgejo v16 upgrade (Philflow/flow.raven-deployment#267).
  • Related: #14 (this repo) — the v16 Actions REST tools work.
  • Downstream: Philflow/flow.raven-deployment#295 will later migrate the composite action this touches into a reusable workflow — keep the token vocabulary migration-safe (see coupling comment below).
## Problem E2E behavior on deploy is driven by a **free-form token in the merge commit message** (`[skip e2e]` / `[e2e: <pattern>]`), parsed by the `parse-selectors` job in flow.raven `deploy-{staging,prod}.yml`. Both deploy workflows are **opt-out**: E2E runs by default unless `[skip e2e]` is present. The recurring failure is behavioral: the agent forgets the token or emits the wrong scope. Consequences: - Unintended full/smoke runs occupy the `staging-stack` / `prod-stack` concurrency lock (`cancel-in-progress: false`) and stall the next deploy → routine manual cancels. - On the other side, a stray `[skip e2e]` on a master merge can silently skip the prod gate. Flipping the workflow default (opt-out → opt-in) was considered and **rejected**: it only reroutes what "forgetting" defaults to, it does not force an intentional decision. The fix must force the decision where the agent actually makes it — the tool. ## Fix — force the decision at the tool boundary Per CLAUDE.md §7 (never commit directly to dev/master), every deploy-triggering push originates from a `merge_pull_request` call. That is the chokepoint. Add a **required, no-default `e2e` parameter** to `merge_pull_request`: - Omitting it → tool error, no merge. No silent fallback. - The MCP resolves the PR's **base branch** and enforces an allowed-value policy per target. - The MCP **generates the exact commit-message token itself** and injects it into the merge commit message — the agent never hand-writes the flag syntax (kills `[e2e:auth]` vs `[e2e: auth]` typos). ## Per-target policy (by PR base branch) | Base | allowed `e2e` values | forbidden | |---|---|---| | `dev` (staging) | `skip` · `smoke` · `full` · `scoped:<pattern>` | — | | `master` (prod) | `full` · `scoped:<pattern>` | `skip` (tool rejects with a clear error) | Rationale: fast iteration on staging (skip allowed, smoke default), clean gate on prod (no skip possible; full or scoped only). ## Coupled change in flow.raven (must land together) The commit-message token vocabulary is a **shared contract** between this tool (emitter) and `parse-selectors` (parser). The parser + `run-e2e` composite action currently understand `[skip e2e]` and `[e2e: <pattern>]`, and deploy-staging hardcodes `suite: smoke`. To support `full` **on staging**, `parse-selectors` needs to treat a reserved token (proposal: `[e2e: full]` → `suite=all`) as a suite override. **The implementer must read `.forgejo/actions/run-e2e/action.yml` + both deploy workflows first** and pin the exact token↔behavior mapping before wiring, then open the matching flow.raven PR. Proposed token mapping (verify against run-e2e semantics before committing): - `e2e=skip` → append `[skip e2e]` (dev only) - `e2e=smoke` → no token (staging default suite is smoke) - `e2e=full` → dev: `[e2e: full]` (reserved → suite=all); master: **no token** (prod default is already suite=all — do NOT emit `[e2e: full]`, it would set `tests=full` and grep-match zero specs) - `e2e=scoped:<pat>` → `[e2e: <pat>]` ## Defense-in-depth (not the primary mechanism) Keep the workflows opt-out as a backstop, and add a **pre-push git hook** in flow.raven that hard-rejects a raw `git push` to dev/master without an explicit E2E token — covers the one path this tool can't see (a forbidden-but-possible direct push). Track the hook in the flow.raven PR. ## Acceptance criteria - [ ] `merge_pull_request` rejects the call when `e2e` is absent (no default). - [ ] `e2e=skip` on a `master`-based PR is rejected with an actionable error. - [ ] Tool resolves base branch correctly and injects the mapped token into the merge commit message. - [ ] Token vocabulary verified against `run-e2e/action.yml` + deploy-{staging,prod}.yml; documented in the tool's help text. - [ ] Companion flow.raven PR: `parse-selectors` understands the `full`-on-staging reserved token; pre-push hook backstop added. - [ ] End-to-end: a dev merge with `e2e=smoke` runs smoke; `e2e=full` runs the full suite on staging; a master merge with `e2e=full` runs the full prod suite; `e2e=skip` on master is impossible. ## References - Failure background: flow.raven `deploy-staging.yml` / `deploy-prod.yml` `parse-selectors` + `e2e` jobs (both `if: skip != 'true'`). - Depends on nothing — can ship independently of the Forgejo v16 upgrade (`Philflow/flow.raven-deployment#267`). - Related: `#14` (this repo) — the v16 Actions REST tools work. - Downstream: `Philflow/flow.raven-deployment#295` will later migrate the composite action this touches into a reusable workflow — keep the token vocabulary migration-safe (see coupling comment below).
Author
Owner

Coupling note — the composite action you edit gets replaced later (Philflow/flow.raven-deployment#295)

The flow.raven side of this work modifies parse-selectors + the staging suite wiring under the current composite shape (.forgejo/actions/run-e2e/action.yml). Be aware that Philflow/flow.raven-deployment#295 (blocked on the Forgejo v16 upgrade Philflow/flow.raven-deployment#267) will later delete that composite action and move it to a reusable workflow .forgejo/workflows/_e2e.yml.

Implications:

  • Keep the token vocabulary ([skip e2e], [e2e: full]→suite=all, [e2e: <pattern>], default) expressed in a way that's straightforward to carry into the reusable-workflow rewrite — ideally concentrated in parse-selectors + a clean suite/tests output contract, not scattered inline in the composite.
  • Philflow/flow.raven-deployment#295 has been annotated to preserve this vocabulary post-migration, but the cleaner your output contract here, the safer that migration is.
  • This does not block #13#13 ships first, under the composite shape. Just don't design the token handling in a way that only works inside the composite.
## Coupling note — the composite action you edit gets replaced later (`Philflow/flow.raven-deployment#295`) The flow.raven side of this work modifies `parse-selectors` + the staging `suite` wiring under the current **composite** shape (`.forgejo/actions/run-e2e/action.yml`). Be aware that **`Philflow/flow.raven-deployment#295`** (blocked on the Forgejo v16 upgrade `Philflow/flow.raven-deployment#267`) will later **delete that composite action and move it to a reusable workflow `.forgejo/workflows/_e2e.yml`**. Implications: - Keep the token vocabulary (`[skip e2e]`, `[e2e: full]`→suite=all, `[e2e: <pattern>]`, default) expressed in a way that's straightforward to carry into the reusable-workflow rewrite — ideally concentrated in `parse-selectors` + a clean `suite`/`tests` output contract, not scattered inline in the composite. - `Philflow/flow.raven-deployment#295` has been annotated to preserve this vocabulary post-migration, but the cleaner your output contract here, the safer that migration is. - This does **not** block `#13` — `#13` ships first, under the composite shape. Just don't design the token handling in a way that only works inside the composite.
Phil closed this issue 2026-07-10 20:35:53 +00:00
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Phil/forgejo-mcp#13
No description provided.