docsOperate

Cutting a release

Only owners may cut a new version of MemQL, and the platform does it end to end: it computes the next version, creates the tag, and publishes the GitHub Release that starts the image build for every node type.

This is the manual half of an otherwise automatic pipeline. Nothing here builds images, pushes images, or goes near scripts/release/release.sh -- whose --push path remains break-glass.

The repo-root VERSION file is read, never written. VERSION equals the tag of the commit a cut tags (VERSIONING.md), so a pull request setting it lands on main first and the cut then refuses with version_file_stale if the file names anything but the version it computes. Section 5 has the order.

Design record: 2026-08-23-release-cut-automation-design.md.


1. What a cut actually does

Three things happen, in this order, and the order is load-bearing. Before any of them, the cut reads VERSION at main's head -- the exact commit it is about to tag -- and refuses unless the file reads the version it computed.

  1. The tag is created at the current head of main. GitHub's ref-create is atomic, which makes this the concurrency gate for the whole feature: two owners cutting at the same moment produce one tag and one ref_exists refusal. There is no lock anywhere because none is needed.
  2. A GitHub Release is published for that tag -- published, never a draft. Only release: published fires the cascade, so a draft would create a Release that builds nothing while looking exactly like success.
  3. The cascade runs. .github/workflows/dispatch-engine-images-on-release.yml (#2519) fires on the published Release and re-dispatches build-engine-images.yml with the bare version -- it strips the leading v, because git tags carry it and image tags do not (memql#4061). That workflow builds every node type as a product-agnostic image.

MemQL does step 1 and 2. CI does step 3. The platform records what it did on a v1:cluster:releaseCut row and writes a release_cut audit event beside it.

It cannot tell you the images exist. Publishing a Release means the build was asked for. Whether it finished is a question only the container registry answers, which is what releaseCutStatus is for -- see section 6.


2. Before a cut works: two values to seed

The engine is product-agnostic and carries no repository default. An installation that cuts releases says which repository it cuts, and supplies a credential. Until both are seeded, a cut refuses with release_repo_unconfigured or credential_unavailable, naming the variable to seed and never a value.

MEMQL_RELEASE_REPO -- a global variable

The repository, in owner/name form. Not a URL, no .git suffix:

text
acme/widget

MEMQL_GITHUB_RELEASE_TOKEN -- a global secret

A fine-grained personal access token (or a GitHub App installation token) scoped to that one repository:

PermissionLevelNeeded for
ContentsRead and writecreating the tag and publishing the Release
Pull requestsRead and writeonly the optional extension pin-bump PR

Mint it at Settings -> Developer settings -> Personal access tokens -> Fine-grained tokens, with Repository access limited to the single repository. Give it the shortest expiry your release cadence tolerates.

The Pull requests scope is genuinely optional. A token holding Contents alone is a correct token: a cut works, and the pin-bump follow-on records a note saying it could not open a PR rather than failing the release.

Both are rows in scripts/secrets/manifest.yaml, so the secrets seeder writes them (go run ./scripts/secrets seed --env-file <path>, see adding a global secret); the setGlobalSecret / setGlobalVariable mutations write them directly, and environment values on the node work too. Resolution order is global secret -> global variable -> environment, secret first because the token is a credential; the environment tier exists for the bootstrap window after make up-refresh, when concept storage is empty.


A leaked MEMQL_GITHUB_RELEASE_TOKEN can tag and publish in the one repository it is scoped to. That is bounded by the fine-grained scope, by encryption at rest, by owner-only reach, and by the audit row -- and you can bound it further on GitHub's side.

Add a tag protection rule for v* (Settings -> Rules -> Rulesets, targeting tags), restricting tag creation to the identities that should be cutting releases. A token that leaks then cannot create a release tag at all.

Do this before you seed the token, not after.


4. First run: a dry run

A dry run is how you validate a freshly seeded credential, and how you see the plan before cutting for real:

text
builtin releaseCut(bump: "patch", dryRun: true)

It computes the plan -- the next version and the base sha -- and creates nothing, publishes nothing, writes nothing. It exercises the real credential against the real API, so it is a genuine test of the setup rather than a simulation of one.

A successful dry run tells you four things: the token can read the repository, the repository name is right, the version arithmetic found your existing tags, and VERSION at main's head already reads the version the cut would create.

A dry run stops before the first write. It runs every check that comes before the tag is created and none of the ones that come with the writes, so it cannot prove the token may create tags or Releases: a token with only Contents: read passes a dry run, and the real cut then refuses with credential_unavailable when GitHub rejects the tag. ref_exists (somebody cut the same version in between), a github_unreachable on a write and tag_created_release_failed also appear only on a real cut.

Between cuts, the dry run answers version_file_stale, and that is expected. VERSION reads the release main was last cut at until the prepare pull request for the next one merges (section 5), so until then the call above is refused with version_file_stale. The refusal comes after the token has listed the tags, read main's head and read VERSION, and its message names the version the cut computed and the sha it would tag. So it still proves the credential can read the repository, that the repository name is right, and that the arithmetic found your tags. What it cannot tell you yet is whether the cut would go through; that waits for the prepare pull request.


5. Cutting

First: set VERSION by pull request

VERSION equals the tag of the commit a cut tags, and main refuses direct pushes, so the value arrives the way every other change does:

  1. Open a pull request whose one change sets VERSION to the version you are about to cut -- 0.24.0, bare, no leading v. A minor release also records in releasedForms (deprecation_window_version_test.go) every deprecated form whose warning it is the first to carry; that test says so when it applies.
  2. Merge it.
  3. Cut at once, with the bump that reaches that version. A change merged in between moves main's head, and the cut tags the new head -- which still carries the right VERSION, but is not the commit you reviewed.

A cut whose computed version differs from VERSION is refused with version_file_stale before anything is created, and so is its dry run. The refusal names both values. It is what stops the lag that shipped docs bundle 0.21.25 labelled as engine 0.15.0: VERSION read 0.15.0 at every tag from v0.16.1 to v0.21.25.

Then: the cut

There is no UI control for a cut today. The browser card that drove it was retired in epic memql#4984, and neither MemQL OS nor the VS Code extension replaced it. A cut is the releaseCut builtin, called by an owner:

text
builtin releaseCut(bump: "patch", notes: "Why this cut", bumpExtensionPin: true)

or from an SDK: releaseCut({ bump: "patch" }) on the TS QueryClient, ReleaseCut in sdk/go/client. The engine refuses a non-owner in Go before any network request is made.

The arguments:

  • bump -- patch, minor or major; major and minor zero the parts below them. The next version is computed from the repository's newest vX.Y.Z TAG, not from any row here: a release cut by hand creates a tag this cluster never hears about, so the tag is the truth for "newest" and the rows are only what this installation did.
  • notes -- optional. Notes are prepended to GitHub's generated release notes rather than replacing them, so a sentence about why you cut sits above the generated list of changes.
  • bumpExtensionPin -- also open the pin-bump pull request (section 8).
  • dryRun -- compute the plan and create nothing (section 4).

The call asks for no confirmation, and a release is not undoable from here: reversing one means deleting the tag and the Release on GitHub by hand. Run the dry run first and read its plan.


6. Afterwards: check the images

releaseCutStatus checks one cut version:

text
builtin releaseCutStatus(version: "v0.19.10")

(releaseCutStatus({ version }) on the TS QueryClient, ReleaseCutStatus in sdk/go/client). It asks the container registry for the manifests of a representative node-image set at the bare version, and gives one of three answers:

AnswerWhat it means
Every image is publishedthe row moves to images_available
Still building, cut N minutes agothe row stays dispatched; the missing images are named
The check could not tellthe registry errored; the status is unchanged and nothing is guessed

The third answer is the point of the design. A workflow can fail after the Release publishes, so only the registry knows whether a version is deployable -- and a registry that cannot be reached knows nothing either way. Reporting that as "not built" would call a good release broken; reporting it as "built" would call a failed build deployable.

The check is on demand. There is no poller and no schedule.


7. What each refusal means

CodeWhat happenedWhat to do
release_repo_unconfiguredno repository configured, or GitHub cannot see itseed MEMQL_RELEASE_REPO; check the token's repository access
credential_unavailableno token, or GitHub rejected it (401/403)seed or re-mint MEMQL_GITHUB_RELEASE_TOKEN with Contents: read/write
github_unreachabletransport failure or a 5xx. Nothing was createdretry; check GitHub's status
ref_existsthe computed tag already existssomeone else cut it, or it was cut by hand. Run the dry run again and cut if you still need to
already_released_at_headmain's head already carries a release tagland a change first. Cutting again would publish a second version of identical code
version_file_staleVERSION at main's head is missing, or does not read the version this cut would create. Nothing was createdthe message says which case it is. VERSION still names a release that is already cut: the prepare pull request has not merged, so merge one setting VERSION to the version you mean to cut (section 5) and cut again -- and if you were only validating a credential, this is the expected answer (section 4). VERSION names a newer release that a different bump reaches: cut with that bump. Anything else: fix VERSION by pull request
no_release_tagsthe repository has no vX.Y.Z tag at allcreate the first tag and Release by hand. The first version of a repository is one a human chooses; releaseCut takes over after that
tag_created_release_failedhalf done -- see belowact; nothing is building
version_not_cutreleaseCutStatus was asked about a version with no row hereit was cut by hand or on another installation. There is no row to move
registry_check_failedthe image check itself erroredthe status is unchanged. Retry later
not_ownerthe caller does not hold the owner roleask a cluster owner. The check runs before any network request, so nothing was created
invalid_bumpthe bump was not major/minor/patchpass one of the three

The half-done state

tag_created_release_failed means the tag exists on the repository and the Release does not. The cascade never fired, so nothing is building, and the tag is invisible to everyone except this cluster's history.

Two ways out, both by hand on GitHub:

  • publish a Release for that tag, which starts the build; or
  • delete the tag, which undoes the cut.

Until you do one of them, the row keeps saying so and the next cut of the same bump will refuse with ref_exists naming that tag.


8. The extension pin-bump follow-on

With bumpExtensionPin: true, a successful cut also opens a pull request bumping editors/vscode/src/install/stackPin.ts's DEFAULT_STACK_TAG to the new tag -- the release an install checks out when it is not told otherwise.

A pull request and never a push: main refuses direct pushes (a repository ruleset, not a convention), so the change goes through review and the merge queue like any other. That is the right shape for a value this consequential, not a workaround.

It can never fail the cut. By the time it runs the Release is published and the build is going. If the token lacks the Pull requests scope, or the branch exists, or the constant has moved, the reason is recorded as a note on the release row and the cut still reports success -- because reporting a shipped release as failed would invite you to cut again, producing a second version of the same code.


9. What this deliberately does not do

  • No scheduled cuts. The construct exists for something to call; no automation is seeded and none should be. An unattended release with nobody watching is not wanted.

  • No image building or pushing. The CI cascade owns that.

  • No writing VERSION, and no touching release.sh or the dispatch workflows. VERSION is read and checked; the value itself reaches main through a reviewed pull request, because main takes no push.

  • No cutting other repositories. The bundle and client repos have their own dispatch paths; the repository variable is singular on purpose.

  • No workflow-run mapping. The Actions API does not expose a run's dispatch inputs, so matching a run to a version is a guess. The registry is the truth an operator actually needs.

    This is still true, and the releaseEngine capability below does not contradict it. That capability asserts something strictly weaker and checkable -- a build was dispatched after this release was published -- which is what a bridge firing looks like and what its not firing does not. It never claims the run it finds is building that exact version.


10. The same cut from a lifecycle automation

The releaseCut builtin above is one path. releaseEngine (scripts/release/release-engine.sh, capability release.engine) is the other: the deploy pack's action, for a lifecycle that cuts a version as a step rather than a person making the call.

It exists because of a failure the builtin cannot have and a script can:

A pushed git tag builds nothing. build-engine-images is workflow_dispatch-only and its single automatic trigger is a release: [published] event.

Thirteen tags between v0.16.0 and v0.19.7 carry no release, so every 0.19.x image came from a dispatch somebody remembered to run. A tag nobody dispatched for is a version that looks cut and cannot be deployed.

So the capability publishes a release, then waits for a build-engine-images run that postdates it, and fails when none appears. That second step is the point: the bridge is an event handler, and one that silently does not fire is indistinguishable from one that has not fired yet -- until the version is deployed and every pod lands in ImagePullBackOff.

Five behaviours worth knowing before using it:

SituationWhat happens
the release already exists, publishedidempotent; it still waits for the build
the release exists as a draftrefused. A draft emits no release: [published] event, so it builds no images while looking like a release in the UI
VERSION at the commit to be tagged is missing or names another versionrefused (exit 3, result.reason version_file_stale), dry run included, before anything is created -- the same rule as version_file_stale above. The commit checked is the existing tag, else targetSha, else the default branch's head read once and passed as --target, so the commit checked is the commit tagged. For a tag not yet created, a prepare pull request setting VERSION resolves it. For a tag that already exists it cannot: no merge moves a tag. Delete the tag and re-run at a commit whose VERSION reads the version (passed as targetSha, or the default branch's head once a prepare pull request has merged). A tag cut before VERSION had to equal the tag -- the unreleased v0.16.0 to v0.19.7 tags among them -- cannot pass where it stands
version carries a pre-release or build suffixrefused as a bad parameter (exit 2). VERSION is never suffixed and must equal the tag, so a suffixed release could pass only by breaking that rule; the console cut and the docs bundle accept bare X.Y.Z alone too
the build ran and failedreported (buildRunConclusion), not swallowed. The bridge fired, which is what this checks; a failed build is a different problem with a different fix
shell
scripts/release/release-engine.sh --version=0.24.0 --dryRun=true # verify credential + state, create nothing
scripts/release/release-engine.sh --version=0.24.0

The result reports the GHCR prefix as well as the tag. That half of the build is public and tenant-independent, and it is what made a full cloud bring-up possible without ever authenticating to the retired subscription -- an instance lifecycle should pull from there by digest and az acr import into its own registry, rather than assuming access to whichever ACR the build workflow targets.


11. Where the records live

  • v1:cluster:releaseCut -- append-only, one timeline per version, cluster-owner tier. The row id is the version.
  • v1:identity:auditEvent with action=release_cut -- who cut what, from which sha. The decisions log (the split), not the high-volume activity stream.

Read the history with query releaseCuts() as an owner.