Release Guide¶

This guide documents the complete release process for gbcms using git-flow workflow.

Pre-Release Checklist¶

Before starting a release, ensure:

• All CI checks pass on develop
• All features for the release are merged to develop
• No blocking issues in milestone
• The regression panel has passed its gate: the candidate against the previous release on HPC, every changed cell attributed

Version Locations¶

One version source per ecosystem: rust/Cargo.toml for the Python package and the extension (maturin converts X.Y.Z-dev.N to PEP 440 X.Y.Z.devN), and the Nextflow manifest for the pipeline's container image and banner. A release edits three files; the lock follows, and the rest is derived:

On develop the manifest names the last released image (images publish only on tags), so it trails the package version; at a release both equal X.Y.Z.

python scripts/release.py check --tag X.Y.Z

Release Workflow¶

gitGraph LR:
   commit id: "ongoing develop work"
   branch release/X.Y.Z
   commit id: "bump the version (Cargo.toml, manifest)"
   commit id: "update CHANGELOG.md"
   checkout main
   merge release/X.Y.Z id: "PR merged" tag: "X.Y.Z"
   checkout develop
   merge release/X.Y.Z id: "back-merge"

Step-by-Step Instructions¶

1. Create Release Branch¶

# From develop
git checkout develop
git pull origin develop

# Create release branch
git checkout -b release/X.Y.Z

2. Update the Version¶

Edit version in rust/Cargo.toml and manifest.version in nextflow/nextflow.config to X.Y.Z, refresh the lock, and check:

cd rust && cargo check && cd ..        # updates the gbcms_rs entry in Cargo.lock
python scripts/release.py check --tag X.Y.Z

3. Update CHANGELOG.md¶

Add new section at top. The text after the em dash becomes the GitHub Release title (X.Y.Z — summary), and the section body its notes:

## [X.Y.Z] - YYYY-MM-DD — short summary

### ✨ Added
- New feature description

### 🔧 Fixed
- Bug fix description

### 🔄 Changed
- Changes description

3b. Distill and archive cycle plans¶

A cycle plan (CYCLE_X.Y.Z_PLAN.md) is the cycle's working record on develop, and often the only place its measurements were written. Before the cut, distill it as an issue is distilled before closing (.agents/rules/git-conventions.md §Issues). Each decision, with the numbers that decided it, goes to the read-rule register or a proposal doc. Rejected ideas go to .agents/learnings/REJECTED.md, and open work to tickets. Then move the plan out of the repo root on the release branch, where grep still finds it, and point CONTINUITY.md there if it references the plan:

git mv CYCLE_X.Y.Z_PLAN.md .agents/archive/

The back-merge (step 10) carries the move to develop. The next cycle's plan is added to develop after the back-merge.

4. Run Pre-Release Checks¶

Refresh the Docker image's dependency lock first: the newest releases that satisfy pyproject.toml. Run latest-deps.yml by hand on the release branch first (Actions → Latest dependencies → Run workflow → release/X.Y.Z; its monthly runs test main): it tests those releases on every Python, and its summary shows what the refresh changes. Commit the lock with the release.

uv pip compile pyproject.toml --upgrade --generate-hashes --python-version 3.11 --python-platform x86_64-manylinux_2_28 -o docker/requirements.lock

The image installs exactly these versions (--require-hashes) and runs pip check, so a lock that no longer satisfies pyproject.toml fails the Docker build in CI.

# Python linting + type checking
ruff check src/ tests/
black --check src/ tests/
mypy src/

# Rust linting + unit tests
cd rust && cargo clippy --all-targets -- -D warnings && cargo test && cd ..

# Integration tests
pytest -v

5. Commit and Push¶

git add -A
git commit -m "chore: bump version to X.Y.Z"
git push origin release/X.Y.Z

6. Create PR: release/X.Y.Z → main¶

• Title: Release X.Y.Z
• Describe changes from CHANGELOG
• Wait for CI to pass

gh pr checks <PR>            # expect Tests jobs + Nextflow Lint, not just GitBook
gh run list --branch release/X.Y.Z

# verify the exact release SHA (no PR churn, no notifications)
gh workflow run test.yml --ref release/X.Y.Z
gh workflow run nextflow-lint.yml --ref release/X.Y.Z

# or re-fire the pull_request event so checks attach to the PR itself
gh pr close <PR> && gh pr reopen <PR>

7. Merge to main (creates tag)¶

After PR approval: - Merge commit (do NOT squash) to main - Create tag: git tag X.Y.Z && git push origin X.Y.Z

8. CI Release Pipeline¶

The tag triggers .github/workflows/release.yml:

1. Verify — scripts/release.py check --tag X.Y.Z; every other job waits for it
2. Build two wheels — cp310-abi3, manylinux_2_28_x86_64 (the stable ABI, so one wheel serves every CPython from 3.10; AlmaLinux 8, glibc 2.28: it installs on RHEL 8 and newer) and cp310-abi3, macosx_11_0_arm64 (Apple silicon) — plus an sdist (jobs linux, macos and sdist). The linux and macos jobs build with the local actions .github/actions/linux-wheel and .github/actions/macos-wheel; the Wheels workflow builds with the same actions on every change to the build and installs the wheels (the Linux one in Rocky Linux 8 and 9 containers, the macOS one on an Apple silicon runner): pip with binary wheels only, the suite, and a counting run whose output digests must match everywhere
3. Publish to PyPI (via maturin)
4. Build the Docker image on an amd64 and an arm64 runner (natively, no emulation), push each by digest, and join them (job docker-merge) into one multi-arch tag, ghcr.io/msk-access/gbcms:X.Y.Z and latest
5. Create the GitHub Release — title and notes from the CHANGELOG section, the wheel and sdist attached with SHA256SUMS, and a build-provenance attestation for each artifact (verify with gh attestation verify <file> --repo msk-access/gbcms)

The docs are not part of this workflow: deploy-docs.yml publishes them (via mike, as X.Y.Z / stable) when the release merge reaches main.

9. Check the GitHub Release¶

The github-release job creates the Releases page entry once PyPI and the image are published (or updates it, re-uploading the assets, if one already exists). Verify with gh release list: the new version should show Latest. If the job failed, re-run it from the Actions tab (failed jobs only). By hand, the page alone (without the assets, checksums and attestations, which only the job adds) is:

python scripts/release.py notes --tag X.Y.Z --title-out title.txt --notes-out notes.md
gh release create X.Y.Z --title "$(cat title.txt)" --notes-file notes.md --latest --verify-tag

10. Merge main back to develop¶

git checkout develop
git pull origin develop
git merge main
git push origin develop

10b. Move develop to the next dev version¶

Right after the back-merge, bump rust/Cargo.toml on develop to the next dev version (X.Y+1.0-dev.0), run cargo check, and add an empty ## [Unreleased] section. Leave the Nextflow manifest at X.Y.Z: it names the newest published image. Without the bump every dev build stamps the released version in its provenance line; the version check on develop fails until it is done.

# rust/Cargo.toml: version = "X.Y+1.0-dev.0"
cd rust && cargo check && cd ..
python scripts/release.py check --branch develop

11. Cleanup¶

# Delete local release branch
git branch -d release/X.Y.Z

# Delete remote release branch (optional)
git push origin --delete release/X.Y.Z

Hotfix Workflow¶

For critical production fixes:

# Create hotfix from main
git checkout main
git checkout -b hotfix/X.Y.Z

# Fix, commit; then set the version to X.Y.Z in rust/Cargo.toml (cargo check) and the
# Nextflow manifest, add a dated CHANGELOG section, and check
python scripts/release.py check --tag X.Y.Z
git add -A
git commit -m "fix: critical issue description"
git push origin hotfix/X.Y.Z

# PR to main, tag X.Y.Z, then merge back to develop (keeping develop's dev version)

Automation Scripts¶

git-flow-helper.sh¶

Interactive helper for git-flow operations:

./git-flow-helper.sh
# Options:
# 1) Create feature branch
# 2) Create release branch
# 3) Show git status
# 4) Cleanup merged branches

Makefile Targets¶

CI Workflows¶

Troubleshooting¶

PyPI Upload Fails¶

• Check if version already exists on PyPI (versions cannot be overwritten)
• Verify PYPI_TOKEN secret is set in GitHub repository

Docker Build Fails¶

• Check Dockerfile paths match the new folder structure
• Run python scripts/release.py check --tag X.Y.Z (the release workflow's first job)

Docs Build Fails¶

• Verify mkdocs-mermaid2-plugin is installed in workflow
• Check snippet paths are correct (relative to root)

Related¶

• Developer Guide — Setup, build commands, and project layout
• Contributing — Contribution workflow and code standards
• Testing Guide — Running and writing tests before a release
• Changelog — Version history