Compare commits

..
Author SHA1 Message Date
agent 7052b41f5e build: chromium, font-noto and postgresql-client for the browser gates
zegit's `browser tests` (chromedp + CDP virtual authenticator) and
`lighthouse` jobs were github.com-only, so when `ci` was routed to this
runner on 2026-08-31 they stopped running anywhere. Chromium 149 (Alpine
3.23) with font-noto lets both run here; psql lets the lighthouse job
create and drop its own database on ci-postgres instead of migrating the
shared maintenance database from under the integration suite's template.

Verified locally (docker build, amd64): chromium-browser 149.0.7827.53,
psql 18.6, 45 font files, node 24.18.1, still uid 1000.
2026-09-08 07:51:10 +02:00
markus ad19f03297 Merge pull request 'chore(ci): assert the busybox/coreutils layer, and record the derived audit' (#9) from claude/chore/derive-toolchain-assertion into master
build image / build (push) Skipped
Reviewed-on: #9
2026-08-31 19:00:56 +00:00
agent 47d5342dc4 chore(ci): assert the busybox/coreutils layer, and record the derived audit
The toolchain check's own comment said the quiet part: "the list only covers what
someone remembered to add". So the list was derived instead of remembered — every
command invoked from a `run:` block across zegit's Gitea-executed workflows and
composite actions was extracted and checked against this image.

Result: no missing apk package. Everything resolves to one of three sources —
baked in here, provided by busybox/coreutils, or downloaded per-run and
immediately version-checked (go via setup-go, buildctl, flux, cosign, syft,
goreleaser). The download path needs nothing from this image beyond curl and tar,
both already asserted.

What the audit did find is that the busybox/coreutils layer was entirely
unasserted. Dropping `coreutils` from the apk line would pass this check while
base64, mktemp, seq and sort silently vanished — and on this repo's own history,
their absence would be narrated as whatever the gate reaching for them happened
to be doing, not as a missing tool. That is how openssl became "signature FAILED
verification" and jq became "not a signed DSSE envelope".

Verified by executing the assertion, not by reading it: extracted the inner
script and ran it against a PATH with base64 deliberately absent. It printed
base64=MISSING and the outer `grep -q MISSING` failed the build. Against a full
PATH it reports zero MISSING.
2026-08-31 20:53:32 +02:00
markus 48d5ccebdc Merge pull request 'fix: combine per-arch tags with imagetools, not docker manifest' (#8) from claude/fix/push-image-manifest-list into master
Reviewed-on: #8
2026-08-31 12:46:48 +00:00
agent 30dabb5056 fix: combine per-arch tags with imagetools, not docker manifest
docker manifest create refuses a source that is itself a manifest list:
  ERROR: ...:<tag>-amd64 is a manifest list
With Docker containerd image store, docker build --platform linux/amd64
produces an OCI index rather than a plain manifest, so BOTH per-arch tags are
lists and the combine fails — after the slow multi-hundred-MB pushes have
already succeeded. The failure mode is nasty: the per-arch tags are published
and the PLAIN tag, the one both Deployments reference, is absent. Hit on
12.11.1-ci6, 2026-08-31.

buildx imagetools create accepts indexes as sources and combines registry-side,
so it works with either image store. Verification switches to imagetools
inspect and now asserts both linux/amd64 and linux/arm64 specifically, rather
than counting two distinct architecture strings.
2026-08-31 14:44:59 +02:00
5 changed files with 56 additions and 120 deletions

No files matched your search

+21
View File
@@ -109,6 +109,22 @@ jobs:
# missing tool, blocking v0.0.134 and v0.0.135. The list only covers what # missing tool, blocking v0.0.134 and v0.0.135. The list only covers what
# someone remembered to add, so add to it whenever a gate grows a new # someone remembered to add, so add to it whenever a gate grows a new
# shell dependency. # shell dependency.
#
# DERIVED once, 2026-08-31, rather than remembered: every command invoked
# from a `run:` block across zegit's Gitea-executed workflows and its
# composite actions was extracted and checked against this list. Result:
# no missing apk package. Everything resolved to one of three sources —
# baked in here; provided by busybox/coreutils; or downloaded per-run and
# immediately version-checked (go via setup-go, buildctl, flux, cosign,
# syft, goreleaser). The download path needs nothing from this image
# beyond curl and tar, both already asserted.
#
# What the audit DID find is that the busybox/coreutils layer was entirely
# unasserted, so dropping `coreutils` from the apk line above would pass
# this check while base64, mktemp, seq and sort silently vanished — and by
# the history above, their absence would be narrated as whatever the gate
# using them happened to be doing. They are asserted below now. They are
# cheap to check and the failure they prevent is expensive to read.
- name: Verify the image still contains the toolchain the gates need - name: Verify the image still contains the toolchain the gates need
if: ${{ steps.meta.outputs.push == 'true' }} if: ${{ steps.meta.outputs.push == 'true' }}
run: | run: |
@@ -129,6 +145,11 @@ jobs:
command -v tar >/dev/null && echo "tar=ok" || echo "tar=MISSING" command -v tar >/dev/null && echo "tar=ok" || echo "tar=MISSING"
command -v openssl >/dev/null && echo "openssl=ok" || echo "openssl=MISSING" command -v openssl >/dev/null && echo "openssl=ok" || echo "openssl=MISSING"
command -v jq >/dev/null && echo "jq=ok" || echo "jq=MISSING" command -v jq >/dev/null && echo "jq=ok" || echo "jq=MISSING"
# busybox/coreutils layer: used across zegit's gates for base64
# decoding of DSSE payloads, temp dirs, poll loops and sorting.
for t in base64 mktemp seq sort sed grep cut head wc uname date; do
command -v "$t" >/dev/null && echo "$t=ok" || echo "$t=MISSING"
done
[ -x "$(git --exec-path)/git-http-backend" ] \ [ -x "$(git --exec-path)/git-http-backend" ] \
&& echo "git-http-backend=ok" || echo "git-http-backend=MISSING" && echo "git-http-backend=ok" || echo "git-http-backend=MISSING"
' 2>&1) || { echo "::error::could not run $REF on $plat"; fail=1; continue; } ' 2>&1) || { echo "::error::could not run $REF on $plat"; fail=1; continue; }
-66
View File
@@ -1,66 +0,0 @@
name: probe registry from buildkitd
# TEMPORARY probe branch — not for merge.
#
# The first probe ran in the RUNNER pod. buildkitd is a different pod in a
# different namespace with its own NetworkPolicies, so those results cannot be
# transferred: an UNREACHABLE there may be the runner's policy, not routing.
# This runs the same probes THROUGH buildkitd, by building a stage whose RUN
# does the curling — that egresses from the buildkitd pod, which is the network
# position that actually has to push.
#
# It prints results and always exits 0: this is a measurement, not a gate.
on:
workflow_dispatch:
env:
BUILDKIT_HOST: tcp://buildkitd.buildkit-system.svc.cluster.local:1234
BUILDKIT_VERSION: v0.32.2
jobs:
probe:
if: ${{ github.server_url != 'https://github.com' }}
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Install buildctl
run: |
set -euo pipefail
case "$(uname -m)" in x86_64) arch=amd64;; aarch64) arch=arm64;; *) exit 1;; esac
mkdir -p "$GITHUB_WORKSPACE/.bin"
curl -fsSL "https://github.com/moby/buildkit/releases/download/${BUILDKIT_VERSION}/buildkit-${BUILDKIT_VERSION}.linux-${arch}.tar.gz" \
| tar -xz -C "$GITHUB_WORKSPACE/.bin" --strip-components=1 bin/buildctl
echo "$GITHUB_WORKSPACE/.bin" >> "$GITHUB_PATH"
"$GITHUB_WORKSPACE/.bin/buildctl" --version
- name: Probe from inside buildkitd
run: |
set -euo pipefail
work=$(mktemp -d)
{
echo 'FROM alpine:3.20'
echo 'RUN apk add --no-cache curl >/dev/null 2>&1 || true'
echo 'RUN set -u; \'
echo ' f="http=%{http_code} tls=%{ssl_verify_result} t=%{time_total}\\n"; \'
echo ' echo "== public name via cluster DNS =="; \'
echo ' curl -s -o /dev/null -w "$f" --max-time 8 https://gitea.langenoja.se/v2/ || echo UNREACHABLE; \'
echo ' echo "== resolves to =="; getent hosts gitea.langenoja.se || echo "(none)"; \'
echo ' echo "== istio-gateway external lb 10.40.0.10 =="; \'
echo ' curl -s -o /dev/null -w "$f" --max-time 8 --resolve gitea.langenoja.se:443:10.40.0.10 https://gitea.langenoja.se/v2/ || echo UNREACHABLE; \'
echo ' echo "== istio-gateway clusterip 10.43.1.114 =="; \'
echo ' curl -s -o /dev/null -w "$f" --max-time 8 --resolve gitea.langenoja.se:443:10.43.1.114 https://gitea.langenoja.se/v2/ || echo UNREACHABLE; \'
echo ' echo "== gitea http service in-cluster =="; \'
echo ' curl -s -o /dev/null -w "$f" --max-time 8 http://gitea-system-gitea-http.gitea-system.svc.cluster.local:3000/v2/ || echo UNREACHABLE; \'
echo ' echo "== gitea http service, headers =="; \'
echo ' curl -s -D- -o /dev/null --max-time 8 http://gitea-system-gitea-http.gitea-system.svc.cluster.local:3000/v2/ | head -5 || true'
} > "$work/Dockerfile"
cat "$work/Dockerfile"
buildctl build --progress=plain \
--frontend dockerfile.v0 \
--local context="$work" \
--local dockerfile="$work" \
--opt platform=linux/amd64 2>&1 | tee /tmp/probe.log | tail -80
echo "=== PROBE RESULTS ==="
grep -E "^#[0-9]+ [0-9.]+ " /tmp/probe.log | sed -E "s/^#[0-9]+ [0-9.]+ //" || echo "(no RUN output captured)"
echo "probe finished (results above, from buildkitd's network position)"
-47
View File
@@ -1,47 +0,0 @@
name: probe registry endpoints
# TEMPORARY probe branch — not for merge. Answers one question with evidence
# instead of inference: which registry endpoints are reachable from a pod inside
# this cluster, after 10.40.0.10 (the istio-gateway LoadBalancer EXTERNAL
# address) turned out to be connection-refused from the pod network while
# answering fine from a LAN workstation.
on:
workflow_dispatch:
jobs:
probe:
if: ${{ github.server_url != 'https://github.com' }}
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Probe candidate registry endpoints
run: |
set -u
fmt='http=%{http_code} tls=%{ssl_verify_result} t=%{time_total}\n'
echo "--- public name via cluster DNS (the path that works today) ---"
printf '%-46s ' 'gitea.langenoja.se:443'
curl -s -o /dev/null -w "$fmt" --max-time 8 https://gitea.langenoja.se/v2/ \
|| echo "UNREACHABLE"
echo "--- istio-gateway EXTERNAL lb ip (what #949 pinned) ---"
printf '%-46s ' '10.40.0.10:443'
curl -s -o /dev/null -w "$fmt" --max-time 8 \
--resolve gitea.langenoja.se:443:10.40.0.10 https://gitea.langenoja.se/v2/ \
|| echo "UNREACHABLE"
echo "--- istio-gateway CLUSTER ip (candidate) ---"
printf '%-46s ' '10.43.1.114:443'
curl -s -o /dev/null -w "$fmt" --max-time 8 \
--resolve gitea.langenoja.se:443:10.43.1.114 https://gitea.langenoja.se/v2/ \
|| echo "UNREACHABLE"
echo "--- gitea http service, in-cluster, plain http ---"
printf '%-46s ' 'gitea-system-gitea-http:3000'
curl -s -o /dev/null -w "$fmt" --max-time 8 \
http://gitea-system-gitea-http.gitea-system.svc.cluster.local:3000/v2/ \
|| echo "UNREACHABLE"
echo "--- what this pod resolves the public name to ---"
getent hosts gitea.langenoja.se || echo "(getent failed)"
+17 -1
View File
@@ -58,6 +58,19 @@ USER root
# validate-admission.sh and feature-lint.sh all shell out to # validate-admission.sh and feature-lint.sh all shell out to
# python3, and credentials-coverage parses manifests with yaml # python3, and credentials-coverage parses manifests with yaml
# #
# chromium, font-noto -> the browser gates (2026-09-08): zegit's `browser
# tests` (chromedp + a CDP virtual authenticator over the WebAuthn
# ceremony pages) and `lighthouse` (treosh/lighthouse-ci-action
# via chrome-launcher) were github.com-only jobs, so when `ci`
# was routed to this runner on 2026-08-31 they stopped running
# anywhere. chromedp finds `chromium`/`chromium-browser` on PATH;
# chrome-launcher honours CHROME_PATH. font-noto so text renders
# (an empty font set makes layout/contrast audits meaningless).
# postgresql-client -> `psql`, so the lighthouse job can CREATE/DROP its own
# database on the runner's ci-postgres (ZEGIT_TESTPG_DSN) instead
# of migrating the shared maintenance database from under the
# integration suite's template.
#
# Go is intentionally NOT baked in: actions/setup-go fetches the pinned version # Go is intentionally NOT baked in: actions/setup-go fetches the pinned version
# per-run (node + tar are present), so the image needs no rebuild on Go bumps. # per-run (node + tar are present), so the image needs no rebuild on Go bumps.
RUN apk add --no-cache \ RUN apk add --no-cache \
@@ -75,7 +88,10 @@ RUN apk add --no-cache \
python3 \ python3 \
py3-yaml \ py3-yaml \
openssl \ openssl \
jq jq \
chromium \
font-noto \
postgresql-client
# Match the runner pod's runAsUser (1000); keep the image non-root by default. # Match the runner pod's runAsUser (1000); keep the image non-root by default.
USER 1000 USER 1000
+18 -6
View File
@@ -99,15 +99,27 @@ for arch in amd64 arm64; do
docker push "$REPO:$TAG-$arch" || die "push failed for $arch" docker push "$REPO:$TAG-$arch" || die "push failed for $arch"
done done
docker manifest rm "$REPO:$TAG" >/dev/null 2>&1 || true # `docker manifest create` REFUSES a source that is itself a manifest list:
docker manifest create "$REPO:$TAG" --amend "$REPO:$TAG-amd64" --amend "$REPO:$TAG-arm64" # ERROR: gitea.langenoja.se/...:<tag>-amd64 is a manifest list
docker manifest push "$REPO:$TAG" # With Docker's containerd image store, `docker build --platform linux/amd64`
# produces an OCI index rather than a plain manifest, so both per-arch tags are
# lists and the combine step fails after the (slow, large) pushes have already
# succeeded — leaving the per-arch tags published and the plain tag absent,
# which is the one the Deployments reference. Seen on 12.11.1-ci6, 2026-08-31.
#
# `buildx imagetools create` accepts indexes as sources and does the combine
# registry-side, so it works with either image store.
docker buildx imagetools create -t "$REPO:$TAG" "$REPO:$TAG-amd64" "$REPO:$TAG-arm64" \
|| die "failed to combine the per-arch tags into $REPO:$TAG"
# Verify by reading it back, rather than trusting the push's exit code. # Verify by reading it back, rather than trusting the push's exit code.
echo "verifying the published manifest lists both architectures" echo "verifying the published manifest lists both architectures"
got=$(docker manifest inspect "$REPO:$TAG" | grep -o '"architecture": *"[a-z0-9]*"' | sort -u | wc -l) inspect=$(docker buildx imagetools inspect "$REPO:$TAG") \
[ "$got" -ge 2 ] || die "published manifest does not list two architectures" || die "published $REPO:$TAG cannot be read back"
docker manifest inspect "$REPO:$TAG" | grep -o '"architecture": *"[a-z0-9]*"' | sort -u got=$(echo "$inspect" | grep -o 'linux/[a-z0-9]*' | sort -u | grep -cE 'linux/(amd64|arm64)')
[ "$got" -ge 2 ] || die "published manifest does not list both architectures:
$inspect"
echo "$inspect" | grep -o 'linux/[a-z0-9]*' | sort -u
cat <<EOF cat <<EOF