Step 6 — Apply, watch it come up, go to production
Everything is GitOps except one command: telling the cluster that the new xplatformapp.yaml exists. After
that, every deploy is a click in GitLab.
6.0 Order matters
- The image tag
<sha>-stagingexists in the registry (step 4). Kargo needs something to pick up; until it finds one the values file saystag: initialand pods sit inImagePullBackOff. - The files from step 5 are pushed to
main. ArgoCD reads values frommain; another branch is invisible. - Then apply. Doing it the other way round just means a few minutes of red status while things catch up, but don't confuse yourself.
6.1 Apply the definition
You need the cluster kubeconfig and an Auth0 login; set that up once via Connecting to MySQL
(sections "Prerequisites" and "Log in"). Then, from your oym-k8s-cluster checkout on main:
git pull
./argocd/sync-apps.sh
The script re-applies every definition in the repo, which is harmless; your new one is the only change. If you get
Forbidden, ask DevOps to add you to the cluster admin group.
You'll need this command again whenever xplatformapp.yaml itself changes (typically a chartVersion bump).
Values files never need it; ArgoCD watches those on its own.
6.2 Watch it come up
Within a minute:
kubectl get xplatformapp <slug> # READY True once everything is generated
kubectl -n <slug>-kargo get warehouse,stage,freight # a Freight = Kargo found your image tag
Then open Kargo (https://kargo.k8s.instance.onyourmarks.tech, project <slug>-kargo). The staging stage
should show a promotion that ended in a commit on main called build: automatic update of <slug>-staging.
That commit is the cluster picking your image.
Then open ArgoCD (https://argocd.k8s.instance.onyourmarks.tech, app <slug>-staging). Wait for
Synced / Healthy. Meanwhile:
kubectl -n <slug>-staging get externalsecret,mysqlapp,pods
kubectl -n <slug>-staging logs -l app=app --tail=100
Finally:
curl -i https://<slug>.staging.k8s.instance.onyourmarks.tech/health
6.3 When it doesn't come up
Work top to bottom; each row is one layer.
| Symptom | Where to look | Cause and fix |
|---|---|---|
kubectl get xplatformapp shows nothing | 6.1 | The apply didn't happen or the file has a YAML error. Re-run sync-apps.sh and read its output. |
No Freight in <slug>-kargo, warehouse shows an error | Kargo UI → warehouse | No tag matching ^[0-9a-f]{40}-staging$ in registry.gitlab.com/onyourmarks/k8s/<slug>/app. Check step 4's registry view; check CI_PROJECT_NAME equals the slug. |
Freight exists but no commit on main | Kargo UI → stage → promotion | Promotion error is shown there. Usually the values file path: it must be k8s/app/platform/<slug>/helm/values-staging.yaml. |
ArgoCD app Unknown / ComparisonError | ArgoCD UI → app → conditions | Values file doesn't render: wrong chartVersion (not published), YAML typo, or a wrong chart name. Re-run the helm template check from step 5. |
externalsecret shows SecretSyncedError | kubectl -n <slug>-staging describe externalsecret app-secrets | Infisical: project slug not equal to <slug>, environment slug not staging/prod, or machine identity CI/CD not added (step 2). |
mysqlapp shows waiting for … key DATABASE_PASSWORD | kubectl -n <slug>-staging get mysqlapp database | The key named in passwordField isn't in Infisical for this environment, or the name differs. Add it; the user is created within a minute. |
Pods ImagePullBackOff | kubectl describe pod | Tag in the values file doesn't exist in the registry. Still initial: Kargo hasn't found a <sha>-staging tag yet, see the Freight row. Otherwise re-run the tag job. |
Pods CrashLoopBackOff | kubectl logs (add --previous) | App crashes on start: usually a missing env var. Compare .env.example with Infisical. |
Pods Init:Error | kubectl logs <pod> -c app-init | .container-init.sh failed (migration error). |
Pods Running but never Ready, ArgoCD stuck Progressing | kubectl describe pod → Events | Probes fail: wrong port, /health or /ready not answering 200 within 2s, or auth middleware blocking them. Step 1. |
503/default certificate in the browser | kubectl -n <slug>-staging describe ingress | Host in hosts:/tls: doesn't match the URL, or for custom domains DNS isn't pointing at 57.129.55.146 yet. |
| Site works but robots can index staging | values file | traefik-block-robots@kubernetescrd missing from the middlewares. |
Everything in that table lives in git or Infisical. Fix the source, push, and ArgoCD reconciles within a few minutes (the External Secret refreshes on every image bump and hourly otherwise).
6.4 Production
Same pipeline, one more click: open the pipeline whose staging build you are happy with and run
tag:image:production. Kargo commits build: automatic update of <slug>-production, ArgoCD rolls it out to
<slug>-production, and the tag job in GitLab turns green once ArgoCD reports healthy.
Before the first production promotion make sure Infisical prod is complete (step 2) and, for a custom domain,
DNS points at the cluster (step 5).
6.5 After go-live
- Every later deploy: push, wait for
build:image, click the tag job. Nothing in this repo changes by hand. - Chart upgrades: DevOps bumps
chartVersionacross all apps periodically and re-applies. If you do it for your own app, remember./argocd/sync-apps.shafterwards. - Database access from your laptop: Connecting to MySQL. Your database is
stg-<slug>/prod-<slug>. - Manual jobs and cron: component-tasks.
- Removing an environment or the app: delete the environment from
environments:(or the whole XPlatformApp) and re-apply; follow the teardown order in component-database so the MySQL user is cleaned up.
Done when
-
kubectl get xplatformapp <slug>isREADY True - Kargo Bot commit
build: automatic update of <slug>-stagingexists onmain - ArgoCD
<slug>-stagingisSynced/Healthy -
https://<slug>.staging.k8s.instance.onyourmarks.techserves the app - Production tagged and healthy when you are ready