Skip to content

Running smoke tests

Smoke tests for argocd-oc-apps are tricky to implement, as the repo is on GitHub but the smoke tests need to access ATLNZ's intranet to fetch images hosted on an internal docker repository (harbor). External services can't innitiate network requests into ATLNZ's intranet, due to the corporate firewall.

To get around these limitations, part of the smoke tests are run on Jenkins in ATLNZ's intranet. A technique using GitHub statuses on commits is used so a PR on GitHub can indicate a Jenkins smoke test needs to be run, and Jenkins can update the status on GitHub when it starts running a job and when it passes/fails.

A brief overview of how this is done:

  1. When a push request is created or modified for argocd-oc-apps, the GitHub workflow init-jenkins-smoke-tests-check triggers and (using GitHub's API) adds a check with the context/name jenkins/smoke-tests and description state=waiting-for-jenkins-job to the HEAD commit on the pull request.
  2. A Jenkins job (argocd-oc-apps-continuous-polling) runs on a schedule at ATLNZ, checking for argocd-oc-apps pull requests that are requesting a smoke test. It does this by calling GitHub's API to get all open pull requests for argocd-oc-apps, and then for each pull request, it calls GitHub's API to get all statuses on the HEAD commit. If the job finds a status with the context jenkins/smoke-tests, state pending, and description state=waiting-for-jenkins-job, then the job succeeds, else it fails.
  3. A 2nd Jenkins job (the actual smoke test job) argocd-oc-apps_smoke-tests triggers when a argocd-oc-apps-continuous-polling job succeeds.
  4. The 2nd Jenkins job, the actual smoke test job (argocd-oc-apps_smoke-tests), does the same check as argocd-oc-apps-continuous-polling to check if there is a argocd-oc-apps pull request that is requesting a smoke test, and gets the branch and HEAD commit SHA.
  5. argocd-oc-apps_smoke-tests exists early if it does not find a pull request that is requesting a smoke test.
  6. argocd-oc-apps_smoke-tests, when there is a valid pull request, immediately calls GitHub's API on the PR's HEAD commit to update the status with context jenkins/smoke-tests. It sets the status description to state=jenkins-job-running, as a lock, so other jobs do not process that PR too.
  7. argocd-oc-apps_smoke-tests sets up the local KinD based One Connect Kubernetes cluster.
  8. argocd-oc-apps_smoke-tests gets the One Connect IP address by querying the nginx-ingress-controller, and cat's to a file so it can be read in the test environment later.
  9. argocd-oc-apps_smoke-tests runs the serenity tests using the oneconnect-test-env.sh environment. oneconnect-test-env.sh runs a docker container with the --network=kind argument, to connect the test container to the same docker custom bridge network One Connect is in, in the KinD cluster. oneconnect-test-env.sh also runs the docker container with the --add-host local.oneconnect.st.atlnz.lc:$ONE_CONNECT_IP argument, to setup a /etc/hosts DNS entry to point the local.oneconnect.st.atlnz.lc URL at the IP address of the local One Connect app. The combination of these two arguments lets the serenity tests access One Connect running in KinD, without modifying anything on the host OS (like the host's /etc/hosts file).
  10. Finally, depending on whether something in the tests failed or everything passed, the argocd-oc-apps_smoke-tests Jenkins job sends a final API call to GitHub to update the PR HEAD commit status with context jenkins/smoke-tests, to state success or failure, and description to state=jenkins-job-success or state=jenkins-job-failed.
  11. The status description state=jenkins-job-running, state=jenkins-job-success, ..., and the state pending/success/failure can all be viewed in the checks area on a pull request, so the result of the Jenkins smoke tests can be seen on GitHub.