Github actions selfhosted runner on kubernetes

Using Github’s CI tool Github Actions can be very useful for projects that are already hosted on Github. For public available projects you get unlimited build minutes and thats amazing. But for private repositories the number of build minutes is limited to the owners account. Even though 2000 build minutes are included in the free account plan, you might exceed this by having many repositories. Having a self hosted runner will give you an infite amount of build minutes.

More relevant can the self hosted runner be when you have specific build requirements such as building on M1 Apple Silicon, optimized for aarch64 or such. With the self hosted runner you have more control over OS and machine.

Setup cert manager

One requirement for the Github Self Hosted Runner is a kubernetes project called “cert manager”. It can be easily installed as a helm chart, however I prefer using ArgoC to manage my helm charts:

certmanager.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: certmanager
  namespace: argocd
  finalizers:
  - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: 'https://charts.jetstack.io'
    targetRevision: 1.8.2
    chart: cert-manager
    helm:
      values: |
        installCRDs: "true"            
  destination:
    server: {{ .Values.spec.destination.server }}
    namespace: certmanager
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true

Creating the Github Personal Access Token

In order to authenticate the self hosted runner against the github servers we need to issue a personal access token. You can do that at https://github.com/settings/tokens. You now have to decide whether the self hosted runner should be repository specific or organization specific.

For repository specific runner it is sufficient to give the “repo” scope.

If you want a self hosted runner for the whole organization you have to additionally grant the “admin:org” scope.

Creating the controller

Now we have to create the Github Actions Runner Controller. The easiest way is by using a helm chart. I use ArgoCD for me to handle that. In the helm chart values we must provide the Personal Access Token generated at the previous step.

github-actions-runner-controller.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: github-selfhosted-runner
  namespace: argocd
  finalizers:
  - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: 'https://actions-runner-controller.github.io/actions-runner-controller'
    targetRevision: 0.19.1
    chart: actions-runner-controller
    helm:
      values: |
        authSecret:
          github_token: ghp_T69u...
          create: true         
  destination:
    server: {{ .Values.spec.destination.server }}
    namespace: github-selfhosted-runner
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true

It can happen, that the CRDs are not correctly created by ArgoCD or Helm. There is a workaround here.

Creating the runner

The controller can now launch Github Actions Runners described in the CRDs. The easiest way is to create a runner deployment. The .yaml files for repository specific and organization specific differ by the spec attribute.

repo-runner-deployment.yaml

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: repo-runner-deployment
spec:
  replicas: 1
  template:
    spec:
      repository: my-username/my-repositoryname

orga-runner-deployment.yaml

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: orga-runner-deployment
spec:
  replicas: 1
  template:
    spec:
      organization: your-organization-name

You can create the fitting deployment.yaml by:

kubectl apply -f repo-runner-deployment.yaml

After that you should see a pod starting in kubernetes and after a while the self hosted runner will be visible on one of the following links:

https://github.com/organizations/your-organization/settings/actions/runners https://github.com/your-username/your-reponame/settings/actions/runners

Use the self hosted runner

To make your github action run on your self hosted runner you have to set the “runs-on” attribute to “self-hosted”.

.github/workflows/my-action.yaml

// ...
jobs:
  build:
    runs-on: self-hosted
    steps:
    // ...