Jenkins on Kubernetes ( 2024 )

HELM Chart Documentation:
https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins

Jenkins Helm Value file
https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/values.yaml

Pre-Requisites

• A Kubernetes cluster with admin privilege.
• Helm & Kubectl install at your local.
• Download Value file

Topic

• Value file setup for basic installation.
• Install Jenkins on K8 under the Jenkins namespace.
• Additional installation part.
• Agent configuration options & Pipeline examples.
• Advance value file and agent configuration

Introduction to value file

Download value.yaml from link and replace the below values.

 
## kubectl cluster-info | grep 'Kubernetes control plane'
## Update the url 
kubernetesURL: "https://******"

## Initial Password Setup.
controller:
 adminSecret: true
 adminUser: "jadmin"
 adminPassword: "kernel@123"

Install Jenkins on K8 under the Jenkins namespace

$ kubectl create namespace jenkin

$ helm repo add jenkins https://charts.jenkins.io 
$ helm repo update

$ helm search repo jenkinsci

  NAME              CHART VERSION APP VERSION DESCRIPTION
  jenkinsci/jenkins 4.12.0        2.426.2     Jenkins - Build great things at any scale! The ...

The helm upgrade command is typically used to upgrade an existing release. However, the --install argument ensures the release is created if it does not exist. This means helm upgrade --install creates and updates a release, removing the need to juggle installation and upgrade commands depending on whether or not the release exists.

The name of the release is future-jenkins, and the final argument jenkinsci/jenkins defines the chart to be installed.

The output looks something like this:

$ helm upgrade --install future-jenkins jenkins/jenkins --values values.yaml -n jenkin

Release "future-jenkins" does not exist. Installing it now.
NAME: future-jenkins
LAST DEPLOYED: Fri Feb 02 08:13:11 2024
NAMESPACE: jenkins
STATUS: deployed
REVISION: 1
NOTES:
1. Get your 'jadmin' user password by running:
  kubectl exec --namespace default -it svc/myjenkins -c jenkins -- /bin/cat /run/secrets/chart-admin-password && echo
2. Get the Jenkins URL to visit by running these commands in the same shell:
  echo http://127.0.0.1:8080
  kubectl --namespace default port-forward svc/myjenkins 8080:8080

3. Login with the password from step 1 and the username: admin
4. Configure security realm and authorization strategy
5. Use Jenkins Configuration as Code by specifying configScripts in your values.yaml file, see documentation: http:///configuration-as-code and examples: https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demos

For more information on running Jenkins on Kubernetes, visit:
https://cloud.google.com/solutions/jenkins-on-container-engine

For more information about Jenkins Configuration as Code, visit:
https://jenkins.io/projects/jcasc/


NOTE: Consider using a custom image with pre-installed plugins

It’s time to connect

kubectl --namespace jenkin port-forward svc/future-jenkins 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

http://localhost:8080/
adminUser: “jadmin”
adminPassword: “kernel@123”

Additional installation part

For private clusters, Jenkins plugin needs to be installed manually.

Comment below lines and set installLatestPlugins as false

  installPlugins:
  #    - kubernetes:4174.v4230d0ccd951
  #    - workflow-aggregator:596.v8c21c963d92d
  #    - git:5.1.0
  #    - configuration-as-code:1670.v564dc8b_982d0
  installLatestPlugins: false

Offline Jenkins Plugin Installation
https://kernelv5.medium.com/offline-jenkins-plugin-installation-33e4f41b3847

Note: Once the Helm installation is done, Jenkins won't function properly until you manually install the plugins because Jenkins Helm designed to work with some minimum settings comes from those plugins.

Agent configuration options & Pipeline examples.

Agent configuration can be done using UI or Helm

Login UI > Manage Jenkins > Clouds > Kubernetes > Pod Template > default.

Under Pod template, you can add multiple containers; the default is jnlp ( established connection to Jenkins controller).

Example 1: Let’s consider: Need a docker runtime for the pipeline.

Jenkins HELM Chart offer a default docker dind ( docker inside docker ), from value file uncomment under additionalContainers and remote the []

additionalContainers:
   - sideContainerName: dind
     image:
       repository: docker
       tag: dind
     command: dockerd-entrypoint.sh
     args: ""
     privileged: true
     resources:
       requests:
         cpu: 500m
         memory: 1Gi
       limits:
         cpu: 1
         memory: 2Gi

Or Using UI > Login
Scroll down > Add Container > Container Template
Name : dind
Docker Image : docker.io/docker:20.10-dind
Working Directory : /home/jenkins/agent
Environment : dockerd-entrypoint.sh

Advance > Run in privileged mode user ID and Group id 0

Add Environment Variable > Environment Variable ( Applicable for all additional containers )
JENKINS_URL : http://jenkins.jenkins.svc.cluster.local:8080 [ may vary depends on your setup]

Pipeline Examples 1 :

podTemplate {
    node("jenkins-agent") {
        stage('Run shell') {
           container('dind') {
            sh 'docker'
        }
        }
    }
}

jenkins-agent : It’s the pod template label.
Manage Jenkins > Clouds > Kubernetes > Pod Template > default > Labels
container : additional container name

Example 2: Let’s consider Needing a docker runtime for the pipeline but in your custom docker image.

Above settings ( Default agent jnlp + Dind ) and add another container ( docker cli installed to the container) and environment variable JENKINS_URL and below ( Container specific )

DOCKER_TLS_VERIFY = 1
DOCKER_HOST = tcp://localhost:2376
DOCKER_CERT_PATH = /certs/client

Inside pod they all can access each other using localhost. DOCKER_HOST connecting your custom container to docker dind runtime and you can get full fledge docker runtime inside your container.

>> Scroll down to the bottom > Volumes [ This volume setting is global for all containers under the pod template ]

podTemplate {
    node("jenkins-agent") {
        stage('Run shell') {
           container('custom_container') {
            sh 'docker'
        }
        }
    }
}

Note :::::::::

For additional URL path
- jenkinsUriPrefix: “/jenkins
Security Header required if you Jenkins behind proxy
- X-Forwarded-Host
- X-Forwarded-Port

An example of running behind Azure APIM
Apim URL : example.com
Jenkins Path : example.com/jenkins
Jenkin namespace : jenkins
Istio — DestinationRule & VirtualService

APIM -> K8 endpoint with header -> Istio traffic to SVC

APIM Configuration Sample.

<policies>
    <inbound>
        <set-header name="host" exists-action="override">
            <value>jenkins.jenkins</value>
        </set-header>
        <set-header name="X-Forwarded-Host" exists-action="override">
            <value>example.com</value>
        </set-header>
        <set-header name="app" exists-action="override">
            <value>jenkins.jenkins</value>
        </set-header>
        <base />
        <rewrite-uri template="@(context.Request.OriginalUrl.Path.Replace(context.Api.Path+"/jenkins",""))" copy-unmatched-params="true" />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>