before we show how we can apply continuous delivery and deployment using Jenkins pipeline as a code , we need to refresh our mind about the difference between continuous integration vs delivery vs deployment as being show below :
What about releasing process and versioning in continuous delivery :
basically i am not big fan of maven release plugin for some reasons :
Let’s consider the principles of Continuous Delivery.
- “Every build is a potential release”
- “Team need to eliminate manual bottlenecks”
- “Team need to automate wherever possible”
Current drawbacks and limitations of maven release way :
- Overhead. The maven-release plugin runs 3 (!) full build and test cycles, manipulates the POM 2 times and creates 3 Git revisions.
- Not isolated. The plugin can easily get in a mess when someone else commits changes during the release build.
- Not atomic. If something went wrong in the last step (e.g. during the artifact upload), we are facing an invalid state. We have to clean up the created Git tag, Git revisions and fix the wrong version in the pom manually.
So let us simplify the versioning process in an efficient way within out continuous delivery model :
- checking out the software as it is
- building, testing and packaging it
- giving it a version so it can be uniquely identified and tractable (release step)
Version=Major.Minor.Patch.GitCommit-Id
- deploying it to an artifact repository where it can then be picked for actual roll out on target machines
- tagging this state in SCM so it can be associated with the matching artifact
Which leads to :
- Every artifact is potentially shippable. So there is no need for a dedicated release workflow anymore!
- The delivery pipeline is significantly simplified and automated
- Traceable. It’s obvious, which commits are included.
So how we can implement continuous delivery and a conditional continuous deployment using Jenkins pipeline , off-course you can add more stages based into team process like for example performance test, just added till acceptance as production will be just the same :
we will use Jenkins pipeline as a code as it give more power to the team to control they delivery process:
- when a Git commit is triggered to development branch , it will trigger Jenkins pipeline to checkout the code , build it , unit test it and if all is green it will trigger the integration test plus sonar check for your code quality gate
- if all green , deploy to development server using the created package from Jenkins work space or Nexus Snapshots if you prefer
- then if there is a release candidate which means a merge request to your master branch , the it will do the same steps as above plus creating the release candidate using the release versioning explained above.
- Push the released artifact to Nexus and do acceptance deployment and auto acceptance functional testing
- Then prompt to production and if it is approved it will be deployed into production
- Off-course deployment task must be automated if you need continuous deployment using an automation tools like Ansible or other options.
Now let us explain how it will be done via Jenkins Pipeline code as the following :
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| pipeline { | |
| // run on jenkins nodes tha has java 8 label | |
| agent { label 'java8' } | |
| // global env variables | |
| environment { | |
| EMAIL_RECIPIENTS = 'mahmoud.romeh@test.com' | |
| } | |
| stages { | |
| stage('Build with unit testing') { | |
| steps { | |
| // Run the maven build | |
| script { | |
| // Get the Maven tool. | |
| // ** NOTE: This 'M3' Maven tool must be configured | |
| // ** in the global configuration. | |
| echo 'Pulling…' + env.BRANCH_NAME | |
| def mvnHome = tool 'Maven 3.3.9' | |
| if (isUnix()) { | |
| def targetVersion = getDevVersion() | |
| print 'target build version…' | |
| print targetVersion | |
| sh "'${mvnHome}/bin/mvn' -Dintegration-tests.skip=true -Dbuild.number=${targetVersion} clean package" | |
| def pom = readMavenPom file: 'pom.xml' | |
| // get the current development version | |
| developmentArtifactVersion = "${pom.version}–${targetVersion}" | |
| print pom.version | |
| // execute the unit testing and collect the reports | |
| junit '**//*target/surefire-reports/TEST-*.xml' | |
| archive 'target*//*.jar' | |
| } else { | |
| bat(/"${mvnHome}\bin\mvn" -Dintegration-tests.skip=true clean package/) | |
| def pom = readMavenPom file: 'pom.xml' | |
| print pom.version | |
| junit '**//*target/surefire-reports/TEST-*.xml' | |
| archive 'target*//*.jar' | |
| } | |
| } | |
| } | |
| } | |
| stage('Integration tests') { | |
| // Run integration test | |
| steps { | |
| script { | |
| def mvnHome = tool 'Maven 3.3.9' | |
| if (isUnix()) { | |
| // just to trigger the integration test without unit testing | |
| sh "'${mvnHome}/bin/mvn' verify -Dunit-tests.skip=true" | |
| } else { | |
| bat(/"${mvnHome}\bin\mvn" verify -Dunit-tests.skip=true/) | |
| } | |
| } | |
| // cucumber reports collection | |
| cucumber buildStatus: null, fileIncludePattern: '**/cucumber.json', jsonReportDirectory: 'target', sortingMethod: 'ALPHABETICAL' | |
| } | |
| } | |
| stage('Sonar scan execution') { | |
| // Run the sonar scan | |
| steps { | |
| script { | |
| def mvnHome = tool 'Maven 3.3.9' | |
| withSonarQubeEnv { | |
| sh "'${mvnHome}/bin/mvn' verify sonar:sonar -Dintegration-tests.skip=true -Dmaven.test.failure.ignore=true" | |
| } | |
| } | |
| } | |
| } | |
| // waiting for sonar results based into the configured web hook in Sonar server which push the status back to jenkins | |
| stage('Sonar scan result check') { | |
| steps { | |
| timeout(time: 2, unit: 'MINUTES') { | |
| retry(3) { | |
| script { | |
| def qg = waitForQualityGate() | |
| if (qg.status != 'OK') { | |
| error "Pipeline aborted due to quality gate failure: ${qg.status}" | |
| } | |
| } | |
| } | |
| } | |
| } | |
| } | |
| stage('Development deploy approval and deployment') { | |
| steps { | |
| script { | |
| if (currentBuild.result == null || currentBuild.result == 'SUCCESS') { | |
| timeout(time: 3, unit: 'MINUTES') { | |
| // you can use the commented line if u have specific user group who CAN ONLY approve | |
| //input message:'Approve deployment?', submitter: 'it-ops' | |
| input message: 'Approve deployment?' | |
| } | |
| timeout(time: 2, unit: 'MINUTES') { | |
| // | |
| if (developmentArtifactVersion != null && !developmentArtifactVersion.isEmpty()) { | |
| // replace it with your application name or make it easily loaded from pom.xml | |
| def jarName = "application-${developmentArtifactVersion}.jar" | |
| echo "the application is deploying ${jarName}" | |
| // NOTE : CREATE your deployemnt JOB, where it can take parameters whoch is the jar name to fetch from jenkins workspace | |
| build job: 'ApplicationToDev', parameters: [[$class: 'StringParameterValue', name: 'jarName', value: jarName]] | |
| echo 'the application is deployed !' | |
| } else { | |
| error 'the application is not deployed as development version is null!' | |
| } | |
| } | |
| } | |
| } | |
| } | |
| } | |
| stage('DEV sanity check') { | |
| steps { | |
| // give some time till the deployment is done, so we wait 45 seconds | |
| sleep(45) | |
| script { | |
| if (currentBuild.result == null || currentBuild.result == 'SUCCESS') { | |
| timeout(time: 1, unit: 'MINUTES') { | |
| script { | |
| def mvnHome = tool 'Maven 3.3.9' | |
| //NOTE : if u change the sanity test class name , change it here as well | |
| sh "'${mvnHome}/bin/mvn' -Dtest=ApplicationSanityCheck_ITT surefire:test" | |
| } | |
| } | |
| } | |
| } | |
| } | |
| } | |
| stage('Release and publish artifact') { | |
| when { | |
| // check if branch is master | |
| branch 'master' | |
| } | |
| steps { | |
| // create the release version then create a tage with it , then push to nexus releases the released jar | |
| script { | |
| def mvnHome = tool 'Maven 3.3.9' // | |
| if (currentBuild.result == null || currentBuild.result == 'SUCCESS') { | |
| def v = getReleaseVersion() | |
| releasedVersion = v; | |
| if (v) { | |
| echo "Building version ${v} – so released version is ${releasedVersion}" | |
| } | |
| // jenkins user credentials ID which is transparent to the user and password change | |
| sshagent(['0000000-3b5a-454e-a8e6-c6b6114d36000']) { | |
| sh "git tag -f v${v}" | |
| sh "git push -f –tags" | |
| } | |
| sh "'${mvnHome}/bin/mvn' -Dmaven.test.skip=true versions:set -DgenerateBackupPoms=false -DnewVersion=${v}" | |
| sh "'${mvnHome}/bin/mvn' -Dmaven.test.skip=true clean deploy" | |
| } else { | |
| error "Release is not possible. as build is not successful" | |
| } | |
| } | |
| } | |
| } | |
| stage('Deploy to Acceptance') { | |
| when { | |
| // check if branch is master | |
| branch 'master' | |
| } | |
| steps { | |
| script { | |
| if (currentBuild.result == null || currentBuild.result == 'SUCCESS') { | |
| timeout(time: 3, unit: 'MINUTES') { | |
| //input message:'Approve deployment?', submitter: 'it-ops' | |
| input message: 'Approve deployment to UAT?' | |
| } | |
| timeout(time: 3, unit: 'MINUTES') { | |
| // deployment job which will take the relasesed version | |
| if (releasedVersion != null && !releasedVersion.isEmpty()) { | |
| // make the applciation name for the jar configurable | |
| def jarName = "application-${releasedVersion}.jar" | |
| echo "the application is deploying ${jarName}" | |
| // NOTE : DO NOT FORGET to create your UAT deployment jar , check Job AlertManagerToUAT in Jenkins for reference | |
| // the deployemnt should be based into Nexus repo | |
| build job: 'AApplicationToACC', parameters: [[$class: 'StringParameterValue', name: 'jarName', value: jarName], [$class: 'StringParameterValue', name: 'appVersion', value: releasedVersion]] | |
| echo 'the application is deployed !' | |
| } else { | |
| error 'the application is not deployed as released version is null!' | |
| } | |
| } | |
| } | |
| } | |
| } | |
| } | |
| stage('ACC E2E tests') { | |
| when { | |
| // check if branch is master | |
| branch 'master' | |
| } | |
| steps { | |
| // give some time till the deployment is done, so we wait 45 seconds | |
| sleep(45) | |
| script { | |
| if (currentBuild.result == null || currentBuild.result == 'SUCCESS') { | |
| timeout(time: 1, unit: 'MINUTES') { | |
| script { | |
| def mvnHome = tool 'Maven 3.3.9' | |
| // NOTE : if you change the test class name change it here as well | |
| sh "'${mvnHome}/bin/mvn' -Dtest=ApplicationE2E surefire:test" | |
| } | |
| } | |
| } | |
| } | |
| } | |
| } | |
| } | |
| post { | |
| // Always runs. And it runs before any of the other post conditions. | |
| always { | |
| // Let's wipe out the workspace before we finish! | |
| deleteDir() | |
| } | |
| success { | |
| sendEmail("Successful"); | |
| } | |
| unstable { | |
| sendEmail("Unstable"); | |
| } | |
| failure { | |
| sendEmail("Failed"); | |
| } | |
| } | |
| // The options directive is for configuration that applies to the whole job. | |
| options { | |
| // For example, we'd like to make sure we only keep 10 builds at a time, so | |
| // we don't fill up our storage! | |
| buildDiscarder(logRotator(numToKeepStr: '5')) | |
| // And we'd really like to be sure that this build doesn't hang forever, so | |
| // let's time it out after an hour. | |
| timeout(time: 25, unit: 'MINUTES') | |
| } | |
| } | |
| def developmentArtifactVersion = '' | |
| def releasedVersion = '' | |
| // get change log to be send over the mail | |
| @NonCPS | |
| def getChangeString() { | |
| MAX_MSG_LEN = 100 | |
| def changeString = "" | |
| echo "Gathering SCM changes" | |
| def changeLogSets = currentBuild.changeSets | |
| for (int i = 0; i < changeLogSets.size(); i++) { | |
| def entries = changeLogSets[i].items | |
| for (int j = 0; j < entries.length; j++) { | |
| def entry = entries[j] | |
| truncated_msg = entry.msg.take(MAX_MSG_LEN) | |
| changeString += " – ${truncated_msg} [${entry.author}]\n" | |
| } | |
| } | |
| if (!changeString) { | |
| changeString = " – No new changes" | |
| } | |
| return changeString | |
| } | |
| def sendEmail(status) { | |
| mail( | |
| to: "$EMAIL_RECIPIENTS", | |
| subject: "Build $BUILD_NUMBER – " + status + " (${currentBuild.fullDisplayName})", | |
| body: "Changes:\n " + getChangeString() + "\n\n Check console output at: $BUILD_URL/console" + "\n") | |
| } | |
| def getDevVersion() { | |
| def gitCommit = sh(returnStdout: true, script: 'git rev-parse HEAD').trim() | |
| def versionNumber; | |
| if (gitCommit == null) { | |
| versionNumber = env.BUILD_NUMBER; | |
| } else { | |
| versionNumber = gitCommit.take(8); | |
| } | |
| print 'build versions…' | |
| print versionNumber | |
| return versionNumber | |
| } | |
| def getReleaseVersion() { | |
| def pom = readMavenPom file: 'pom.xml' | |
| def gitCommit = sh(returnStdout: true, script: 'git rev-parse HEAD').trim() | |
| def versionNumber; | |
| if (gitCommit == null) { | |
| versionNumber = env.BUILD_NUMBER; | |
| } else { | |
| versionNumber = gitCommit.take(8); | |
| } | |
| return pom.version.replace("-SNAPSHOT", ".${versionNumber}") | |
| } | |
| // if you want parallel execution , check below : | |
| /* stage('Quality Gate(Integration Tests and Sonar Scan)') { | |
| // Run the maven build | |
| steps { | |
| parallel( | |
| IntegrationTest: { | |
| script { | |
| def mvnHome = tool 'Maven 3.3.9' | |
| if (isUnix()) { | |
| sh "'${mvnHome}/bin/mvn' verify -Dunit-tests.skip=true" | |
| } else { | |
| bat(/"${mvnHome}\bin\mvn" verify -Dunit-tests.skip=true/) | |
| } | |
| } | |
| }, | |
| SonarCheck: { | |
| script { | |
| def mvnHome = tool 'Maven 3.3.9' | |
| withSonarQubeEnv { | |
| // sh "'${mvnHome}/bin/mvn' verify sonar:sonar -Dsonar.host.url=http://bicsjava.bc/sonar/ -Dmaven.test.failure.ignore=true" | |
| sh "'${mvnHome}/bin/mvn' verify sonar:sonar -Dmaven.test.failure.ignore=true" | |
| } | |
| } | |
| }, | |
| failFast: true) | |
| } | |
| }*/ |
then you can use it by creating multi branch project in Jenkins and mark it to get use Jenkins pipeline as a code
References :
- Jenkins pipeline : https://jenkins.io/doc/book/pipeline/
- Jenkins file with a sample app for testing the pipeline execution on GitHub: https://github.com/Romeh/spring-boot-sample-app
Romeh's TechBlog 