How Netflix & Instagram are using Jenkins: 𝗖𝗮𝘀𝗲 𝗦𝘁𝘂𝗱𝘆
Hello guys, In this blog I’m going to explain the automation using Jenkins and a case study on Netflix & Instagram.
What is Jenkins ?
Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software.
Jenkins can be installed through native system packages, Docker, or even run standalone by any machine with a Java Runtime Environment (JRE) installed.
But, why do we need to automate the tasks? What are the benefits?
Benefits :-
- It is open source and it is user-friendly, easy to install and does not require additional installations or components.
- It is free of cost.
- Easily Configurable. Jenkins can be easily modified and extended. It deploys code instantly, generates test reports. Jenkins can be configured according to the requirements for continuous integrations and continuous delivery.
- Platform Independent. Jenkins is available for all platforms and different operating systems, whether OS X, Windows or Linux.
- Rich Plugin ecosystem. The extensive pool of plugins makes Jenkins flexible and allows building, deploying and automating across various platforms.
- Easy support. Because it is open source and widely used, there is no shortage of support from large online communities of agile teams.
- Developers write the tests to detect the errors of their code as soon as possible. So the developers don’t waste time on large-scale error-ridden integrations.
- Issues are detected and resolved almost right away which keeps the software in a state where it can be released at any time safely.
- Most of the integration work is automated. Hence fewer integration issues. This saves both time and money over the lifespan of a project.
Continuous Integration/Continuous Delivery commonly known as CI/CD. A developer writes the code and pushes the code to any of the DSCM (Distributed Source Control Management) tool like Git and Github. Then, further tasks like build, test, and deploy can be carried out using Jenkins. You can use triggers to track the changes in the Github repo and as soon as the code is pushed. It will build and test the code on the automated server architecture in any of the setups like Master-slave architecture. This is known as continuous integration and the code is deployed and a new feature is delivered without any human intervention. This is known as continuous delivery.
1. Netfilx :-
So Lets Start With How Netflix Uses Jenkins??
- Netflix, Inc. is an American over-the-top content platform and production company headquartered in Los Gatos, California. Netflix was founded in 1997 by Reed Hastings and Marc Randolph in Scotts Valley, California,
- The company’s primary business is a subscription-based streaming service offering online streaming from a library of films and television series, including those produced in-house.
Netflix & Jenkins
- Netflix uses Jenkins for continuous integration and deployment. As soon as a line of code has been built and tested locally using Nebula(A collection of Gradle plugins built by Netflix). The first step is to push the updated source code to a git repository.
- Once the change is committed, a Jenkins Job is triggered. Netflix started with a single massive Jenkins master in their datacenter, and now they run 25 Jenkins masters in AWS.
- Jenkins Job is configured to invoke Nebula to build, test and package the application code. If the repository built is a library, Nebula publishes the .jar to the artifact repository and if the repository built is an application, then the Nebula os package(operation system package) plugin would be executed.
- Using the Nebula os package plugin, an application’s build artifact would be bundled into either a Debian or RPM package whose contents are defined via a simple Gradle-based DSL(Domain Specific Language).
2. Instagram :-
At Instagram, they deploy their backend code 30–50 times a day…
Before continuous deployment, engineers deployed changes on an ad-hoc basis.
They’d land changes, and if they wanted them deployed soon, they’d run a rollout. Otherwise they’d wait for another engineer to come along and do so. Engineers were expected to know how to do a small scale test beforehand: they would do a rollout targeting one machine, log into that machine and check the logs, and then run a second rollout targeting the entire fleet. This was all implemented as a Fabric script, and we had a very basic database and UI called “Sauron” which stored a log of rollouts.
They already had a test suite, but it was only run by engineers on their development machines. Code reviewers had to take the author’s word that the tests passed, and they didn’t know the test status of the resulting commit in master. So they setup Jenkins to run tests on new commits in master and report the result to Sauron.
Some tasks include doing a rollout when things were normal simply consisted of answering “yes” a couple times (accepting the suggested commit, starting the canary, and continuing to the full deploy). So they allowed these questions to be answered automatically, and got Jenkins to run the rollout script. At first engineers implementing this only enabled Jenkins when they were at their desks supervising, until they didn’t need to supervise it anymore.
Thank you for reading my article🤗
