The Meaning of 'Success' in Deployment Frequency

‍

The term "success" in the context of Deployment Frequency often refers to the completion of the deployment process, regardless of potential software issues. The aim of the metric is to push teams to have reliable, repeatable, fast deployment processes. When the deployment completed with success we count it as deployed and don’t check if has bugs or issues on running code. Change failure rate will measures the bugs/failures on the running code so if the deployment is a success than it is counted as deployed.

‍

‍

Considering Issues and Regressions

‍

As we mentioned above, the post-deployment effects should be measured on another important metric called “Change failure rate”. It is difficult to measure that properly and have many debates on how to measure it. Our aim with the deployment frequency is to help teams to deploy everyday however the change failure rate can be in a range between 0-15% of the changes made.

‍

‍

Distinguishing Deployment Frequency from Code Change Issues

‍

We saw some teams tried to measure only deployments that has zero issues or failures after deployment. However, this will make deployments to be happen less. Deployment and changing code or writing code are two different aspects of shipping code to production. The benefit of measuring those two is to help teams to focus on separate processes individually. Instead of merging two different processes, “deployment” and “code change failures”,  we measure them separately and focus each of them individually.

‍

‍

The Natural Throttle: Quality Issues

‍

Quality issues can serve as a natural throttle to cycle time and deployment frequency. Dora Metric system is by nature have correlation between 4 keys. All metrics, “deployment frequency”, “change failure rate”, “mean time to recover”, “lead time for changes”, have correlation between each other and even successfully measuring one of them help teams to predict the others. This is the beauty of dore metric system.

‍

‍

The Role of Deployment Failures

‍

As we discussed above, deployment failure can effect all 4 metrics on dora metric system. The failure of the deployment will be reflected to higher change failure rate, if the deployment failure cause an outage on system than that will increase the time to restore service metric, lead time for changes will also increase with the failure on deployment very similar reason as time to restore a service since the lead time is calculated by measuring the time between pull request creation to deployment to production.

This is again the beauty of dora metrics, one metric can have significant impact on others and can tell more than numbers. Teams should monitor their CI/CD pipeline carefully. The main goal should be having less changes per deployment with higher deployment rates so that the problems arise and solved granularly.

‍

‍

The Big Picture

‍

You might have calculating the dora metrics or not but dora metric system can identify multiple problems and help teams to have better developer experience. Each metric has impacts on the others and opens new ways to deploy better.