Originally published at https://cyberbakery.net on January 23, 2022.
Agile software development methodologies have become popular in the last decade or so. One of such methodologies, DevOps, has established itself among the most used methodologies across various industries throughout the globe. However, for security teams, this popularity of DevOps has increased attack surface, which is continuously changing that requires protection. This post will focus on how we got here and why DevOps developers are becoming the bad actor’s favourite target.
The last two decades have seen a tremendous change in how companies have adopted new ways of doing business. Digital transformation helped businesses innovate, bringing new products and offerings for their customers. However, since every business is doing the same, time to market has gone down, and businesses need to keep innovating at speed and scale. DevOps helped application delivery to meet the business requirement.
DevOps teams are usually part of the business and always looking for quick innovations that leave them vulnerable. Business loves this flexibility and not being part of the traditional technology or IT teams. Since the DevOps teams are often part of the business, they give into the business demands, especially during the times like today when COVID19 has presented unseen business challenges. This meant the rigour and the traditional development process were either not properly followed or were deliberately ignored in the name of need and speed.
Companies need for speed and innovation often leads teams to use tools and techniques to create new DevOps pipelines, and in doing so, many new threat vectors are introduced, giving hackers their target. Hackers are very much aware of developer psychology, where they use third-party tools and libraries to shorten the development lifecycle. Today, everything is a code, and DevOps teams are pursued by the threat actors targeting cloud credentials stored in repositories like GitHub.
In the same way, the source code, usually a business IP(intellectual property), is targeted to gain direct access to the IT infrastructure and data. Hackers often leverage outdated third-party libraries in the build pipelines to inject malicious code before the wider deployment of the code as a primary or a secondary attack. Supply chain attacks are a classic example of such breaches. The recent SolarWind and Kaseya attacks were executed by inserting malicious code in their products.
Every stage of the DevOps process is vulnerable to the threats of malicious activity leading to data loss for the business, which may also impact the downstream customers of the company.
As mentioned above, due to how DevOps teams are structured, segregation of duties is often ignored, making them the most privileged users in the business having access to production and non-production environments. Therefore, if developer credentials are compromised by a well-crafted phishing attack or by other means such as machine compromise, then it is not only the sources code is at risk but also the risk of losing the customer data.
We have often seen that business demands excessive access for their DevOps developers. They are often over-provisioned with systems access in the name of speed and the anticipation that the developer might need to do their jobs in the future. For sure, this improves developer efficiency, but all the security principles regarding access controls learnt and developed over the years go down the drain. As noted above in the post, if a hacker sets his hand on such an over-provisioned developer credentials, it can create havoc by inserting malicious code into the application, leaking source code, changing security control, and the most pervasive of all is by stealing production customer data.
Due to this very reason, developer credentials have become an attractive target for hackers. Therefore, organisations must take adequate steps to protect developer credentials as “crown jewels”.
It isn’t easy to get this right unless the business and the security organisation create an integrated DevOps where security is inherently integrated with the development process.
The following are some of the recommendations to start this journey.
* Least-privilege policy: Implementing a least privileges policy is the only real deal without which it is only a matter of time that you will be breached. Traditionally, user access is only reviewed for the end-users and not the developer’s access. Businesses must implement user access reviews for developer access so that access privileges are always aligned to their roles in performing their jobs. This will help in limiting exposure to credential breaches.
* Make it hard: Implement strong authentication practices for privileged users. Implement multi-factor authentication, single sign-on and IP safe listing practices to help reduce the risk of developer credentials falling into the hands of hackers.
* There is no place for credentials in the code: Whether production or non-production environments, storing secrets in the code is a “bad idea”. Code must be checked for stored secrets, especially in production environments. Automated tools such as AWS Secrets Manager or CredStash be used for managing developer credential management.
* Automate developer access management: The developer accounts are often kept outside the automated user access provisioning and de-provisioning using single sign-on or linked with HR systems. This leads to poor management of developer access, failing to revoke their access on role change or leaving the company.
* Shift left (Automate, automate, automate…): Embed security processes in DevOps pipelines to ensure security controls are embedded right in the development process. Seek opportunities to automate these processes to reduce human intervention as much as possible. For example, no code is merged into the master branch without going through the review process, checking for a series of security tests that include secret detection and peer review. Enforce automated policies to stop code from being pushed without these tests. Repositories are rigorously monitored for changes, so malicious actors cannot circumvent the security tests.
* Audit, audit and audit: Ensure systems are not provisioned with default credentials. Periodically audit production systems for default or embedded credentials so that the credentials are not left in the code, helping hackers to have a field day.
* Train business and DevOps developers: To err is human, and developers are humans bound to make mistakes. There is no substitute for training, and organisations must spend energy to equally train their Developers and business. Security teams must develop security champions and DevOps teams to promote and implement good security practices.
It is important to focus on better protection of the DevOps environment. Threat actors, including nation-states, will continue to find new ways to breach your defences. If businesses fail to protect DevOps teams, they will continue targeting developers as a way of gaining access to “crown jewels”.
Please enable JavaScript to view the
Originally published at https://cyberbakery.net on January 23, 2022.