Level19 has developed a methodology that helps our clients accelerate the remediation of security vulnerabilities while ensuring that the most critical are dealt with first, reducing our clients’ risk exposure as soon as possible.
Jason Gray, CEO of Level19
Introduction
While we all stayed home, the COVID pandemic proved pivotal in accelerating a global shift to a more digital and more online world. This rapid evolution also made technology more vulnerable to attacks. A recent McKinsey survey [1] estimates that due to the pandemic, companies fast-tracked their digitization to online channels by a significant three to four years. Over the same timeframe, the National Institute of Standards and Technology (NIST) observed an increase in the number of security vulnerabilities that are identified year over year, with over 20,000 new ones identified in 2021 [2]. As these are identified, it’s important to make sure you have the tools to spot and address the threats before falling victim to a security breach.
Three key questions to ask about your company’s security are:
- How vulnerable is your existing code to security breaches?
- How are you keeping new code from introducing new vulnerabilities?
- How can you actively manage, understand and reduce your risk profile?
Incorporating a Static Application Security Testing (SAST) tool into your regular development process is an excellent way to catch and fix security exposures. Vulnerabilities can be introduced through existing legacy code, new development or via third-party software. A SAST tool identifies potential security threats by analyzing source code in applications. They are automated, scalable solutions often incorporated into a company’s CI/CD process to identify issues early and allow them to be addressed prior to production deployment. SAST tools focus on identifying Open Web Application Security Project (OWASP) Top 10 issues [3], allowing developers to fix these issues early in the process and protect your systems and data.
Implementing SAST
Given the stakes when it comes to protecting your data security, it’s important not to take shortcuts when implementing SAST. The approach Level19 recommends is as follows:
- Assess maturity of your organization
- Define problem size
- Plan how to address risk
- Prioritize applications
- Triage issues by type
- Promote remediated code to production
- Apply a gating process
- Define an exception process
- Train your team
- Developers are responsible for secure code
- Security sets the standards
- Others contribute
- Monitor your security risk profile
- Stay current and onboard new applications
- Document your processes
Table 1 – Level19’s methodology to incorporate SAST in your company.
1. Assess your current state
An assessment of your organization’s current state is key to setting up your SAST tool. This helps identify gaps, ascertain security risks and set an initial jumping off point to improve your risk profile quickly and effectively. The process Level19 recommends involves determining the current situation, defining the size of the problem and developing a plan to address existing vulnerabilities and avoid introducing new ones.
A) ASSESS MATURITY OF YOUR ORGANIZATION
Implementing SAST tools correctly is not just a matter of turning on a new tool, they need to be integrated into your organization across people, processes and technology. Prior to rolling out a code scanning tool, it helps to understand the current maturity of your organization. Assessment of roles and responsibilities, processes and technology help identify gaps and needs requiring attention. Bringing in a developer with SAST expertise can help you implement leading practices when deploying the tool and increase the odds of a successful implementation.
B) DEFINE PROBLEM SIZE
An overall inventory of the applications that need to be scanned should be developed to ensure all relevant code is identified and scanned as necessary. Libraries and rules are set to inform the tool how to identify security defects and bugs. The initial scan identifies the magnitude of the problem in the legacy code base, including third-party libraries, providing insight for management and executives into the security posture of the organization.
C) PLAN HOW TO ADDRESS RISK
Once the initial scan is complete, a baseline report provides insights that make it possible to manage risk on two fronts:
- Put a plan in place to remediate vulnerabilities in the existing code, and
- The SAST tool and processes should be integrated into the existing software development life cycle (SDLC) process, ensuring that new code does not introduce new security issues to your applications.
2. REMEDIATE LEGACY SECURITY VULNERABILITIES
To address existing vulnerabilities, Level19 finds the following approach is the best way to get up and running and begin addressing critical issues requiring immediate fixes as soon as possible.
A) PRIORITIZE APPLICATIONS
Once the initial assessment scans are completed, prioritize remediation activities based on the risk profile of the applications. Internet-facing applications, applications with elevated privileges and mission critical software are all at high risk of hacking. As each application scan is completed, issues requiring immediate remediation should be identified so they can be addressed in a timely manner.
B) TRIAGE ISSUES BY TYPE
The SAST tool typically rates vulnerabilities as critical, high, medium and low. It remains important, however, to do a manual review to appropriately classify the issues, determining the ones that require immediate fixes and informing how to address the remaining vulnerabilities. Level19 uses the following classifications:
- Immediate fixes – the highest risk vulnerabilities that must be fixed ASAP,
- Third-party libraries – requiring software upgrades and often involving dependencies between components,
- Remaining SAST rated critical/high – code changes for important issues where remediation can be planned as part of an overall strategy,
- False positives/won’t fix – vulnerabilities that upon review are not actually issues and can be closed.
On first introducing code scanning, companies typically discover thousands of security weaknesses. It’s important to complete the reviews as efficiently as possible so the team can work on any immediate hotfix issues. Level19 has significant experience and proven methodology in remediating vulnerabilities, expediting the review process and, by triaging items requiring urgent intervention and distinguishing them from all others, our team can begin fixing the most critical issues immediately.
As those issues are addressed, further passes should break out third-party library issues, SAST rated critical/high vulnerabilities and – also important – false positives and issues that will not be fixed due to sunsetting an application or other considered reasons. By classifying one type with each pass, the team can focus on narrow criteria to assign the defects quickly and effectively. The classification passes of the issues do not always represent the priority order in which they are addressed as, due to dependencies or testing effort, third-party libraries often are dealt with after the SAST issues rated as critical/high.
C) PROMOTE REMEDIATED CODE TO PRODUCTION
Addressing legacy vulnerabilities is labour intensive, complex and time consuming – all reasons addressing them typically gets put on the back burner. We find that a dedicated and experienced team works best to identify and prioritize issues quickly, allowing existing development teams to continue focusing on delivering business value. As the team begins fixing defects in the code – especially in the early stages, when critical problems are addressed – the remediation team needs to work closely with the organization’s release management team to schedule the fixes. In addition to the release team, we also recommend allocating QA resources and deploy required tools to assist with regression testing before promoting any changes to production.
It’s important to take a risk-managed approach when deploying fixes. Level19 recommends the following:
- Immediate fixes should be isolated within their own release, allowing any potential issues to be identified quickly. Any problems that can’t be resolved speedily can be addressed by backing out the change, minimizing the application downtime.
- Identifying the potential impact is important for all changes, but especially hotfixes, as there often are time constraints for resolving the issue. Understanding what needs to be regression tested can help expedite the change with confidence.
- Third-party library fixes often require investigation to understand the upgrade sequence to prevent library version incompatibility within and across components.
- SAST-rated critical/high fixes require knowing the dependencies of the changes, and if there are potential cascading effects.
By understanding the potential impact of changes, an experienced remediation team can work with the QA team to identify risks and testing scope.
3. IMPLEMENT SAST INTO YOUR CI/CD PROCESS
When incorporating a SAST tool into your organization, new processes must be established to ensure it becomes part of your organization’s regular development process. For the code scanning tool to be effective, quality gates need to be defined and implemented. An exception process is also required, for those rare circumstances when a functional change must be promoted to production to meet a launch date, even though it has been flagged as having a vulnerability.
A) APPLY A GATING PROCESS
A gating process ensures that new code doesn’t introduce fresh security vulnerabilities. It remains important that any gates introduced do not impede developers from being able to do their work or meet deadlines. During the development and testing cycle in lower-level environments, we don’t recommend blocking code from deployment. Developers, however, can gain early insight into security issues by using an Integrated Development Environment (IDE) extension connected to your SAST tool as they develop their changes. New defects should not be allowed into higher level environments unless approved by senior management.
B) DEFINE AN EXCEPTION PROCESS
An exception process allows identified issues to pass the quality gate in extenuating circumstances. In these cases, the risks should be understood, and a remediation plan developed to address them by a date agreed to by all stakeholders. Senior management will need to accept the identified risks and the proposed compensating controls. After an exception is made, it must be tracked – ideally in an enterprise risk management tool – to ensure it is addressed in the future. Not having an established exception process can potentially stall a critical release.
C) TRAIN YOUR TEAM
Whatever SAST tool is chosen, it is important to set aside time to train your team on how to use it. Like a gym membership, even the best tool, when used improperly – or not at all – will not provide the benefits you hope to achieve. As part of a commitment to improving the security posture, guidelines should be introduced around secure coding standards for the development team. Building reference documentation is also critical for the success of the implementation. Each organization will have their own spin on their standards and how the tool should be used. Training and documentation should reflect this.
4. UPDATE ROLES AND RESPONSIBILITIES
When focused on building a more secure SDLC process, everyone on the team plays their part in improving the quality and security of the code going to production. Leadership must be responsible for oversight at every stage. Typical stakeholder teams found within the process are as follows:
A) DEVELOPERS ARE RESPONSIBLE FOR SECURE CODE
Developers are responsible for the quality and security of their code and should follow all secure coding guidelines introduced by the organization. Their role will expand to include new responsibilities, and they will need to integrate a SAST tool into their development. When merging code, they should be aware of the tools and use them to address identified issues as soon as possible – before these become issues in higher environments. Dev leads also need to incorporate the scans and results as part of their reviews, and work with developers and the security team when issues are identified. We have found that defining a “Security Champion” within the development team, whose role is to interact with security, helps drive more secure coding practices.
B) SECURITY SETS THE STANDARDS
Security touchpoints should become a regular part of the development lifecycle. Security representatives negotiate with development teams through a security lens to continuously set standards and priorities for security risks. Initially, the focus may be on critical and high issues, but as the program matures standards can become more exacting. The security team works closely with development to agree on any false positives identified by the tool, so potential risks are discussed and reasons why they have been identified as a false positive are well understood.
C) OTHERS CONTRIBUTE
Other groups have their roles to play as security vulnerabilities are addressed through remediation or new gating processes. These include:
- QA – critical for regression and performance testing when releasing remediated fixes,
- Operations – works with the remediation team to get fixes released and helps monitor the process; release management could also be integral to any gating or exception process,
- DevOps – the team that manages and supports the tool for the development team,
- Business – needs to understand the value of addressing security vulnerabilities and reducing risk, and why funding should be dedicated to reducing issues in the code, and
- Leadership – across the organization, leadership prioritizes the importance of security and emphasizes the importance of new protocols. Leadership also defines the risks your organization can accept, as their approval is required for any exceptions made to production. The leadership of both the security and development domains holds their groups accountable and ensures that any exceptions are remediated according to plan.
5. MAINTAIN AND CONTROL
Security needs to become part of your organization’s DNA. The goal should be continuously improving and refining how things are done.
A) MONITOR YOUR SECURITY RISK PROFILE
Dashboards give visibility to your risk profile while also keeping a security mindset front and centre. Building a proper security dashboard for review by both executives and teams is key to managing risks going forward and to avoid creating technical debt. Implementing criteria to indicate application risk scores based on the number and severity of security vulnerabilities, as well as tracking the status of issues requiring remediation, are just some of the metrics that should be added to security dashboards as part of an ongoing effort to improve your risk profile.
B) STAY CURRENT AND ONBOARD NEW APPLICATIONS
New threats are continuously being identified and it’s important to stay up to date. As part of any secure SDLC, it is vital to regularly review the criteria and rules used for your code scanning to make sure they remain up to date. The rules, libraries and languages used for your tool should be reviewed quarterly by the security and development teams to revise any obsolete rules and incorporate the latest knowledge into scans and developer training materials. Establish a process to ensure all new applications are onboarded into your tool, consistently and across the enterprise.
C) DOCUMENT YOUR PROCESSES
As part of any good security posture, it’s important that all processes surrounding your secure SDLC, including code scanning, be properly documented and up to date. For those in regulated industries with heavy compliance burdens, documenting the processes and tracking the reviews and approvals are critical for any certifications you may be seeking. Decisions regarding exceptions that make it to production and require a remediation plan should properly document the rationale, decisions and approvals for transparency and auditability.
Final Thoughts
There are obvious benefits to integrating a SAST tool into your existing development process:
- It helps identify existing security issues that can be addressed to reduce your organization’s risk profile,
- Improves your company’s security profile by significantly reducing the number of vulnerabilities introduced as part of new development, and
- There is increased visibility and knowledge that, if new issues are introduced, a plan will be created to address them within a defined timeframe.
Building a secure SDLC process is an ongoing journey for any organization, with lessons learned continuously. It is important that everyone in the organization, from leadership through individual developers, remains diligent and treats security as a top priority.
Level19 has assisted clients in establishing security remediation programs, remediated tens of thousands of security vulnerabilities and helped set up the tools and processes to reduce the introduction of new issues. We believe that with the right approach and proper methodology it is possible to greatly reduce the risk of security vulnerabilities, benefitting both you and your customers.
If you are considering introducing a new SAST tool or are looking to reduce security vulnerabilities in your existing code, please reach out to us at security@level19.com. We’d be happy to discuss your needs and see how we can help.
References
[1] LaBerge, C. O’Toole, J. Schneider, and K. Smaje, “How COVID-19 has pushed companies over the technology tipping point–and transformed business forever,” McKinsey & Company, 18-Feb-2021. [Online]. Available at: https://www.mckinsey.com/capabilities/strategy-and-corporate-finance/our-insights/how-covid-19-has-pushed-companies-over-the-technology-tipping-point-and-transformed-business-forever. [Accessed 01-Oct-2022].
[2] “CVSS Severity Distribution Over Time,” National Vulnerability Database, 2022. [Online]. Available at: https://nvd.nist.gov/general/visualizations/vulnerability-visualizations/cvss-severity-distribution-over-time. [Accessed 01-Oct-2022].
[3] Owasp.org. 2022. OWASP Top Ten | OWASP Foundation. [Online] Available at: https://owasp.org/www-project-top-ten/ [Accessed 01-Oct-2022].