Outdated Systems, Open Doors: The Dangers of Unpatched Software in CPG Environments

By Ahmik Hindman, Senior Network & Solution Consultant at Rockwell Automation

Key Takeaways:

• Unpatched software in CPG environments poses significant cybersecurity risks, with attacks on operational technology systems increasing.
• While patching is crucial, it’s complex due to legacy systems, potential disruptions, and the need for careful cost-benefit analysis.
• Effective patch management strategies include automated asset inventory, prioritization, change review boards, thorough testing, and consistent documentation and deployment processes.

In the digital world, the consumer packaged goods (CPG) industry faces increasing cyberattacks targeting operational technology (OT) systems with recent reports finding 47% of respondents experienced an increase of cyberattack exposure in the previous 12 months. These systems, often comprising legacy equipment, play a crucial role in managing and controlling various stages of the production process, from ingredient handling and mixing to packaging and distribution. However, unpatched vulnerabilities within these systems create significant entry points for malicious actors, exposing critical infrastructure to manipulation, disruption, and data breaches. 

In the first half of 2023, the rate of unfixed industrial control systems (ICS) flaws rose from 13% to about 34%. Every day, unpatched security software exposes assets to constant cyber threats, with devastating consequences for data, finances, and reputation should the attackers succeed. Ransomware attacks are on the rise: the time to take action is now, not after an attack has occurred.

While patching outdated software is a crucial step in securing operations, CPG companies face two significant hurdles: legacy systems lacking vendor support and the inherent complexity of integrating patches into intricate production environments. Beyond creating a cybersecurity headache, the burden of unpatched software and other technical debt has ballooned to an estimated cost of $1.52 trillion to fix. 

This article will offer a roadmap for CPG manufacturers to help secure their operations in a smart manufacturing environment. Manufacturers should deploy robust cybersecurity strategies like effective risk assessments, well-defined patching schedules, and layered security measures to address the threat outdated software has on their operations. 

To patch or not to patch? That is the question

Although patching vulnerabilities seems like a straightforward solution to improve OT network security, the reality in OT environments is far more nuanced. Patching every single flaw can be a complex and resource-intensive undertaking. Legacy systems, often unsupported by vendors, may lack readily available patches. Further complicating the issue is the complexity of updating intricate production environments, which can be time-consuming and disruptive, often requiring rigorous testing and potentially leading to downtime. Applying unnecessary patches may introduce unforeseen complications. Disruptions from unsuccessful patch applications can cause unwanted downtime and potentially jeopardize critical operations.

Furthermore, not all vulnerabilities require immediate patching. A cost-benefit analysis should be conducted to evaluate the potential impact of a specific vulnerability against the complexity and potential disruptions associated with patching. If the existing security controls, such as network segmentation and access controls, effectively mitigate the risk posed by the vulnerability, a delayed patch application, alongside close monitoring, might be a more practical approach. This measured approach helps ensure that OT security is maintained while minimizing the risk of operational disruptions.

Key elements for an effective patch management strategy

Developing well-defined policies and procedures is the cornerstone of an effective, repeatable patch management strategy for IACS. These policies establish a clear roadmap for managing vulnerabilities and maintain consistency in the patching process. Key elements manufacturers should incorporate into their cybersecurity strategies include:

Automated IACS asset inventory and vulnerability correlation

An automated IACS asset inventory forms the foundation of a robust patch management system, providing a complete and up-to-date picture of all IACS assets within the organization. Pairing an automated asset inventory with vulnerability databases and manufacturer patch lists provides a complete and up-to-date picture of all IACS assets within the organization.

Prioritization 

Not all vulnerabilities pose the same level of risk, which is why it’s crucial to determine your patch management strategy based on the potential impact vulnerabilities may have on your organization. To help determine vulnerability priorities, consider how critical the affected equipment is to overall operations, if there are any known exploits targeting the specific vulnerability and what potential disruptions could be caused by a successful attack.

Change review board and patch validation

A Change Review Board, comprised of members from maintenance, engineering, and operations, is crucial for assessing the comprehensive impact of proposed patch prioritization. This board is instrumental in verifying that patches for OT systems, applications, and firmware updates comply with the manufacturer’s approved standards. This ensures that only authorized updates are implemented, that overall risk is evaluated and considered, and that this aligns with business objectives and IACS asset criticality. 

Testing, deployment, and documentation

Thorough testing of patches in a controlled environment like a sandbox is essential before deploying them to production systems. This helps identify and mitigate potential conflicts with local applications and configurations. After testing, patches should be deployed based on the established criticality assessment. Documenting the entire deployment process through a change/configuration management solution provides a clear audit trail and facilitates maintaining the newly established baseline for IACS assets.

Change management and patch frequency

Documenting all patching activities via a change management solution achieves transparency and facilitates future audits. Establishing a baseline for IACS assets after successful patching allows for continuous monitoring of compliance and identification of any deviations. Maintaining a consistent patching frequency is crucial, striking a balance between addressing vulnerabilities and minimizing operational disruptions.

By implementing these comprehensive policies and procedures, CPG manufacturers can build a robust patch management strategy that effectively safeguards their critical IACS infrastructure from evolving cyber threats.

Defusing the ticking time bomb

Navigating the complexities of smart manufacturing while maintaining robust cybersecurity requires a proactive and multifaceted approach. By prioritizing effective risk assessments, implementing well-defined patching schedules, and adopting layered security measures, CPG manufacturers can proactively mitigate threats posed by outdated software and build a foundation for secure and resilient operations in the digital age. Embracing this proactive approach is not just an option, but a necessity to achieve continued success and consumer trust in the ever-evolving CPG industry.

Ahmik Hindman is a Senior Network & Solution Consultant at Rockwell Automation with 28 years of experience in ICS cybersecurity. A Senior Member of the International Society of Automation, Ahmik has a BS in Electrical Engineering and an MBA in IT.