The XZ exploit is a clever supply chain attack, placing a RCE in sshd using a vulnerability in XZ, a compression library that many Linux distributions call when running OpenSSH (sshd -> libsystemd (for systemd notifications) -> liblzma (for logging)). It was discovered by Andres Freund when he noticed that sshd was running with abnormally high CPU usage.
The vulnerability was submitted by Jia Tan, a maintainer of XZ project who had been an active contributor since 2021. The vulnerability was extremely obfuscated. Read the original backdoor report by Andres Freund for more details on this.
Maintainers
There are two maintainers of the XZ projects, Jia Tan and Lasse Collin. Lasse Collin is the original author of the XZ Utils library, whereas Jia Tan has only been a maintainer since 2022.
Tan appears so have little to no online footprint prior to 2021 when they started making useful contributions to XZ. It is speculated that Jia Tan is an invented persona.
He has been part of the xz project for 2 years, adding all sorts of binary test files, and to be honest with this level of sophistication I would be suspicious of even older versions of xz until proven otherwise.
- A Microcosm of the interactions in Open Source projects, and interesting analysis of mailing list discussions about Jia Tan's contributions to the XZ project before Tan was made maintainer
- Lasse Collin's page tracking the vulnerability
Features
- kill switch based on environment variable
yolAbejyiejuvnup=Evjtgvsh5okmkAvjis hardcoded (and then obfuscated)
- backdoor itself is secured as it checks for a private key
- this is an interesting argument against "when you make a backdoor for good guys you also make a backdoor for bad guys" though that's another conversation entirely
Oddities
- In 2022 there were other suspected sock puppet accounts pushing Collin to introduce new maintainers
- Tan disabled a function in oss-fuzzer in 2023 that the exploit relied on that may have revealed the exploit (source)
- the day before the exploit was found the xz-java project added a SECURITY.md file
If you discover a security vulnerability in this project please report it privately. Do not disclose it as a public issue. This gives us time to work with you to fix the issue before public exposure, reducing the chance that the exploit will be used before a patch is released.
- It disabled Linux landlock?
- This commit introduces a syntax error that prevents Linux landlock from being activated. The test program designed to test if landlock was supported would always fail due to this errant
..
- This commit introduces a syntax error that prevents Linux landlock from being activated. The test program designed to test if landlock was supported would always fail due to this errant
- Tan's first GitHub PR is to libarchive and is suspect as it subtly replaces
safe_printfcalls with unsafefprintf
Discovery
I was doing some micro-benchmarking at the time, needed to quiesce the system to reduce noise. Saw sshd processes were using a surprising amount of CPU, despite immediately failing because of wrong usernames etc. Profiled sshd, showing lots of cpu time in liblzma, with perf unable to attribute it to a symbol. Got suspicious. Recalled that I had seen an odd valgrind complaint in automated testing of postgres, a few weeks earlier, after package updates.
Really required a lot of coincidences.
Andres Freund (a Postgres developer at Microsoft)
As noted by nurple on HN, this exploit was found for similar reasons to Ken Thompson's supply chain exploit was detected. Strange performance sparking curiosity.
Links
- FAQ on the exploit by Sam James
- Initial discussion on hn
- CVE-2024-3094 Report
- Original backdoor report by Freund
- Discovery of RCE
- Technologist vs Spy, making the case that based on the behaviour and circumstances surrounding the bug were that of organised actors
- XZ Backdoor Documentation
- XZ Bash-Stage Obfuscation Explained
- Ars Technica Overview
- PoC with substituted public key
- Even more links
- Timeline of XZ
- Jia Tanning Go Code
