Those of you who know me, know that I put the facts first. Normally. Today I venture into the realm of conjecture. I hope that I have enough clout, and enough circumstantial evidence, to hold your attention while I present my case.
I wrote a post last week about why we need to talk more about Foreshadow. The same week, I wrote a post discussing ways to avoid Microsoft Patch Hell. I kept thinking something wasn’t right – why was I getting so many BSODs in my test-environment? Why were there so many security patches on Patch Tuesday? Why wasn’t Microsoft doing the usual level of QA I would expect?
Get your tinfoil hat on, because it is time to synthesize these real-world happenings into a zany-yet-seemingly-plausible conspiracy theory!
I must emphasize that this is only a theory – albeit one backed-up by a little circumstantial evidence, and some not-so-deductive logic. The theory is that the lack of discussion about a recent Intel processor vulnerability (Foreshadow) and the recent BSOD-producing Microsoft Patch Hell people have been going through, are related. I present three possible explanations of why this could be. First, what are these theories based on?
We do a lot of testing and analysis to ensure that our service can operate in environments of extreme variation. So, it was surprising for us to see so many failures (BSODs) when testing recent Microsoft patches. To me, it really looked like a driver issue. By looking at post-crash dumps, and through our testing methodology, the netIO.sys errors we were seeing seemed to be diagnosed.
Ok, kudos to us at IntelliGO, we found a potential cause, so just don’t roll-out updates until Microsoft has corrected this… right? Except that these issues persisted for weeks, across multiple hardware configurations. I wondered, why is Microsoft deliberately abandoning patch QA or not at least halting roll out?
The only reason to sacrifice safety (system stability and proper testing) in change management is security. Naturally this brings me back to the frequent and recent Intel vulnerabilities – Microsoft and their own reputation for patches withstanding, has nothing to gain by letting QA fall by the wayside. Similarly, Intel had already disclosed models affected by Foreshadow vulnerability, so why now?
Here are three possible reasons why Intel and/or Microsoft would make these concessions on the patching process; ordered from most-to-least feasible.
1) A hasty fix for the Foreshadow Exploit
The idea here is that with the new discovery of an exploit for the Foreshadow vulnerability, there was an incentive for Intel to ‘rush through’ the fix, sacrificing the normal precautions/collaboration with OS developers (Microsoft). In my opinion, this is both the most likely and the least ‘tinfoil-hat-requiring’ explanation. They won’t suffer commercially if they address the problem before they have to start talking about it.
2) An as of yet undisclosed processor vulnerability
Intel’s knowledge of another vulnerability could explain hasty-patching. We could be seeing an effort to get in front of it, by patching before the vulnerability is revealed. Adding to the list of publicly disclosed vulnerabilities would be difficult to do without some serious fallout/impact for Intel. Intel has already been afflicted by several high-profile vulnerabilities this year. There have long been suggestions of complacency from having been ‘on top’ so long. You can imagine how another would impact them.
3) A secret patching collaboration between MS and Intel
The most out-there possibility is that Intel is leveraging a back-door relationship with Microsoft to ‘sneak’ the patches in. The ‘conspiracy’ being not only that Intel knows (as described above) but that Microsoft knows too and is helping to mitigate the risk of the vulnerability being exploited. Though that may seem a noble outcome, I argue below that the end wouldn’t justify the means (if this were really happening).
Conclusion: We Need to Demand Better Visibility
Conspiracy theories aside, the most interesting thing about any of this is that we have collectively accepted this practice of sacrificing system stability for security: Intel and Microsoft both appear to have thrown QA out the window in order to rush these patches through.
This shouldn’t be a surprise: drawing from past experience there are many examples of foundational flaws creating some surprising and dangerous vulnerabilities (remember the KRACKER Wi-Fi exploit?). The typical response to such a vulnerability has been to release emergency patches (not always easy) – and that’s okay, that’s the established solution. I talk about patching in the context of hygiene all the time with clients.
What is not okay is how the potential impact is being explained to us. So far, the impact of the patches that has been discussed is the performance impact. The market has tolerated these performance impacts of up to 30% percent! Yet, that is not really the biggest problem in terms of business impact. The biggest problem is the BSODs. As I mentioned in my previous post, the irony of systems going down in an attempt to prevent systems going down is not lost on me.
MDR services can protect you from attacks that rely on exploits of vulnerabilities like Foreshadow; but what can protect you from BSODs brought about by careless patching?