ETW: The EDR Substrate

The answer turns on one architectural asymmetry. Event Tracing for Windows is a high-rate, kernel-buffered observability bus modern Windows EDRs commonly consume, and a 2007-era decision (letting eight sessions read the same provider concurrently) is what lets multiple vendors coexist on a single host. Microsoft's Microsoft-Windows-Threat-Intelligence provider, gated behind Protected Process Light and an ELAM-signed Antimalware certificate in the Windows 10 RS-era (the exact public onboarding date is not pinned by Microsoft), fires from the kernel side of memory-modifying syscalls and survives the user-mode EtwEventWrite patch class that defined red-team tradecraft from 2020 to 2022. The remaining attack surface (BYOVD-driven kernel tampering) is structurally narrowed by the Vulnerable Driver Blocklist enabled by default since Windows 11 22H2, leaving the sub-microsecond-payload gap as ETW's irreducible "observation, not enforcement" limit.

Why the patch did not silence Defender

As a representative 2026 endpoint-detection scene, imagine a red-team operator on a Defender-protected box running the move that worked five years ago. They locate ntdll!EtwEventWrite in the calling process, write the byte 0xC3 over the function prologue, and the calling process now silently fails to emit user-mode ETW events. The .NET CLR provider goes dark. Invoke-Mimikatz loads from execute-assembly without lighting up Microsoft-Windows-DotNETRuntime. A Defender alert can still arrive shortly afterward, because the detection pipeline is not limited to that one user-mode provider.

The patch worked. The .NET tracing provider in that process is mute. Attach a debugger and disassemble the function prologue: the first byte is now 0xC3, the near-return opcode [1072], and any caller falls straight back to its return address before producing a single event. The technique is the one Adam Chester documented in March 2020 [1073], and to a generation of red teamers it has functioned as a near-universal ETW evasion ever since.

So why did Defender still fire?

Because Defender does not have to rely on Microsoft-Windows-DotNETRuntime alone to detect credential-theft tradecraft. One load-bearing signal in that class is Microsoft-Windows-Threat-Intelligence [1074]: a provider whose GUID is {f4e1897c-bb5d-5668-f1d8-040f4d8dd344}, whose events fire from inside the kernel side of memory-modifying syscalls, and whose producer the user-mode patcher cannot reach. Defender and MDE detections are multi-signal and may be cloud-enriched; the architectural point is narrower: the patch operated on a ntdll trampoline, while EtwTi is emitted from a different layer entirely.

That single asymmetry (one provider goes dark to a one-byte patch, another fires from a place the patcher cannot touch) is the spine of the interlude. Around it sits a 26-year story of one Microsoft team accidentally building the substrate of every modern Windows endpoint security product. The credential dump the operator tried to hide is exactly the tradecraft the Mimikatz chapter (Chapter 14) traces and the secret the Credential Guard chapter (Chapter 15) moves out of reach; this chapter is about the layer that sees the attempt.

To understand why a one-byte patch silences one provider but not another, we have to go back to a Windows 2000 design decision about per-CPU ring buffers.

ETW in Windows 2000: the performance problem that started it all

Imagine a 1999 network-driver author. A customer's NT4 production server is corrupting packets under load and the only available instrumentation is DbgPrint. Each call serializes through a kernel debug port, costs measurable percentage points of CPU on a busy box, and ships data to whoever happens to have the kernel debugger attached. The customer says no. The bug reproduces only at production traffic levels. You cannot ship enough printf-debugging through a debug port to find it.

That is the engineering pain Insung Park and Ricky Buch's team was solving when ETW shipped with Windows 2000. Their design moves (recorded years later in the definitive April 2007 MSDN Magazine article on the Vista upgrade [1075]) still define the architecture two and a half decades later.

The first move was per-CPU ring buffers. A producer on CPU 7 writes to CPU 7's buffer with no lock contention against producers on other CPUs. Hot-path tracing on a 64-core machine does not serialize. The kernel allocates at least two buffers per logical processor [1076] so a producer can keep writing while a writer thread drains the previous buffer.

The second move was an asynchronous writer thread. The producer never blocks on disk I/O. It writes to its CPU's buffer and returns. A separate kernel thread drains buffers to file or hands them to a real-time consumer. ETW pushes the latency tax onto the consumer and the storage path, never onto the producer's hot loop.

The third move was dynamic enable and disable. Park and Buch describe the resulting capability in one sentence:

ETW gives you the ability to enable and disable logging dynamically, making it easy to perform detailed tracing in production environments without requiring reboots or application restarts.: Park & Buch, MSDN Magazine, April 2007 [1075]

That sentence is the entire reason ETW could later become the EDR substrate. A producer compiles its trace points into shipping code at low cost; a controller flips them on at runtime when somebody actually wants the data. Without that property, you cannot build a security product that ships universal kernel tracing on a billion endpoints.

The fourth move was the trichotomy of providers, controllers, and consumers [1077]. Microsoft did not write ETW as an internal-only facility. From the start, third parties could write providers (driver authors instrumenting their own code), controllers (performance tools starting and stopping sessions), and consumers (analyzers reading event streams). The architecture is open by design.

The original Windows 2000 implementation supported 32 trace sessions running simultaneously [1078], a number Microsoft later raised to 64 globally. ETW was framed as a developer-diagnostics facility (the Windows Driver Kit primary still describes it that way [1077]) and the security-telemetry use case did not exist for almost a decade.

But the design choices that made ETW good for low-overhead production diagnostics turn out to be exactly the design choices a security telemetry bus needs. Per-CPU buffers solve the multi-core throughput problem. Asynchronous writes solve the producer-latency problem. Dynamic enable solves the always-shipping-but-mostly-off problem. The trichotomy solves the third-party-extensibility problem. Twenty-six years later, modern Windows EDRs commonly consume telemetry through the same four primitives, alongside product-specific drivers, callbacks, and cloud pipelines.

The 2000-era design carried one limit, however, that turned out to matter for security: only one trace session could enable a classic provider at a time. The next ten years would be defined by the consequences.

The MOF era: one session, one steal, one decade of coexistence pain

In 2005, a third-party performance monitor that registered a classic provider could find itself silently disabled the moment Microsoft's wprui.exe started its own session against the same provider GUID. The first session got no error. It just stopped receiving events. That second-consumer-steals-first behavior is the architectural fact of the entire 2000-2007 era.

Microsoft Learn still documents the rule in one sentence:

That single rule made multi-EDR coexistence on classic providers structurally impossible. If Defender's predecessor and a third-party HIPS both wanted real-time process events from the same classic provider, they had to fight for it. The loser got silence with no notification.

The provider class involved was MOF-based, named after the schema language that described its events.

The MOF model was workable for a single-consumer world. A performance-tuning team running an in-house tool could enable the provider, capture, and disable. As the substrate of a security stack with multiple agents on the same host, it could not work. The mid-2000s had not yet produced a "multiple agents on the same host" world, so the limit did not bite immediately. By 2007 it would.

A handful of classic providers survived the 2007 transition and are still significant. The most important legacy anchor is the NT Kernel Logger [1078], the special-purpose system session that captures high-throughput kernel events such as file I/O, disk I/O, registry operations, and TCP/IP network events; Microsoft documents it as the only session that can accept events from the classic kernel event providers [1080]. On Windows 7 and later, the SystemTraceProvider path widened that model, and on Windows 8 and later it can be multiplexed for up to eight logger sessions [1081]. Diagnostics tools use this kernel-trace family when they need line-rate kernel events; Sysmon, by contrast, is better understood as a callback-then-publish design, as described later.

By 2007 Microsoft knew the one-session limit had to go. The fix shipped with Windows Vista in January 2007, and it was the central architectural decision of the entire ETW-as-EDR-substrate story.

Vista's eight sessions: the architectural decision that made the modern EDR endpoint possible

Park and Buch open their April 2007 MSDN Magazine article with the line that frames every later development:

On Windows Vista, ETW has gone through a major upgrade, and one of the most significant changes is the introduction of the unified event provider model and APIs.: Park & Buch, MSDN Magazine, April 2007 [1075]

The new model raised the per-provider session cap from one to eight. That single number is why Defender, CrowdStrike Falcon, SentinelOne, Sysmon, and a researcher's SilkETW tap can all read Microsoft-Windows-Kernel-Process [1082] from the same host today without one of them stealing events from the others.

The Vista model also unified two things that had been separate. ETW providers wrote to per-CPU ring buffers; the Win32 Event Log was a different facility with its own writer, its own format, and its own consumers. Park and Buch describe the unification verbatim:

The new unified APIs combine logging traces and writing to the Event Viewer into one consistent, easy-to-use mechanism for event providers.: Park & Buch, MSDN Magazine, April 2007 [1075]

After Vista, a manifest-based provider can use the unified eventing APIs to feed ETW trace sessions and Windows Event Log channels from the same provider definition, depending on how the manifest's channel mappings are configured. Event Viewer is therefore a view over Windows Event Log channels backed by that unified provider model, not a real-time ETW consumer equivalent to an EDR sensor.

The deployment pipeline for a manifest-based provider is heavier than for a classic provider. The author writes a manifest, compiles it, embeds the resource, and runs wevtutil im at install time. Microsoft Learn calls out the distinction between provider registration and manifest installation [1083] explicitly, and notes that each process can register up to 1,024 providers [1083]. In practice few processes come close.

The cap rules now read like this: eight trace sessions can enable a manifest-based provider concurrently [1084]; up to 64 sessions can run on the system at once [1078]; EnableTraceEx2 returns ERROR_NO_SYSTEM_RESOURCES when the per-provider cap binds [1085]. The 8-session number was chosen for ergonomics, not for security planning, but it is the load-bearing number in modern Windows endpoint security.

A Windows 7-era driver author shipping the inaugural Microsoft-Windows-Kernel-Process provider, GUID {22fb2cd6-0e7b-422b-a0c7-2fad1fd0e716}, authored a manifest declaring ProcessStart (event ID 1), ProcessStop (event ID 2), ImageLoad (event ID 5), and so on. Defender's MsMpEng.exe could subscribe; the future CrowdStrike Falcon could subscribe; the future Sysmon could subscribe; the future SilkETW researchers could subscribe. None starves another. The Vista unification is the architectural enabler of the modern multi-EDR Windows endpoint.

With multi-consumer concurrency solved, the next problems were authoring overhead and producer integrity. Two parallel paths branched off the Vista manifest model: TraceLogging for the first, the EtwTi PPL/ELAM gate for the second.

Two more provider classes: WPP for the kernel tree, TraceLogging for the app tier

Vista's manifest-based providers solved coexistence and decoding, but they were heavy to deploy. Microsoft shipped two more provider classes (one older than Vista and one younger) that traded manifest deployment for two different kinds of simplicity.

WPP: the C-preprocessor approach

WPP (Windows software trace PreProcessor) predates Vista. Community references and the Park & Buch description of ETW being "abstracted into the Windows preprocessor (WPP) software tracing technology" [1075] place its first WDK ship in the Windows XP era; no Microsoft primary pins a specific build. It became the standard tracing facility inside the Windows kernel tree itself for years. The WDK page [1086] frames its purpose:

"WPP software tracing supplements and enhances WMI event tracing by adding ways to simplify tracing the operation of the trace provider. It is an efficient mechanism for the trace provider to log real-time binary messages."

A WPP provider is authored in C with macros that look like printf calls. The C preprocessor expands DoTraceMessage(FlagId, "Frobnicating widget %d", widgetId) into an EventWrite call against an auto-generated provider GUID. Format strings are extracted at build time into a Trace Message Format file embedded in the binary's .pdb. The producer cost is the smallest of any ETW provider class: emitting an event is a function call plus a few stores into a buffer. There is no manifest to deploy, no XML to author.

The corresponding decode cost is the highest. A WPP event arrives at the consumer as a binary payload referencing a TMF identifier. To turn that into a human-readable message the consumer needs the producer's .pdb file. If you do not have the symbols for the binary that emitted the event, you do not know what the event means.

That decode cost is why WPP did not become the EDR substrate. Sealighter's README puts the operational consequence verbatim:

WPP providers also fail the multi-EDR test for practical reasons even where the underlying ETW session rules vary by implementation: consumers need symbols/TMF data, provider inventories are hard to discover, and the resulting stream is not a stable public security contract. So WPP became the kernel-tree internal tracing facility: ubiquitous inside Microsoft's source tree, marginal for public EDR telemetry.

TraceLogging: schema in the payload

Eight years after Vista, in Windows 10 (2015), Microsoft shipped a parallel path that solved a different problem. TraceLogging [1088] keeps the eight-session cap of manifest providers but eliminates the manifest deployment burden:

"TraceLogging is a system for logging events that can be decoded without a manifest.": Microsoft Learn, About TraceLogging [1088]

A TraceLogging event carries its own schema inline. The event payload is a sequence of typed-length-value triples: a one-byte type tag, a length, and the data. A consumer that has never seen the provider before can still decode the event because the names and types of every field are in the event. The provider author needs no XML manifest, no mc.exe, no wevtutil im.

The trade-off is per-event size. Inline schema strings cost bytes per event. For a high-volume provider emitting millions of events per minute, the per-event size matters and a manifest-based provider is correct. For a new component author who wants tracing without an install-time deployment dance, TraceLogging is the right answer.

TraceLogging is also the unified path across runtimes. The same self-describing payload format is emitted from native C/C++, from .NET (when an EventSource opts into EtwSelfDescribingEventFormat), and from kernel-mode drivers [1089]. A consumer using TDH (the Trace Data Helper API) decodes them without distinguishing between the runtime that emitted them.

Four classes, four trade-offs

Sources for the table: [1084], [1079], [1088], [1086].

Four provider classes, four trade-offs. But every one of them shares a structural weakness: the producer fires from inside the calling process, and any code in that process can patch the runtime entry-point and silence the provider for itself. That is the weakness Adam Chester made famous in 2020, and the one EtwTi was built to defeat.

Sessions, buffers, and the autologger registry: where the telemetry actually lives

Open regedit on a Windows host and navigate to HKLM\SYSTEM\CurrentControlSet\Control\WMI\Autologger. You are looking at the persistence surface of every trace session that survives a reboot on this machine, and the persistence surface every modern EDR uses to install itself.

A session is the unit ETW actually exposes to controllers. It owns a per-session pool of buffers, a writer thread, a destination (file or real-time consumer), and a list of providers it has subscribed to. The lifecycle is short. A controller fills out an EVENT_TRACE_PROPERTIES structure [1076] with a session name, buffer size, logging mode, and destination, then calls StartTrace. The kernel allocates the buffers (at least two per logical processor [1076]) and returns a session handle. The controller then calls EnableTraceEx2 [1085] for each provider it wants to subscribe to, passing EVENT_CONTROL_CODE_ENABLE_PROVIDER along with the provider GUID, level, and keyword bitmask.

If the provider's per-class session cap is already saturated, EnableTraceEx2 returns ERROR_NO_SYSTEM_RESOURCES. If the caller lacks the privilege to enable that provider, it returns ERROR_ACCESS_DENIED. We will see both error codes again later, on different paths.

There are three logging modes. File mode writes events to a sequential .etl file on disk; the writer thread drains buffers to disk and the file grows. Circular mode writes to a fixed-size file in a circular buffer; old events are overwritten when the file fills. Real-time mode delivers events to a real-time consumer process, which receives them through its registered event-record callback. Defender, EDR sensors, and Sysmon all use real-time mode for their hot paths; they may also write to file as a forensic backup.

The autologger registry path is what makes a session survive a reboot. A subkey under HKLM\SYSTEM\CurrentControlSet\Control\WMI\Autologger\ defines a session that the kernel starts at boot, before most user-mode services are running. Each subkey's values configure the session: BufferSize, MaximumBuffers, LogFileMode, FileName, plus a nested <SessionName>\<ProviderGuid> subkey for each provider to enable.

Defender's DiagTrack, Microsoft-Windows-Diagnosis-PCW, the SQM kernel logger, the EventLog-Application channel autologger. All live here (observable via logman query -ets on a stock Windows install). Third-party EDRs add their own. The Palantir CIRT taxonomy [1090] (about which more in the gap-analysis section) frames this registry surface as the persistent-tampering target: an attacker who can write to this subtree can disable an EDR's boot-time tracing without ever interacting with the running EDR process. The events of interest never get captured because the session never starts.

There is a related concept worth naming: the Global Logger. This is a special autologger session whose configuration lives in HKLM\SYSTEM\CurrentControlSet\Control\WMI\GlobalLogger. It is the boot-time tracing path that comes online before any user-mode service, including before Sense and the EDR sensor. It exists to capture early-boot kernel events that no later session can record.

Persistence solved: a session the OS starts at every boot. But who reads it? That requires a consumer process, and consumers are where the architecture forks along the security spectrum.

Consumer architecture: from OpenTrace to KrabsETW to a real-time process watcher

The consumer side of ETW is mechanically simple (three calls to open a trace, register a callback, and process events) but the choice of library tells you almost everything about what kind of EDR you are building.

The native pattern is three Win32 calls. EnableTraceEx2 subscribes the session to a provider GUID with a level and keyword bitmask. OpenTrace returns a handle on the session for consumption. ProcessTrace blocks the calling thread, drains events from the kernel's per-CPU buffers, and dispatches each one to a registered callback. Each event arrives as an EVENT_RECORD containing a header (provider GUID, event ID, level, keyword, opcode, timestamp, process ID, thread ID) and a payload that the consumer decodes.

For manifest providers the consumer decodes via TDH (the Trace Data Helper API) against the system-installed manifest. For TraceLogging providers the consumer decodes from the inline TLV payload. For classic and WPP providers the consumer needs the MOF schema or the producer's PDB respectively.

In production almost no one writes the raw three-call pattern. The library universe settled into a small set of widely-used wrappers, and the choice of wrapper maps almost one-to-one onto the kind of EDR the engineering team is building.

krabsetw [1091] is a Microsoft-authored C++ library that simplifies session and provider management. Its README explicitly notes the production caller: a C++/CLI wrapper called Microsoft.O365.Security.Native.ETW, "used in production by the Office 365 Security team. It's affectionately referred to as Lobsters." If you are building an in-house EDR or a security analytics pipeline in C++ on Windows, krabsetw is the default choice.

Microsoft.Diagnostics.Tracing.TraceEvent [1092] is the general-purpose .NET ETW library, distributed as a NuGet package and used heavily inside the .NET diagnostics community. Microsoft's separate Microsoft.Windows.EventTracing.Processing.All package is the .NET TraceProcessing API [1093] that the Windows engineering team uses internally to analyze ETW data from the Windows engineering system.

SilkETW [1094], originally released by Ruben Boonen at FireEye in March 2019 [1082] (now maintained by Mandiant), wraps Microsoft.Diagnostics.Tracing.TraceEvent to expose ETW telemetry to detection-engineering and threat-hunting workflows. SilkETW is the canonical "blue team research" consumer: the tool you reach for when you want to see what events a provider actually emits without writing C++.

Sealighter [1087], by pathtofile, is a krabsetw-wrapping C++ tool that makes multi-provider subscription and filtering tractable from a JSON config. The README states: "Sealighter leverages the feature-rich Krabs ETW Library to enable detailed filtering and triage of ETW and WPP Providers and Events." Sealighter is the canonical "red/blue team triage" consumer: more flexible than SilkETW, less code to write than raw krabsetw.

The pitfalls are universal across all four libraries. The krabsetw README spells two of them out:

"The call to 'start' on the trace object is blocking so thread management may be necessary.", [1091]

"Throwing exceptions in the event handler callback... will cause the trace to stop processing events.", [1091]

Both have caused real production outages. An EDR that throws an unhandled exception in its event callback dies silently as an ETW consumer, and the next event the provider emits goes nowhere.

A real-time Microsoft-Windows-Kernel-Process consumer, in production form, is a working EDR sensor's process watcher. Modern Windows EDRs commonly contain a component with this shape: a controller enabling providers, a real-time consumer draining sessions, and a separate pipeline for enrichment and response. Products that also ship drivers add kernel callbacks beside it rather than replacing it.

This is what Sysmon, Wazuh, and Elastic Defend look like under the hood: a SYSTEM-privileged user-mode service consuming public providers. But there is one provider such a consumer cannot subscribe to. Try it and EnableTraceEx2 returns ERROR_ACCESS_DENIED. The following sections are about the GUID that requires a passport.

Reproducible Windows checks, not captured lab proof

This chapter does not claim a captured lab transcript. The checks below are documented Windows commands whose output is intentionally host-specific: an enterprise Defender endpoint, a CrowdStrike endpoint, a Sysmon lab VM, and a Wazuh collector will not have the same sessions, providers, autologgers, or protection levels. Treat this section as a field checklist for proving the architecture on your host, not as universal sample output.

The useful way to run the checklist is in layers. First ask which trace sessions exist. Then ask which providers are registered. Then inspect the special providers and boot-persistent sessions that determine whether an EDR is seeing early activity, memory-modifying syscalls, and tamper attempts. Finally, inspect the platform controls that decide whether kernel-mode ETW tampering is a realistic path.

The security provider catalog: what EDRs actually read

There are roughly 1,300 manifest-based providers shipped on a 2026 Windows 11 24H2 install: the community-maintained jdu2600 inventory [1095] tracks the count across builds, and the repnz manifest archive [1096] holds byte-stable copies of the manifests for cross-version diffing. Ten rows below carry the core security telemetry most often discussed in public EDR documentation; grouped by function, they collapse into roughly eight categories. This is the catalog.

Microsoft-Windows-Security-Auditing

GUID {54849625-5478-4994-A5BA-3E3B0328C30D}. The audit-policy-driven Security event log producer. Event ID 4624 (logon), 4625 (failed logon), 4634 (logoff), 4688 (process create with command line) [1097] [1098], 4689 (process exit), 4768/4769 (Kerberos TGT and service-ticket requests), and the broader subcategory audit policy events. This is where the interlude watches the authentication links directly: the logon and ticket events are the observable shadow of the protocols the The Death of NTLM chapter (Chapter 16) and the Kerberos chapter (Chapter 17) dissect, and 4769 service-ticket anomalies are the on-host signal of the KRBTGT forgeries the KRBTGT chapter (Chapter 18) describes. This is the closure for the legacy Security event log: when an administrator turns on "audit logon events" in the local security policy, this is the provider that emits the events. EDRs that consume it are reading the same stream the Event Viewer's Security log shows.

Microsoft-Windows-Kernel-Process

GUID {22fb2cd6-0e7b-422b-a0c7-2fad1fd0e716}. The canonical real-time process telemetry source for non-PPL EDR. Event ID 1 fires on ProcessStart with process ID, parent process ID, create time, session ID, and image name (notably not the command line, which is why command-line visibility requires Sysmon Event ID 1 or Security 4688 with command-line auditing enabled); event ID 2 on ProcessStop; event ID 3 on thread create; event ID 4 on thread exit; event ID 5 on ImageLoad with the loaded module name and base address. SilkETW's launch post enumerates the event record format inline [1082]. This provider is widely cited in EDR community documentation as available since Windows 7, though no Microsoft primary pins the exact build.

Microsoft-Windows-kernel-file, Microsoft-Windows-kernel-network, Microsoft-Windows-kernel-registry

The per-subsystem siblings of Kernel-Process. Kernel-File surfaces file open / close / read / write / delete operations with the file path and the operating PID. Kernel-Network surfaces TCP and UDP send / receive with the local and remote endpoints. Kernel-Registry surfaces registry create / open / set value / delete with the key path and value name. On current builds these names appear in community manifest inventories [1095], [1096], while Microsoft documents the related SystemTraceProvider path and its eight-session multiplexing separately [1081]. Treat them as kernel telemetry surfaces whose exact control path is build-dependent; EDRs that want broad file, network, or registry observation can subscribe to these streams, but products that need enforcement or lower latency still write kernel callbacks.

Microsoft-Antimalware-Scan-Interface

GUID {2A576B87-09A7-520E-C21A-4942F0271D67}, documented in the Microsoft Learn AMSI portal [700] and surveyed in the Palantir CIRT taxonomy [1090]. This is the ETW provider that surfaces AMSI scan results: a script block submitted by PowerShell, JScript, VBA, an Office macro engine, or any other AMSI client comes through here after deobfuscation. Whatever string the AMSI client actually submits, the registered antimalware engine can inspect in its deobfuscated form; ETW visibility then depends on the provider, keyword mask, channel configuration, and consumer permissions rather than on provider existence alone.