Windows System Programming In Rust

Rust is a system programming language, but it’s also a modern language with a strong safety story. That combination is the reason it keeps showing up in places where C and C++ used to be the only viable option.

In this post I’m going to show how to get started with Windows system programming in Rust. The assumption is that you know a bit of Rust already, but I’ll keep the Rust-specific parts simple and focus on what changes when you start calling Win32 APIs.

The example I’m going to build is a classic: enumerate processes using the Toolhelp APIs (CreateToolhelp32Snapshot, Process32First, Process32Next). It’s simple enough to be a first project, but it forces us to deal with the key topics: crates, features, unsafe, error handling, and UTF-16 strings.

Safe Rust And Unsafe Rust

Rust’s safety guarantees apply to safe Rust: you don’t get use-after-free bugs, garbled pointers, or the usual memory hazards that come from manual lifetime management. But the moment you start interacting with an operating system API that was designed for C, you’re going to cross into unsafe territory.

That’s not a Rust limitation. It’s just reality: the OS APIs are mostly C-style interfaces. Rust can’t prove at compile time that you’re using them correctly, so you explicitly mark the call site as unsafe. Think of it as telling the compiler: “I’m taking responsibility for this part.”4).

Choosing Windows Bindings: windows-sys vs windows

For Windows specifically, you have a few options in the ecosystem, but Microsoft maintains two crates that matter here:

• windows-sys: low-level bindings, close to the raw Win32 types and functions.
• windows: higher-level wrappers built on top of windows-sys, generally more ergonomic for Rust code.

For this example, I use the windows crate. It still exposes the Win32 calls, but it’s a better starting point if you want code that feels like Rust rather than a direct mechanical translation of C headers.

Creating The Project

Start a new executable project:

• cargo new (I call the sample project ProcList)
• Open it in your IDE of choice

Most people use VS Code with the Rust Analyzer plugin, which is fine. I’m using RustRover from JetBrains in the video, but the tooling choice isn’t the point—Cargo and the crate setup are what matter.

You’ll see two things immediately:

• Cargo.toml (your project metadata and dependencies)
• main.rs (the default “Hello world”)

Adding The windows Crate And Enabling Features

Add the windows crate to Cargo.toml. The key detail is features.

Win32 is huge. You do not want to enable everything. The correct approach is to enable the minimal set of modules you actually use.

For Toolhelp process enumeration, you’ll need the module that contains the Toolhelp APIs, and typically you’ll also need Foundation types.

A practical tip: don’t guess feature names. Use the windows-rs documentation tooling that maps API names to features. If you search for a function like CreateToolhelp32Snapshot, it tells you which feature to enable. This saves time, and it avoids the “why won’t this import compile?” loop.

Calling Win32 APIs From Rust

Once you import the right modules, you can start calling Win32.

The first Toolhelp call is CreateToolhelp32Snapshot.

Two things you’ll notice immediately:

1. It’s unsafe
   All Win32 calls will require unsafe { ... }. That’s normal, as these are C APIs, which are unsafe by definition.
2. You’ll get Rust-style Results
   Instead of manually calling GetLastError after every step, many of the bindings surface results as Rust Result<> values.

For a first demo, it’s common to use unwrap() to keep the example focused. That means: if something fails, crash and show the error. For production code, you’ll usually do proper error handling, but for learning the mechanics, unwrap() keeps the noise down.

Enumerating Processes With Toolhelp

The Toolhelp enumeration pattern is the same as in C and C++:

1. Create a snapshot
2. Initialize a PROCESSENTRY32 structure
3. Call Process32First
4. Loop with Process32Next

In Rust, there are a few details you must get right.

Using The Wide (UTF-16) Variants

On Windows, the “real” string API surface is UTF-16. For Toolhelp, that means using the wide types and functions:

• PROCESSENTRY32W
• Process32FirstW
• Process32NextW

The process name field is a fixed-size UTF-16 buffer. If you print it naïvely, you’ll get a dump of numbers, not a readable name.

We’ll fix that in a minute.

Initializing The Structure Correctly

Toolhelp functions require you to set dwSize to the size of the structure.

In C++ you often do something like:

• zero the struct
• set dwSize

In Rust, you can’t leave memory uninitialized. You typically start with a zeroed structure (or a default initializer provided by the bindings), then set:

• entry.dwSize = size_of::<PROCESSENTRY32W>() as u32;

If dwSize is wrong, enumeration fails. This is one of those Win32 rules you can’t ignore.

Handling Mutability The Rust Way

Rust variables are immutable by default.

If the API is going to write into your process entry structure, the structure must be declared with mut. The snapshot handle doesn’t necessarily need to be mutable, but the entry does.

This is a small Rust difference that matters when you translate Win32 patterns into Rust.

Converting UTF-16 Process Names Into Strings

The executable name in PROCESSENTRY32W is a UTF-16 buffer. Rust natural strings are UTF-8.

A common and practical approach is:

1. Convert UTF-16 into a Rust String
2. Trim at the first null terminator

For example (conceptually):

• Convert the entire array with String::from_utf16_lossy(&entry.szExeFile)
• Then cut it at the first '\0'

That gives you a normal readable process name.

A small Rust feature that helps here is variable shadowing. You can reuse the same variable name for each transformation step (raw → string → trimmed) without mutating the original binding. It keeps the code readable.

A Minimal Mental Model For The Final Loop

At a high level, the logic looks like this:

• Take snapshot
• Initialize entry, set dwSize
• If Process32FirstW succeeds:
  • print PID + process name
  • loop while Process32NextW succeeds:

The “Rust work” is mostly about correctness at the boundaries:

• setting structure sizes
• using unsafe where required
• converting UTF-16 to something printable

Why This Matters For TrainSec Students

If you do Windows Internals work, you end up writing small tools sooner or later. Sometimes it’s for debugging. Sometimes it’s for IR. Sometimes it’s for research or experiments.

Rust gives you a good balance:

• Most of your code stays in safe Rust
• The dangerous parts are constrained to explicit unsafe blocks
• You still get direct access to the Win32 APIs you need

This example is intentionally simple, but it builds the foundation you need for anything bigger:

• using the windows crate properly
• choosing the right features
• dealing with handles and structs
• handling UTF-16 correctly
• working comfortably with Win32 from Rust

Once you can do process enumeration cleanly, it’s a short step to building more advanced tooling.

Keep Learning with TrainSec

This content is part of the free TrainSec Knowledge Library, where students can deepen their understanding of Windows internals, malware analysis, and reverse engineering.Subscribe for free and continue learning with us:

https://trainsec.net/library