3 Driver Development Environment

3.1 What hardware do I need to develop Windows NT (WNT) Kernel Mode Drivers (KMDs)?

If you're using the Microsoft debuggers, then you need two systems running WNT, one to exercise the driver and one to run the kernel debugger UI. They are referred to as the target and host systems, respectively. As far as the debugger is concerned, these need not be the same processor architecture (Intel, Alpha, etc.).

NuMega Technologies has a product, SoftICE for NT, that offers a single-machine kernel debugging environment, for x86 systems only. (See the Related Products links.) If you use that exclusively, you can get away with just one NT system, unless you're also targetting the Alpha as well as x86.

The driver must be compiled and linked on a WNT system of the same processor architecture as that on which it is to run, as there are no cross-compilers available. In other words, if you're building a driver for an Alpha, you have to do the compile and link on an Alpha. This may be the target system or any other system of the same processor architecture, including the debug host if it is of the same architecture of the target.

In the two-machine debugging environment it is easiest if you build the drivers on the host system (the one where you'll run the debugger UI). This requires however that the two machines be the same architecture.

3.1.1 I'm confused about this "host" and "target" stuff.

Ok, let's describe the two most common scenarios:

Compile and link the driver on the host system. Copy the resulting driver executable to the target system. Start the debugger UI on the host. Load and exercise the driver (perhaps by running test programs) on the target. Interact with the debugger on the host.
Compile and link the driver on the target system. Copy the driver executable and source files to the host system, as the debugger needs both of these. Start the debugger UI on the host. Load and exercise the driver (perhaps by running test programs) on the target. Interact with the debugger on the host.

The debugger will work cross-platform (that is, host and target can be different architectures), but the compiler and linker won't. If you're developing for Alpha as well as x86, the absolute bare minimum would be one Alpha and one x86 system; you can use each as a debugger host for the other.

3.1.2 I thought I heard somewhere that you could use a Windows 9x system as the host for the debugger.

Not today. Some time ago, Microsoft unofficially stated that Windows 9x might be usable in the then-near future, but as of this writing the kernel debugger is dependent not just on Windows NT but on the specific version of Windows NT under which the target system is running.

Update: Someone has claimed to have gotten the debugger UI working on Windows 95 by copying the file imagehlp.dll from the NT system to the Win9x system.

3.1.3 I thought I heard somewhere that you could use a serial terminal for the debugger UI.

Nope.

3.2 What software tools do I need to develop KMDs?

Windows NT free build
Windows NT checked build
Software Development Kit (SDK)
Windows NT Device Driver Kit (DDK)
Visual Studio (VS) (sometimes called VC++)

3.2.1 Where do I get all this stuff?

NT free build: Purchase as retail product, or Microsoft Developer Network (MSDN) "Professional" subscription or above
NT checked build: MSDN "Professional" subscription or above
Platform SDK, DDK: MSDN "Professional" subscription or above, or download free (except for network charges) from the Microsoft web site. The CD-ROM is labelled "Platform SDK" (formerly "Win32 SDK").
DDK: MSDN "Professional" subscription or above, or download free (except for network charges) from the Microsoft web site.
Visual Studio: Retail, or MSDN "Universal" subscription or above.

3.2.2 Can't I use someone else's 32-bit C compiler?

Maybe. As long as it generates essentially the same output from the same input, and takes all of the same command line options as the MS compiler, it should work fine. If not, you'll have some work ahead of you. Same for the linker.

Opinion: VS is part of the driver build environment supported by Microsoft, and is furthermore tested nightly by Microsoft on literally tens of millions of lines of NT code. Without a compelling reason to do otherwise, it seems to me to be far easier, not to mention safer, to stick to the supported build environments.

3.2.3 What about other languages like assembler or Pascal?

In theory, as long as you adhere to C calling conventions, don't use too much stack space, etc., you should be fine. In practice, there are several tens of thousands of lines of header files in \ddk\inc that you will have to translate to your language of choice, and this is another of those "unsupported environments" that I worry so much about. Suffice it to say that the productivity gain of the other language would have to be huge before such a project was a net win.

As for assembly language, NT drivers are supposed to be source-portable between processor architectures (x86 vs. Alpha). However, there are of course sometimes good reasons for writing the occasional driver routine in assembly language.

3.2.4 What's all this about C++ and drivers?

The NT kernel team originally shunned C++ and to this day, many people from Microsoft, including the DDK support team (the folks you talk to during DDK support calls), strongly recommend that drivers be written in pure C. However, it is difficult to find an official, unequivocal quote from MS to this effect. As a result, C++'s suitability for KMDs is the subject of a rather spirited ongoing debate. On one side are Microsoft (at least by reputation) and those who (like me) prefer sticking to MS-recommended environments. On the other side are those who feel that the benefits of C++ outweigh the risks of using an unsupported, disrecommended environment. Here is what I hope is a fair presentation of the arguments on both sides.

3.2.5 Why are you so (cautious, cowardly, unimaginative) about using MS-recommended tools and techniques?

Look, I love a good kernel hack as much as anyone. But considering how important it is to get drivers exactly, precisely, positively right, I am reluctant to experiment with other compilers, other languages or language dialects, or anything else that MS does not recommend... especially when I'm shipping a driver to a client. MS's tools may be buggy and their recommendations may be wrong, but at least if I adhere to them I can point back to MS as the culprit. If I ignore MS's recommendations, problems are solely on my head.

3.3 Installing the driver build environment

3.3.1 How do I install all this stuff?

On your "build system" (the one on which you will build the driver), you should have the Free Build of WNT 4.0, Service Pack 3 or later installed. Then install the following packages in this sequence:

Visual C++. Note that VC often gets very unhappy, sometimes failing to complete the installation, if you try to override its default installation directory.
Platform SDK. If disk space is at a premium, ~~get more disk space! It's cheap enough~~ you may choose only the "Build environment" and "Tools" subsets.

Be sure to check the "Yes, update environment variables as part of setup" and "Yes, update Visual C++ directory settings as part of setup" options during the SDK install.
Windows NT DDK. Install it all, it's only 100 MB, and the installation seems to go considerably faster if you don't try to subset it.

3.3.2 Exactly what is the "Platform SDK" and why do we need it?

This question often arises because the SDK seems to have a lot of overlap with the Win32 development environment in Visual Studio, particularly in documentation, header files, libraries, tools, and example sources.

The stock answer is: "Without the SDK installed, the DDK build environment won't work," but it is possible to "fake out" the DDK build environment so as to build drivers without having the SDK present (the result is of course an unsupported build environment).

WinDbg, the MS-supplied GUI debugger for kernel mode drivers, is also part of the SDK, though recent versions of the Beta DDKs for Windows 2000 have included it.

What else? Well, in the early days of WNT, there was no separate 32-bit C compiler product. The Windows NT SDK provided the compiler and rest of the environment necessary for building Win32 apps. Ever since the 32-bit VC/VS products started shipping, the SDK has been less important.

However, there are still some good reasons to install the SDK on top of Visual Studio, even if you're "only" doing application development. WNT Service Packs often add APIs to the system, and support for these will first show up in the libraries and header files of the SDK, because the SDK is updated far more often than is the Visual Studio product.

The SDK also includes some some sample applications in source form, and some tools such as the "Process Walker" and WinObj, that are not present in Visual Studio.

3.3.3 Do we use the VS IDE to build drivers?

There are ways to make it work, but not "out of the box." Instead you build drivers using the command line based build utility that comes with the DDK. You do end up using the compiler binaries that are part of VS, but you invoke them via build . The methods for using the IDE are described later on this page, but if you're new to the DDK I strongly recommend that you get the standard DDK build environment working first.

3.4 Building some sample drivers

3.4.1 Ok, I installed all this stuff. Now what?

The DDK install has created a \ddk directory. Browse around \ddk\src and you will find many subdirectories full of sample driver source files. In the following steps we will build a subset of these, just to verify that everything's installed correctly. (You can build them all if you like, but it will take a while.)

Go to your Start menu, find the "Windows NT DDK" program group, and click on the shortcut "Checked build environment". The result should be a command window with its default drive and directory set to \ddk.
Set the default directory in this command window to a subdirectory under \ddk\src . I usually use \ddk\src\comm (serial and parallel port drivers) for demonstration purposes.
Type the word build at the command prompt.

Results:

At the end of the run, there should be a report of some number of source files compiled and a smaller number of links performed.
You should find the latter number of new .sys files in the directory \ddk\lib\i386\checked (or \ddk\lib\alpha\checked if you're on an Alpha).
You should find a file called build.log in the directory from which you invoked build.
You should not find files called build.err (hard errors reported by the compiler or linker) or build.wrn (warnings reported by the compiler or linker).

3.4.2 (Assuming it worked) What just happened?

build looks for text files called dirs. and sources.in the current directory. There's a dirs. file in each non-leaf node of the tree, and a sources. file in each leaf node. The dirs. file is a simple text file that defines the branches of the tree under each non-leaf node; the sources. file defines the driver to be built from the source files in its leaf node. This definition is in the form of nmake-style macro definitions for macros of predefined names such as SOURCES, INCLUDES, and so on.

Wherever build finds a sources. file, it invokes nmake, with the default directory set to that of the current sources. file, and with nmake macros defined by the sources. file in effect for that invocation of nmake .

In each directory where there's a sources. file there should also be a file called makefile. This file is the same for every driver; it simply includes the standard DDK makefile, \ddk\inc\makefile.def . \ddk\inc\makefile.def is quite large and complex and should not be modified by the driver writer. It is written in terms of the macros that you can define in your sources. file, and for most drivers, the sources. file allows you to completely describe your driver.

build is further described in the DDK documentation, Driver Writer's Guide, Chapter 3, "Building Windows NT Drivers". A complete description of the possible contents of a sources. file can be found in \ddk\doc\sources.tpl .

3.4.3 What's the "checked" vs. "free" build environment?

You use the two "build environment" program items to create console windows in which to run the build utility to build drivers. Each of these invokes \ddk\bin\setenv.bat, which in turn invokes \mssdk\bin\setenv.bat, with command line options that cause these batch scripts to create environment variables that in turn are use by build to control various compiler and linker options.

The result is that if you build a driver under the "checked build environment", the compiler is invoked with most optimizations turned off, full symbolic debugging information in the driver binary, and so on. This is usually what you want to use if you're going to debug the driver in question.

The compile-time symbol DBG is also defined in this environment, so that code within blocks such as

#ifdef DBG // tests for errors that should never happen #endif
will be included in the generated code.

On the other hand, if you build a driver under the "free build environment", most compiler optimizations are turned on, only very limited symbolic debugging information is generated, and the symbol DBG is undefined. This is usually how you build the version of the driver that will undergo long-duration stress testing and, if it passes, will be shipped to users.

3.4.4 Does `build` have any other options, modes, etc.?

It sure does. Use the command

build -?
to see them all.

3.5 Problems with `build`

3.5.1 I saw some error messages, and the drivers din't build.

"nmake rc=2": That's "reason code 2" being reported by nmake. Briefly, it can't find the compiler or linker in your path. You're probably trying to invoke build in an ordinary "command prompt" window instead of a window created by the "Free build environment" or "Checked build environment" shortcuts.
"Invalid command format" when opening "Build environment" command window: setenv.bat in the later versions of the SDK uses a comment format that is not understood by cmd.exe prior to NT4 Service Pack 3. (Or maybe prior to SP2; I haven't gone back and checked, but I do know that it doesn't work under SP1, but does under SP3.) The right thing is to upgrade your build system to SP3 (at least). If you insist on not doing that, you can delete the leading block of comments (lines that begin with ;;) from \mssdk\bin\setenv.bat .
"Last line incomplete" for various header files: These messages are benign. They are caused by header files whose last visible character is followed by <EOF> instead of <NEWLINE><EOF>.
If you want, you can get rid of these by running LastLine, which you can download free from NuMega's web site.
"Cannot find include file" for various header files: Strange as it may seem, these are benign too. (Notice that they come from build, not from the compiler, and that the compilation proceeds anyway.) build does its own dependency scan of source files and the header files they reference to try to figure out what's changed and therefore what needs to be built. The trouble is that it is not very smart about following the path defined by the include environment variable, so the header files in \ddk\inc aren't found. But the compiler will find them properly. Furthermore build does not understand conditional compilation directives, so it tries to look at a lot of header files that the compiler won't actually see.
In any case, these errors reported by build will not affect the compilation. If you want to get rid of the messages, add

;$(BASEDIR)\inc

to the tail of the INCLUDES directory list in your sources. file.

3.6 Building your drivers

3.6.1 Ok, so how do I set up a build environment for my driver?

Create a directory anywhere on your system (it need not be under \ddk\src, unless you are using OSR's ddkbuild.bat, described below) and place in it your driver source file(s), a sources. file describing your source file(s), and a copy of makefile. from literally anywhere under \ddk\src .

Or, "copy what works:" Copy all the files from one of the "leaf nodes" under \ddk\src into a directory of your own, change the file name(s) for the driver source(s) to your taste, and edit the sources. file accordingly.

3.6.2 I need to define some additional rules in the `makefile` .

It is possible to add your own rules to those provided in \ddk\inc\makefile.def . You do this by defining macros in your sources. file with names of the form NTTARGETFILEn . Any such macros cause makefile.def to look for makefile.inc in the directory containing the sources. file. makefile.inc can specify dependency rules for files that would not otherwise be built in the standard environment. Several examples of makefile.inc files can be found under \ddk\src\... examine the corresponding sources. files to see how they are used.

3.6.3 I don't like using `build` .

You have a lot of company. There are two easy ways and a hard way around it:

Go to OSR's web site for their article describing ddkbuild.bat and how to set it up. This is just a batch script that you set up as the "custom build command" in VS. You are still using build and a sources. file; VS invokes OSR's the batch script, which in turn runs SETENV.BAT and then invokes build . This is probably as close as you can get to the "real" build environment while still staying in the VS IDE.
Fetch the SrcToDsp tool from NuMega's web site. This parses your sources. file and generates a VS project file that references all your source and header files and sets up the proper compiler and linker options.
(The hard way) Read the build.log file that's generated when you use build the regular way, figure out all the compiler options that were used, and manually tweak the compiler options in the VS IDE to match. I don't recommend this.

I still recommend that you use the standard build utility, in its pure form, when you're generating the final retail version of the driver---the one you subject to long-duration stress testing and then ship if it passes. "Just to be sure." But for the development cycle, the IDE has a lot of advantages.

3.6.4 I want separate intermediate files for the "checked" vs. "free" build environments.

Right. Although build generates the final build targets (driver.SYS files) in separate directories for the "checked" vs. "free" build environments, the intermediate object files always go in single obj/ subdirectory under the directory that contains the respective sources. file, regardless of whether you're in the "free" or "checked" build environment. This can lead to nmake thinking that object files are up to date when they're really not, at least not for the type of build you want.

For example: Suppose you've recently built the free version of your driver and then you go to the "checked build environment". nmake will find that the driver object files but not the .sys file are up to date, and so will do a link but not a new compilation. The result will be a driver.SYS file built from the free object files but with the linker options from the checked build environment. This is not what you want for debugging.

A trivial but annoying workaround is to invoke build with the -c option (clean up all target files before building) for the first build after you change environments.

Much more convenient is to define a different subdirectory for the object files for the free vs. checked build environments. This is easy to do; simply put the following line in your sources file:

CHECKED_ALT_DIR=1
You may have to create the checked object directory, called objd\ , under under each such source root directory. Now the object files built under the "checked build environment" will go in the new objd\ directory, while object files built under the "free build environment" will be put in obj\ as usual.

3.7 Using the Checked Build

3.7.1 What is the Windows NT checked build?

It's also known as the "debug build" and it's analagous to the checked version of your driver. In other words, it's built from the same sources as is the "Free" or "Retail" build of NT, but with most compiler optimizations turned off. Also, the compile-time symbol DBG is defined during the compilation of the checked build, so that code of the form

#ifdef DBG // tests for errors that should never happen #endif
will be included in the generated code.

There is a great deal of such "debug code" in the NT sources, mostly argument validation and other consistency checks on work done within kernel mode.

Another interesting difference is that there is no uniprocessor version of the checked build; when you install the checked built you always get the multiprocessor kernel, even on a machine with just one CPU. The result is that the checked build "goes through the motions" of acquiring and releasing spinlocks and performing other serialization that are actually necessary only on a multiprocessor system, and which the free build on a uniprocessor system avoids.

3.7.2 What is the role of the NT checked build in driver development and debugging?

All drivers should be exercised under the checked build, to the extent of traversing all code paths at least once.

Final long-duration stress testing should be done with the target system running under the free build. Do this under the checked build also if you like, but you must do it at least under the free build, since that is the usual environment on customer systems.

Driver debugging may be done with the target system running under either the free or the checked build. Personally, I use the free build most of the time, simply because it is noticeably faster.

So I don't need to be running the checked build on a system if I want to use it as a debug target?

No.

What about the host system?

There is no reason as far as driver development is concerned to run the host system under anything except the free build.

3.7.3 Is there ever a reason to test applications under the NT checked build?

That's an interesting question. In theory, nothing an application should be able to do should bring out any of the problems the checked build is designed to catch; all of the additional tests in the checked build are there to catch mistakes made from kernel mode.

In practice, however, NT isn't perfect, and it is possible that there are things you can do from user mode that could bring out some of its bugs, and that running under the checked build will bring them out sooner.

Then the question becomes, "so my app crashes NT checked build but not the NT free build. It's correct Win32 code, so what do I do about it?" Well, you can probably isolate the Win32 calls that are causing the problems, and then try things like changing the arguments, changing the frequency with which you make the offending calls, or redesigning to avoid them completely.

3.7.4 How do I install the checked build?

You start an install from the "NT Workstation Checked Build" CD-ROM, just as you'd start an install from any other NT distribution CD-ROM. If this is a system that already has a version of NT on it, it will offer to upgrade your existing NT system. You don't want to do that; instead, tell it you want a new installation, specifing a different directory and/or partition (I usually use \winntchk).

Editing the boot.ini

The checked build installation will create a new pair of boot options in your boot.ini (or, for Alphas, in the console firmware). Alas it will not put "Checked Build" or anything like that in the descriptive text; you will have change the boot menu later to put meaningful descriptions into the text of the options that select the checked build installation. (When editing the options, you can tell which is which by the directory names.) You'll probably also want to create a third "checked" boot menu option that will act as a debugger target. Finally, you'll probably want to make the boot options for the checked build not the default. (Whatever you installed last becomes the default.)

Maintenance

The checked installation is a completely separate NT installation---the only thing it has in common with the original is your boot.ini file, ntdetect.exe, and so on, in your system partition's root directory. The files under %systemroot%, registries, etc., etc., are completely separate, so any changes you make to one install won't affect the other. So all maintenance, such as adding accounts, installing applications, installing NT service packs, etc., has to be done to both systems.

So I've just doubled my maintenance work?

Not really; usually you don't install all the apps, create all the user accounts, etc., on the checked install of NT.

Can they share the same paging file?

Yes.

What about service packs for the checked build?

Microsoft has been somewhat lax in getting out checked versions of the service packs. However, recent MSDN shipments have included these for NT4 SP3 and SP4. These have also been available for download from the Microsoft web site.

Can I somehow boot my regular NT installation with the checked kernel?

Yes. First, go ahead and install the checked build as a separate boot option. Then copy ntoskrnl.exe and hal.dll from the checked installation's %systemroot%\system32\ directory to the free installation, same directory, but under different names, such as %systemroot%\system32\ntoschkd.exe and %systemroot%\system32\halchkd.dll . Then create boot options (in your boot.ini (or, for Alphas, in the console firmware) that include

/kernel=ntoschkd.exe /hal=halchkd.dll
with appropriate descriptive text. You can now boot with the checked kernel and HAL, but free versions of everything else (like all the drivers), and you only have one installation to maintain.

Note that I'm recommending that you take these files from a checked installation, not from the "NT Workstation Checked Build" CD-ROM. The reason is the HALs; there are several on the CD; NT Setup picks the correct one and puts it on your hard drive under the name hal.dll . You want to be sure to get the correct checked HAL for your motherboard. If you don't mind a little more risk, you can pick up the right files yourself from the Checked Build CD; read the Microsoft Knowledge Base (MSKB) article Q156358 to identify which HAL is the correct one.

For ntoskrnl.exe you can copy either ntoskrnl.exe or ntkrnlmp.exe from the "NT Workstation Checked Build" CD-ROM to ntoschkd.exe on your free installation; part of the point of the checked build is that these two files are identical---when you're running the checked build, you're always running the MP executive and kernel, even on a single-CPU system.

top of page | up | previous | next | home