Fun with XNA, part 2

It's been four months since I've last written about my XNA project -- an eternity! I haven't been idle, most of my work has been behind the scenes on pipeline stuff.

When I left off last I had the world geometry rendered, with basic lightmaps and textures, but there was quite a bit missing. Quake 3 was quite a vibrant game, in large part due to the multitexturing effects that were used. Surfaces animated and swirled, effects were stacked on top of each other, lights flickered and pulsated. If all I'm drawing is polygonal geometry with one texture and maybe a lightmap it's not going to look anywhere near that good.

Quake 3 achieved most of its visual effects with shaders, though being released in 1999 it was many years ahead of languages like HLSL/GLSL/Cg, and even ahead of fragment/vertex assembly languages. Contrast that with today where our world is nearly at a point where el-cheapo feature phones have programmable graphics pipelines. Quake 3's shaders were relatively simple, specifying a texture that could be used, how it was to be sampled and its colours manipulated, and how it was to be blended with others.

I needed a way to get these simple Quake 3 shaders into my engine, into a more XNA-friendly format. There were a few approaches I tried.

First, I tried hand writing a couple shaders. Brute force is occasionally worth it, but not here. It took way too long to be a viable process, but at least I figured out what I would need to do.

Next, I tried generating shaders at runtime but this I shelved because XNA doesn't support runtime shader compilation when deploying applications.

I took the beginnings of the previous step, generating the shaders programmatically, and worked it into a pipeline step. This first step to generating them in a pipeline was to parse the existing Quake 3 shaders. Since the format was documented (here is the Q3 Shader Manual), I figured this was going to be cake. Sadly, it wasn't documented well enough, and I was getting a slew of unexpected tokens and effects. I ended up guessing as to what they might be, within the context they were used, and discarded the rest with a TODO attached that I will take care of them later :).

Once the Q3 shader was parsed I had to turn it into a list of textures, some flags, and a pile of HLSL. The textures and flags were pretty easy, but the HLSL less so. I tried a simple text-based HLSL generator but keeping track of state and variables became messy in short order.

After putting the project down for a few weeks, I was sitting on the couch thinking that it would be nice if it was as easy to build a shader as it was to build C# using Microsoft's CodeDOM. Out of that, I put together a library, HlslDom (which is hosted on GitHub! https://github.com/maxburke/HlslDom), to stitch together HLSL programs via code.

This weekend I was at home, sick, and didn't have much to do but work on HlslDom and this shader compiler. The shader compiler is now done, it generates valid HLSL, and I'm getting the HLSL shaders integrated back into the game portion! I'm hoping to have screenshots following soon.