Tuesday, December 28, 2010

HlslDom

My latest project is HlslDom (GitHub: https://github.com/maxburke/HlslDom). Its creation was driven by my XNA project where I needed a solution to create HLSL shaders as part of a pipeline.

It's written in C# and is designed to integrate into .NET applications. Because the target application is XNA it is mostly designed for DirectX 9-level HLSL, though any and all patches are welcome if DX10+ support is desired :)

Tuesday, December 21, 2010

Stop! Enough already!

At which point do you stop adding? Is there an end point for product expansion? Is simplicity inevitably doomed to be displaced by shiny new things?

As a kid I used to love playing sports video games, sometimes even more than playing the actual sports, and every few years I still think I enjoy them. Keyword: think. Every other year or so I'll pick up the latest NHL or Madden video game, drop it into my Xbox 360, and start a new game.

Then the loading screens flash, showing the controller mappings, and my interest usually goes from mildly curious to nonexistent. Every button and control stick is mapped to multiple functions depending on whether you're on defense or offense, with the ball, without the ball. It's mind boggling. Modern first person shooters and RPGs aren't much better but there's usually some sort of ramp up in game play for you to adjust to before you're expected to be a Dual Shock Harry Connick.

There's always a rush to add more, to make it new, to justify a purchase. With some video games it's to the point now where my hands have problems contorting themselves to chord the buttons correctly.

It's not just video games that are succumbing to this. It's also Facebook. It's Twitter. It's Ford. Some of these goods aren't forced though, I don't have to buy a new video game or a new car if I like the old one, I can just stick with what I have. This doesn't really apply to Twitter or Facebook or the (somewhat) new Google search results page -- these services are forcing new functionality down my throat.

Is it possible for a product or a service to say, "nah, let's stop, I think we're as good as we're going to get"?

Tuesday, November 16, 2010

Blog layout changed

I wasn't entirely happy with the previous template in terms of readability, so I've gone to a layout with less contrast and colors that are easier on the eyes (less dark grey and blue, more tan).

Now, if I can only get it to be a single column...

edit: success!

Compile-time size determination

WARNING ugly hack ahead! WARNING

I have a project that has two components, written in two languages, that execute in two different execution environments (PPU and SPU). One component is written in C++ and the other in assembly and since data is passed back and forth fairly regularly the C++ component has a ton of static assertions validating structure size and offsets in the hope that if anything breaks it is caught early.

Finding initial, or updating existing, structure offsets and sizes was tedious. Using printf() worked but the whole project would have to build, it'd be better if it would just fail at compile time with the numbers I needed. Oh, hey, templates might actually work here...

Say, I had:

struct Foo {
// some number of fields here
};

Then:

template
struct fail
{
enum { value = a / 0 };
};
fail f1;

Would give me both this:

test.cpp:13: warning: division by zero in 'a / 0'
test.cpp: In instantiation of 'fail<136u>':

... and a slightly dirty feeling.

(Blogspot still sucks for posting code.)

Retailers angry about Steam; consumers shrug.

Some British retailers are all in a snit about Steam and are threatening to not carry titles that are sold online.

I'm not quite sure how this is any different than the way things are now. The last time I stepped into a video game store the PC games section was non-existent save for the token Sims copy, replaced almost entirely with used copies of Gears of War and Halo.

Buying games in a store used to be a treat, you'd get a well designed box loaded with a decent manual, perhaps a book detailing the back story, and sometimes collectors memorabilia. Before the days of the collectors edition that's what you would get if you bought the normal run-of-the-mill standard edition.

Now game boxes contain nothing but the slimmest of slim manuals (soon to disappear, mark my words), and an online activation code. This is somewhat understandable as video game prices have remained static in the last 15-20 years even in the face of ever increasing costs and inflation.

Buying online saves the trip to the mall, waiting in line, pre-ordering, you name it. I get the game I want, when I want, and these days I'm not missing any part of the experience.

Retailers are free to stop carrying PC games but, really, they already stopped three years ago.

Monday, November 8, 2010

My Five Rules of Corporate Communication

I wrote these mostly with email in mind but some apply just as well when dealing with others in person.

Rule #1: Always be sincere and professional.

Sarcasm doesn't convey well on email or any medium where your audience can't read your body language or receive any speaking cues. This especially applies if your targeted audience isn't familiar with you; although your team might understand and laugh when you're sending out the latest Daily WTF found within your own code base with a snide "yeah, THAT was a good idea" attached but the company-wide developers mailing list likely won't.

Rule #2: Assume people are sincere and professional.

This rule just follows from the first. If you are sincere and professional, other people will be sincere and professional to you.

Rule #3: Assume people keep what you send.

My first manager told me that it might be helpful to keep all the email you ever receive. Boy howdy was he ever right. Not only do I have a wealth of information at my fingertips to search through but it has saved me a few times from people getting away with "No no no, I didn't say THAT...". It also helps when dealing with some people whose turnaround on support issues is a day or two longer than Outlook's auto-delete policy.

This, in turn, has me thinking now that I cannot get away with the same dodge because someone out there will have my original email archived and will call me on my BS. Outlook, like the elephant it is, also never forgets -- beware that the rant you send out may not find its way into the bit-bucket in the sky when you hope it does.

Rule 4: Email conversations are meetings.

Well, almost. The big difference is that there's no one to cut you off in the middle of your diatribe and say that it's Dave's turn to write. Have a point or agenda. Don't beat around the bush. If you diarrhea'd three pages of text to the screen and still haven't reached your point then it's time for some editing. It doesn't just take time for you to write your novella but it also takes many people lots of time to read it. Email, like meetings, takes up time.

Rands in Repose has a great piece on running meetings. In it he says that a well run meeting is one that needs to never happen again. The same applies to email.

Rule 5: Consider your audience.

Double check that you are addressing the right people. Does your boss's boss really need to know that Bob broke the build? Probably not.

Use To:/Cc: properly. Put someone on a To: line if you expect a reply from them, Cc: if you just want them to be aware of what's going on.

Monday, November 1, 2010

Pimples are annoying, C++ Clearasil wanted

Hating C++ seems to be the cool thing to do these days and though I can understand why the sentiment exists, I don't share it. One area I often think is lacking though is the ability to really hide private data without reverting to writing C. Not that I think there's anything wrong with writing C but in a codebase that's largely C++ it can look somewhat out of place.

Idiomatic C and C++ differ quite significantly in the way interfaces are exposed to end users. In many cases with C people use declared structures as opaque handles, preventing end users from mucking around with internal structures. External header files for libraries end up looking something like this:

struct foo;
struct foo *foo_create(int a);
void foo_destroy(struct foo *foo_instance);
int foo_method(struct foo *foo_instance, const char *string, int count);

As a user of this code I have no idea what state foo encapsulates because the library author has made it clear that it isn't something that I should worry about.

C++, though, encourages exposing data by bundling all types and methods of a class into the public declaration.

class foo {
const char *m_string;
int m_count;
bar &m_barInstance;

public:
foo(int a);
~foo();
int method(const char *string, int count);
};

The world now knows what foo contains, whether they need to or not. The idiomatic method of hiding internal details in C++ is to use pointers-to-implementation (pimpl) patterns. These do clean up the class declaration a bit:

struct foo_impl;
class foo {
foo_impl *m_implementation;

public:
foo(int a);
~foo();
int method(const char *string, int count);
};

And behind the scenes, in foo.cpp:

struct foo_impl {
const char *m_string;
int m_count;
bar &m_barInstance;
};

foo::foo(int a)
: m_implementation(new foo_impl)
{
m_implementation->m_count = a;
};


The two big downsides with pimpls are that now every allocation of foo comes with a second allocation of foo_impl, and also that our code is littered with m_implementation.

What I wish C++ had was a way to export the public interface of a class without any consideration to storage or to its own private functions. Something like this:

static_interface foo {
int method(const char *string, int count);
};

I really want to avoid having the extra indirection of a pimpl pointer or indirect jump if it were to just be an interface but while still having the convenience of being able to write foo_ptr->method("hello", 5). This would mean that it could only be used as a pointer/reference but that's no different than with the C example above.

Tuesday, September 21, 2010

Shaders

Working currently on a pipeline component to translate the Q3 shader descriptions into something more modern. It looks like it'll be a fairly straightforward translation to HLSL though the surface tessellation will require a load-time step.

Friday, September 10, 2010

Fun with XNA

When XNA debuted back in 2007 I said to one of my coworkers at lunch that I bet you could easily have a game with the level of fidelity of Quake 3 done entirely in XNA on the Xbox 360. As far as I know nothing like this has yet materialized (though I may be mistaken!), but I figured I'd take a stab at it.

Since I have no art skills to speak of beyond drawing stick figures in the margin of my notebook at work while trying to pass the time in yet another ridiculous meeting I figured the best place to get Quake 3-fidelity assets was to start with... Quake 3. I picked q3dm11 as a starting point, mostly because it was the biggest map file I could see after digging into the PAK.



I also don't really have any graphics programming skills to speak of either so I'm mostly figuring this out as I go. I've got the BSP parsing and the world geometry rendering now (including bezier patches), roughly textured and somewhat lit.



I'm currently using raw textures now for surface detail, but I plan on tackling the shader system soon.



It's somewhat (well, almost completely) unoptimized. I'm issuing 12,000 draw calls per frame and haven't begun to cull any objects yet.


There's plenty of work to do, and I still need to get it going on the Xbox 360, and I hope to keep updating as progress is made.

Friday, July 23, 2010

Know Your Compiler: Visual C++'s /showIncludes

If you're trying to track down exactly what a particular compilation unit is including (ie: to publicly shame those who are including Windows.h in a public header, or whatever your reasons may be), Visual C++ provides a handy means of doing so with the /showIncludes command line option.

Patch submission

Working in a central tech development role I have patches sent to me on a regular basis, patches for which I am grateful to receive because being able to adopt the fixes my users make only improves my products.

That said, here is my wishlist for patch submission nirvana:
  1. Describe the problem the patch is meant to solve. Maybe a patch isn't entirely necessary? No matter what, dropping a handful of files on my lap with no explanation as to what they do is going to waste both my time and yours as I will need to figure out what's going on, and you're going to be helping!
  2. Please send diffs, not whole files.
  3. Use unified diffs (diff -u or, with Perforce, p4 diff -du). Non-unified diffs are very hard to follow context-wise. The old-style Perforce visual diff that stylizes the old text with bold, red, and struck out and the new text with bold blue is even worse. Providing a patch set or a Git branch would be the Bestest(tm).
  4. If it isn't possible to provide a Git branch then at least indicate from which version of the library the patch was written against.
  5. If I reject the patch, it's not because I don't like you, it may be that the patch conflicts with design goals of the library or may be tied too closely to the submitter's own parent project. This code serves many people and I need to ensure that no one is hindered by the application of this patch.
Thanks, in advance :)

Friday, July 16, 2010

Of false dilemma's and C++

Periodically some e-famous programmer will rant about the evils of C++. This time it was Zed Shaw, though in the past Linus Torvalds has had a tilt at this windmill. These posts are always an entertaining read but what I find interesting in these discussions are the follow up discussions on sites like Reddit.

C++ is a complex language, no doubt about it, but it is entirely possible to manage the complexity. This has been a stated design goal of the language, any new feature introduced should be at no cost to the user should they not want to use it.

Several of these posters offer up exceptions as a case where C++ needlessly burdens the user, compared to its progenitor C. How do you manage memory in an environment that uses exceptions? I'm not going to lie, handling exceptions properly is a hard problem, so much so that it seems that half of Herb Sutter's Guru of the Week articles are on the subject of writing exception safe code. I'm going to go out on a limb here and say that it really doesn't matter how your application deals with exceptions because when one is thrown your system is in all likelihood hosed.

Imagine that you are in the land of C, free from classes and templates, how would you handle a scenario which could be a candidate for a thrown exception? Say malloc returns a null pointer (if that is a valid option, and your process isn't just killed by the operating system), what do you do? Realistically your program would bomb out.

So, now that C++ allows you to handle such a scenario, do you need to do anything about it?

No.

Let it fail, maybe print out a stack trace if you're feeling generous. Sleep well in the comfort that you will likely not have to roll your own data types and that your program is no less robust than if it were written in C.

Thursday, July 15, 2010

Stack slot collision

I had a frustrating bug reported to me last week -- frustrating in the sense that it was hard to spot the actual problem in the first place and frustrating to fix. The code that we were compiling was quite similar to this, although it wasn't C++:
    Foo::Foo(void *, int, int, float, float, float, FooEnum, const std::string &, bool, bool);

Bar(vector &fooVector) {
for (int i = 0; i < limit; ++i) {
Foo *F = new Foo(NULL, GetNextId(), idx, startTime, timeToLive, velocity, Foo::SOME_ENUM_TYPE, "foo", false, false);
fooVector.push_back(F);
}
}
I was noticing that when fooVector was consumed all the entries in it were zero which was causing a crash as the consuming method attempted to dereference Foo's members. It apparantly contained the number of items, but they were all zero. Since the world wasn't ending and my allocator wasn't returning non-zero values I figured it had to do with our compiler's code generation.

Digging into the assembly generated around the push_back() method, I found this code was being generated:

li r27, 0
...
bl allocate
stw r3, 100(r1)
std r27, 96(r1)
std r27, 88(r1)
...
# The instructions here loaded the registers for the constructor call
...
bl Foo_ctor
lwz r3, 104(r1)
lwz r4, 100(r1)
bl push_back

(This architecture is a variant of PPC64 with its own ABI, for the record, paired with a modified variant of LLVM for code generation.)

Saving the return value of the allocation function to the stack is fairly normal behavior but immediately following this we have a doubleword store that stomps our new pointer! This doesn't affect our call to the constructor because its "this" pointer is always r3 anyways and at that point it is still valid. After returning from the constructor and branching into our push_back method we re-load r3 from the stack which at this time has been clobbered.

Where are these stores coming from? What are they doing? PowerPC ABI's generally pass parameters in registers, unless it runs out of registers in which to pass parameters which it did in this case. Integer types, or those convertible to integers such as our booleans, are converted to register width and then are placed on the stack. Since we were passing two false values to our constructor we stored two zero values to the stack in the parameter passing area.

But why the stomp?

This platform uses a different stack layout than the traditional PPC64 architectures. Specifically, the link register is saved 8 bytes below the backchain pointer rather than 16 bytes above:

This in effect requires that our parameter/linkage area needs to be 8-bytes bigger in order to accomodate the LR's save. We didn't see it before because our compiler's unit tests didn't consider having local variables in slots so close to the save area.

Increasing the size of the parameter/linkage area within the stack frame to account for this intrusion of the link register save was what I did to fix the bug and in the end it was a one line code change for an issue that took days to track down!