And I find it easier to compare by switching fast in full screen than side by side comparisons.
This means I can fast switch between them without developing the raw files and reading from disk. If I have a portrait with 5 different poses and I want only 1, I open them in raw mode (not thumbnail) and add them to image buffer 1-5 (ctrl+1 to ctrl+5). Then I go through it one more time, I might have accepted some duplicates which I can then analyse further. Then I go into categorization dialog and delete all images of category 0 (if I want that, otherwise, save the "Showlist" - which includes category tags - and then select all images from category 9). Takes me around 5-15 min to sift through 500 images this way. Left hand over key 9 ('keep' category) and 0 (default category) and right hand over Page Up (next image). then I go through my new set quickly, marking those that I consider usable (on keys 1-0). Then I turn on "categorization mode" (press 'k').
In Image View Plus More I go to preferences (press 'p'), switch on "read the thumbnail". Most people's workflow is to get their pics off of their card and onto their hard drives ASAP, and so the programs are built with that in mind. In addition to Lightroom, I've also used iPhoto and Picasa for photo management they all have the same general issue. The trick is that you usually have to "import" the program into their libraries before you can zip through them I don't know any that will do buffering on the card itself. Several photo-management programs do this behind the scenes, including Lightroom. The way to get around it is pretty much as you described: load up an "instant preview" version of the file (typically by converting it to a much smaller JPEG version) and buffer them in memory. As such, most programs will have this issue. The reason for this is pretty fundamental: you're loading a lot of data from a large RAW file over a relatively slow USB/memory card. However, it does not get around the delay loading images off of the camera's memory card. Lightroom, which the others have mentioned is a great way to organize & sift through reams of images.