After some googling of the problem, it turns out there is a difference between how xhtml and html documents are handled. Essentially, you need to tell the parser that the document is xHTML rather than HTML. More details here, here, and here.

The solution is to tell RSpec that the document is XML based. The options given in the links above work for RUnit based tests. Taking a look at the rspec source, it is checking to see if the response is XML based by noting the Content-Type header record. So, here's the solution I used:

Now, when I run my specs, it correctly parses the response as XHTML and passes validation without any warnings.