Evan,
thanks for sharing your experience. You have touched upon what seems to be a serious performance issue.
As far as I know, the main additions to mail functionality in Coldfusion 11 are in the area of security and encryption. Spooling should perform as it did in Coldfusion 9. Have you compared your mail settings in the Coldfusion 9 Administrator, one by one, with those in the Coldfusion 11 Administrator?