As a relatively newly converted Mac user, I have to say it makes me uncomfortable to surf the web without virus protection in the traditional form with a PC.
I think what you have said makes sense but I wonder about the hackers out there. They are smart, relentless, and determined to do harm. And, I have to believe that it would be considered a major score for a hacker to 'break' Mac OS. If someone can hack the iPhone, what about Mac OS?
Of course, as anokajim points out, browse the net intelligently in the meantime. And, ClanX AV is quite easy to use.
Antivirus applications - Do you need them?