The only time I ever had those kind of problems with Safari were when 1) the icon cache was corrupted (haven't had this problem in a long time, though), or 2) when I was having an obscure keychain unlock issue and an app would hang until the keychain unlocked.
I would recommend deleting the icons, as Chris Seibold said above.
I am using OmniWeb now, but that's because they finally got it's Tiger quirks worked out. I was using Safari with Saft and Pith Helmet, which gave me some of the features I missed from OmniWeb.
Is it time to ditch Safari?