Yeah it's a very cool instance, and it's HUGE.
I've only gotten into 2 Karazahn groups so far though.
At first it was because I'm not an officer. Our guild was capable of filling a 40 man raid for Naxxramas, but some of the better players left, and the officers only felt the need to fill one 10 man Karazahn group, which was pretty much for officers and long time members.
Now I pretty much refuse any raid invite. Partly it's because of the above. The other part is that while I do enjoy the instances, I like to raid for gear. Then I use the gear to PvP. Well, PvE gear is no longer the best gear for PvP. My desired "set" of gear only has 1 or 2 items obtained from PvE. The rest are non-set PvP epics, or Arena pieces.
I'm staying in the guild so I can eventually go into Karazahn and the other raids and experience them, but I'll let the officers do the dirty work and learn the encounters. I'm arguably the best rogue in the guild, and I get put on the sidelines because I've only been here for about 6 months. Meanwhile the guild we used to compete with on progression is done with Karazahn, Gruul's Lair, has started on Serpentshire Cavern, and has downed both of the new world bosses. My guild can't even complete Karazahn.
