When I grow up, should I be a General Practitioner or an Open Heart Surgeon? Actually, I never had dreams of being a either, but the question is the same - is it better to be a generalist or a specialist? The consensus often leans in the direction of being a specialist. You make more money!
The same holds true in photography. There are photogs that specialize in portraiture, architecture, fashion, editorial, weddings, etc. And then there people like the Hudson Valley photographer extraordinaire, Rick Sammon. Rick states, "My specialty is not to specialize." I like that.
If I had to choose, I'd fall into his camp. Everywhere you look, there is a photograph waiting to be taken. A story to be told, a feeling to be conveyed. And there are so many ways to do it. We have panoramas, image blending, high dynamic range, Focus Stacking, infrared, macro, light painting, etc. It's like being a kid in a candy store. Additionally, I believe that photographing different genres will make you a better photographer.
There is a wonderful saying in the East - the beginner has infinite possibilities, the master has few. I'm not sure which is best, but there is room for all of us.