Honestly, isn't it the quality of the people you're involved with, rather than the sheer number of people? I mean, wouldn't you rather have somebody really kind and intelligent interested in you, than a thousand mindless people? They say the masses only like ordinary stuff because that's all they have the capability to appreciate. Whereas the greater intellects and the greater artistic sensibilities can, accordingly, only be appreciated by those with the capacity to appreciate them.
I know I'm flattered if one substantial person likes me. To be popular with a thousand idiots means nothing to me. Think of the powerful intellect and sensibility of a Beethoven or a Shakespeare. Well, to appreciate such a sensibility as theirs you have to have a similar one.
So when everybody's running after the latest top-40 sensation you just let them run. You appreciate the people with more going for them and you will be appreciated by them in turn. That really means something.