Androsexual or Gay also fit that description. Androsexual is attraction to non femme aligned people, so men / male aligned people and Gay is just non-woman + non-woman. I don't think neither are exactly what you're looking for if your attraction is only *binary* men but they're both pretty close, hope it helps