When discussing this topic things can get dicey, so for the record:

• Intersex people are not inherently non-binary
• Intersex people may or may not see themselves as a sex separate from male and female. Some see their sex as male or female while intersex, others may see their sex itself as intersex.
• Intersex people can be male or female. This is not an argument against allowing intersex people to identify with a binary sex or gender.
• When I use the term "sex-variant," I am talking about any situation where someone's sex characteristics are outside of the norm. This includes a lot more people than one might expect. It is not synonymous with intersex, as it can be used for people with different sex characteristics from artificial or environmental changes.
• When I say sex variant, I am including anyone who falls outside of the typical expectations for male/female. This can be intersex people, trans people who have received gender affirming care, cis endosex people who have had injuries or medical treatment that have lead them with different sex characteristics (getting a hysterectomy for instance), and anyone who can fall inside the intersex category even if there's debate whether or not they can claim intersex (PCOS, infertility, gynecomastia, hirsutism, etc.)
• Trans people who do not take any steps towards transition are transgender. The only reason why they will recieve a lot less focus is because I am focused on percievable sex characteristics. They are always valid, and I hope I can end up including them more as I learn more.

A lot of the time the idea is that if there is no third sex, then sex cannot be non-binary. This is a simple misunderstanding of what a binary is.

This would be a binary. Two options with no possible overlap. Nothing can exist outside of the two options either. There is nothing of existence outside one or the other. You can compare it to black and white thinking.

Adding a third circle in a different color would make this non binary, but not a spectrum. A spectrum can have the two sides with multiple types of variation in between. When it comes to the gender spectrum, there are even options outside the provided line.

Here is a decent infographic that shows what sex as a spectrum is:

There's no claim that there's a third gamete, only that the two sexes that we are familiar with have plenty of overlap in various different ways. Part of the point in getting sex across as a spectrum is to remove pathologization of those who don't fit into the binary sex options, so the wording in the graphic is not the best to say the least. I mainly use it to stress that there are many ways sex can develop.

“But intersex people are rare!”

Sadly I cannot make a decent enough caption for those who cannot see the infographic, but if you can, you will see that someone does not need to identify as intersex and/or have a diagnosed difference in sexual development (DSD) to fall outside of the perfectly expected two sexes. A man with higher estrogen starts sliding into the spectrum. Gynecomastia, something caused by a hormonal disruption, is a result of sex being non-binary. Hirsutism, or women with more body hair than what is considered "normal," is a result of sex being non-binary. Both of those things described as clinical issues can occur naturally without any underlying factors or harmful health issues, yet often are seen as abnormal and stigmatized.

The fact that trans people are even able to go on HRT and see changes in their sex characteristics shows that our genes do not take one or the other and stick to it no matter what. Our bodies are programmed to react and adapt to another sex-dominant hormone. How is that not cool? How is that seen as controversial and not an amazing part of human genetics?

But about intersex rarity

Intersex people are estimated to be 1.7% of the population, that's the same as people who have red hair or green eyes. We do not claim those people do not exist, that eyes cannot be green or hair cannot be red, simply because the population that has them is small percentage wise.

Where's that Tumblr post-

Yes, upon looking up the prevalence of intersex people, there will be various different articles pushing back against that statistic for intersex prevalence, claiming that it is much rarer. The article that's most commonly used to refute this percentage is How common is intersex? a response to Anne Fausto-Sterling by Leonard Sax. It argues that the statistic goes beyond what's typically considered intersex by clinicians. Remember that I am using intersex as the way it's defined by many intersex activists, such as InterACT, who made an entire glossary on different diagnoses that can fall under the intersex umbrella. If I was going by clinicians, then using intersex at all would be outdated by decades now, as it's no longer used as a medical term and usually DSD (Disorders/Differences in Sex Development) is used instead. Intersex as a label goes beyond what is considered a DSD or not, and the 1.7% statistic is so old with little updates that it would likely be larger when accounting for the increased scope. Doctors and clinicians do not use intersex, they haven't for decades, it is not up to them to define it.

Intersex people are not one in a million, they're at least 1 in 100. There is no reason to say that they are "too rare" to count when we think of human diversity.

Extra Sources

Biology Lessons on Sex

Sexing the Body - Anne Fausto-Sterling

Worldcat - Internet Archive

The Spectrum of Sex - Hida Viloria & Maria Nieto, PhD

Worldcat - Viloria's Website

Intersex Abuses & Rights

Contesting Intersex: The Dubious Diagnosis - Georgiann Davis

Worldcat - Internet Archive

Intersex: Stories and Statistics from Australia - Tiffany Jones+

Worldcat - Macquarie University Book Page (Book available for free there)

Narrative Symposium: Intersex - Narrative Inquiry in Bioethics Volume 5, Number 2, Summer 2015

Jounral Webpage