'Pedophilia' used to have a clear, straightforward definition. It referred to a strong sexual attraction to prepubescent children. Those who felt it were considered alien and sick, and those who acted on it were considered despicable criminals.
But of course many people below the age of consent are pubescent or postpubescent. Adult sex with underage teens has garnered increased attention and concern in recent years. Victims of indisputable rape blend in to girls who were coerced, girls who grudgingly consented while feeling confused, and girls who consented but quickly regretted it. All such acts are considered the man's responsibility (which I do not dispute) and a grave crime. The age of consent has been raised in some places to make more such encounters illegal. Also fueling this heightened concern is a more conservative morality -- people shouldn't have sex outside of marriage, or at least outside of long-term monogamous relationships. (My own views on such encounters are nuanced, but I do not favor drastically lower ages of consent.)
When men who have had sex with underage teens are apprehended, they are now labeled as pedophiles by police and press alike. The alternative "statutory rapist" is awkward, and I think has also fallen out of favor because "statutory" is seen as implying that it's not really rape. There is no other obvious word at hand to cover these cases, in part because there is no clear line dividing a prepubescent child from one who is well into puberty. The only clear line is the age of consent in the jurisdiction in question. And whatever its other limitations, "pedophile" suits the aims of law enforcement in one important respect: it suggests they just solved a very serious crime.
But there are unfortunate consequences to thinking of sexual interest in underage teens as the same thing as interest in prepubescents. One is that it is a negative judgment of a natural male sexual interest. Most people recognize at some level that men are interested in girls when they are well into puberty. Girls know that men start looking at them twice long before they turn 16. And for any hetero man reading this who thinks he is above such an attraction, I'll offer you a thought experiment. Suppose an average-looking 13-year-old girl with breasts and hips is diagnosed with a fatal disease. For some reason, the only treatment is also a sure cure -- you and you alone must have sexual intercourse with her. She's desperate to live and is therefore eager for your attentions, approaching it with a positive attitude. Would you have to struggle to get or maintain an erection? Would the act be a distasteful experience? I sincerely doubt it -- I think most men would find it a very exciting prospect. You may have put an attraction to such girls firmly out of your mind as you focus on the women who are appropriate partners for you, but it is still there.
The truth is that people know the attraction is natural, even if they don't like it. A wife who sees her husband stealing way more glances at a 13-year-old girl in a bikini than at boys or younger girls does not think he's truly sick. Society's ferocious wrath is directed at the men who actually try to seduce these girls. So when it comes to crimes involving this age group, there is no distinction between a 'pedophile' and a child molester. Since the same term is used for attraction to prepubescents, this supports the idea that there is no distinction involved there either. But those men do have an unusual attraction that most men don't have. The idea that many don't ever act on it is easily lost when they are lumped with the men who chase young teens -- while the majority of men who lust after them but leave them alone are excluded. At an emotional level, the existence of celibate pedophiles is undercut by extending the word's usage in this way.
Finally, the term "pedophile" is used to describe other situations such as a 59-year-old man involved with a 21-year-old woman. In that case it simply means, "a person sexually involved with someone I think is too young for them". I suspect most aren't suggesting it ought to be a criminal offense -- but I'm not sure.
Some pedophiles have tried to create other words to avoid the stigma of the term. "Girl lover", "boy lover", and "minor-attracted person" (MAP) come to mind. MAP doesn't help with regard to the age confusion, since of course many minors are sexually mature. I personally associate the others with the pro-contact position -- that adult-child sexual contact would be fundamentally just fine if it weren't for misguided laws and attitudes.
The best solution I can think of is "pedophile" with "celibate" welded to its front. "Celibate pedophile" doesn't require any long explanations or qualifications, and it's not avoiding that bad word "pedophile" in a way that will be seen as euphemism or Orwellian Newspeak.