(CNN) -- Famed cyclist Lance Armstrong faces the prospect of losing seven Tour de France titles and his fabled championship legacy after he ended his fight against charges of illegal doping.

Armstrong, who has consistently denied allegations of illegal doping, made his announcement to stop battling the U.S. Anti-Doping Agency's "one-sided and unfair process" move against him after losing a legal bid Monday to stop the probe.

The cyclist's decision prompted the anti-doping agency to say it will slap a lifetime ban on Armstrong and strip him of his wins since 1998 after deciding not to contest the charges.

But there's a question whether the USADA has authority in the case and whether international agencies might have to weigh in before Armstrong would face the prospect of losing his titles.

The International Cycling Union has opposed the anti-doping actions by claiming it has jurisdiction. That position has been recently backed by USA Cycling, the official cycling organization recognized by the U.S. Olympic Committee. The cycling union said Friday it won't comment until the anti-doping group issues "a reasoned decision" explaining its stance.

Armstrong has said the cycling union and not the USADA should conduct the arbitration in his case.



"If I thought for one moment that by participating in USADA's process, I could confront these allegations in a fair setting and -- once and for all -- put these charges to rest, I would jump at the chance," Armstrong said in a statement Thursday. "I refuse to participate in a process that is so one-sided and unfair."

Armstrong has been an icon for his feats and celebrity, bringing more status to a sport wildly popular in some nations but lacking big-name recognition, big money and mass appeal in the United States.

He fought back from testicular cancer to win the Tour from 1999 to 2005. He raised millions via his Lance Armstrong Foundation to help cancer victims and survivors, an effort illustrated by trendy yellow "LiveSTRONG" wristbands that helped bring in the money.

The cyclist's one-time high-profile relationship with singer Sheryl Crow kept him in the public eye.

But Armstrong has been dogged by doping allegations in recent years, with compatriot Floyd Landis -- who was stripped of his 2006 Tour de France title after failing a drug test -- making a series of claims last year.

After the Monday court ruling, Armstrong said "enough is enough" in the face of his battle with the USADA and its CEO, Travis T. Tygart, whose investigation he describes as a farce. The USADA is the official anti-doping agency for Olympic, Pan American and Paralympic sports in the United States.

"I have been dealing with claims that I cheated and had an unfair advantage in winning my seven Tours since 1999. Over the past three years, I have been subjected to a two-year federal criminal investigation followed by Travis Tygart's unconstitutional witch hunt. The toll this has taken on my family, and my work for our foundation and on me leads me to where I am today -- finished with this nonsense," Armstrong said.

Armstrong said he plans to help people "affected by cancer" and is "looking forward to an end to this pointless distraction."

"Today I turn the page. I will no longer address this issue, regardless of the circumstances," he said "I have a responsibility to all those who have stepped forward to devote their time and energy to the cancer cause. I will not stop fighting for that mission. Going forward, I am going to devote myself to raising my five beautiful (and energetic) kids, fighting cancer, and attempting to be the fittest 40-year old on the planet."

The USADA has accused Armstrong of using, possessing, trafficking and giving to others performance-enhancing drugs, as well as covering up doping violations. It alleges he took steroids throughout his career, saying it has testimony from former teammates to support the charges. It has refused to reveal who provided the evidence.

Tygart called this "a sad day for all of us who love sport and our athletic heroes."

"This is a heartbreaking example of how the win-at-all-costs culture of sport, if left unchecked, will overtake fair, safe and honest competition, but for clean athletes, it is a reassuring reminder that there is hope for future generations to compete on a level playing field without the use of performance-enhancing drugs," he said in a statement.

But Armstrong has support. Johan Bruyneel, his former team manager, said he's "disappointed" that the case reached a point where Armstrong is "no longer willing to participate in USADA's campaign against him"

"Lance has never withdrawn from a fair fight in his life, so his decision today underlines what an unjust process this has been," he said.

Armstrong came out fighting in May 2011, in the face of fresh allegations made on CBS News' "60 Minutes" show by another American, Tyler Hamilton.

In the CBS interview, Hamilton -- who retired in 2009 after twice testing positive himself and who, earlier this month, was stripped of his 2004 gold medal by the International Olympic Committee because of doping -- said he first saw Armstrong use EPO in 1999.

EPO, or erythropoietin, boosts the number of red blood cells, which carry oxygen to the muscles.

Armstrong has said in court documents that he has never had "a single positive test" in the 500 to 600 drug tests he's taken in his more than two decades in cycling.

"From the beginning, however, this investigation has not been about learning the truth or cleaning up cycling, but about punishing me at all costs. I am a retired cyclist, yet USADA has lodged charges over 17 years old despite its own 8-year limitation," Armstrong said in Thursday's statement.

"The only physical evidence here is the hundreds of controls I have passed with flying colors. I made myself available around the clock and around the world. In-competition. Out of competition. Blood. Urine. Whatever they asked for I provided. What is the point of all this testing if, in the end, USADA will not stand by it?"

In February, Justice Department prosecutors said they closed a criminal investigation after reviewing allegations against Armstrong. They had called witnesses to a federal grand jury in Los Angeles, but apparently determined they lacked evidence to bring a charge that Armstrong used performance-enhancing drugs.

In a June letter to Armstrong, a copy of which was obtained by CNN, the USADA said it collected blood samples from Armstrong in 2009 and 2010 that were "fully consistent with blood manipulation including EPO use and/or blood transfusions."

Armstrong sued the USADA to stop the investigation, arguing it did not have the right to prosecute him.

But a federal judge on Monday dismissed Armstrong's lawsuit after ruling the court did not have jurisdiction.

U.S. District Judge Sam Sparks acknowledged "the appearance of a conflict on the part of both organizations creates doubt the charges against Armstrong would receive fair consideration in either forum." But that doesn't mean federal courts should intervene, the judge said, adding that "these matters should be resolved internally, by the parties most affected."

"If these bodies wish to damage the image of their sport through bitter infighting, they will have to do so without the involvement of the United States courts," Sparks said.

Armstrong said he had hoped that a federal court would stop what he calls the USADA's "charade."

"Although the court was sympathetic to my concerns and recognized the many improprieties and deficiencies in USADA's motives, its conduct, and its process, the court ultimately decided that it could not intervene."

CNN's Scott Thompson, Rich Porter, Gary Morley, Jo Shelly and Joe Sterling contributed to this report.