WASHINGTON - President Obama may get liberal Elena Kagan on the Supreme Court, but conservative swing-voter Anthony Kennedy says he's not going anywhere anytime soon.

Justice Kennedy, who turns 74 this month, has told relatives and friends he plans to stay on the high court for at least three more years - through the end of Obama's first term, sources said.

That means Kennedy will be around to provide a fifth vote for the court's conservative bloc through the 2012 presidential election. If Obama loses, Kennedy could retire and expect a Republican President to choose a conservative justice.

Kennedy, appointed by President Ronald Reagan, has been on the court 22 years. He has become a bit of a political nemesis at the White House for his increasing tendency to side with the court's four rock-ribbed conservative justices.

Without naming Kennedy, Obama was unusually critical of his majority opinion in the Citizens United case, handed down last January. That 5-4 decision struck down limits on contributions to political campaigns as an abridgement of free speech.

Obama called the ruling "a major victory for big oil, Wall Street banks, health insurance companies and the other powerful interests that marshal their power ... in Washington to drown out the voices of everyday Americans."

He was so angry that he took the unusual step of blasting the decision in his Jan. 27 State of the Union address, with Kennedy and five other justices looking on.

The White House has made Citizens United a populist rallying cry for Democrats, who hope to cut into projected Republican gains in the November elections by painting the GOP as guardians of the rich and powerful.

With the retirement of fellow Stanford graduate Sandra Day O'Connor in 2006, Kennedy has inherited O'Connor's mantle as the court's swing vote.

His voting pattern suggests he's actually become a far more reliable vote for the conservatives.

A Supreme Court spokeswoman said Justice Kennedy could not be reached for comment

tdefrank@nydailynews.com