Washington (CNN) -- The divisive issue of abortion is once again causing problems for House Democrats as they plan to move forward with the health care bill.

Anti-abortion Rep. Bart Stupak, D-Michigan, is asking for a vote on his language restricting taxpayer funding for abortion, and a group of female abortion-rights Democrats came out of an emergency meeting in House Speaker Nancy Pelosi's office Friday evening visibly angry about the request.

Rep. Diana Degette, D-Colorado, told reporters a vote on Stupak's measure was a "nonstarter," saying, "somewhere between 40 and 55" abortion-rights Democrats would bolt from the bill.

"We are holding firm this time," she said. "If Mr. Stupak and a few members along with the Republicans decide to use this to take health care down, that loss of health care will be in their hands."

It wasn't clear whether Democratic leaders were seriously considering allowing a vote on Stupak's abortion language, or in what form.

But Stupak said he is in talks with Pelosi, D-California.

Video: Health care homestretch

"We had a good discussion with the speaker on the floor," said Stupak. "But there was no agreement and there's no agreement until we see it in writing and we get a chance to massage it."

The House is expected to vote on the health care bill on Sunday.

One possibility appears to be allowing a separate vote on Stupak's abortion measure, which would go to the Senate independent of the health care bill. If it's an amendment to the Senate bill, and it passes, that could throw the whole complicated but carefully choreographed process of passing health care into flux. It would mean sending the health care bill back to the Senate instead of to the White House for the president's signature.