The Colorado baker who won a partial U.S. Supreme Court victory after refusing to make a gay couple’s wedding cake because of his Christian faith lost an appeal Thursday in his latest legal fight, involving his rejection of a request for a birthday cake celebrating a gender transition.

The Colorado Court of Appeals ruled that that the cake Autumn Scardina requested from Jack Phillips and Masterpiece Cakeshop, which was to be pink with blue frosting, is not a form of speech.

It also found that the state law that makes it illegal to refuse to provide services to people based on protected characteristics like race, religion or sexual orientation does not violate business owners’ right to practice or express their religion.

Relying on the findings of a Denver judge in a 2021 trial in the dispute, the appeals court said Phillips’ shop initially agreed to make the cake but then refused after Scardina explained that she was going to use it to celebrate her transition from male to female.

John McHugh, one of the lawyers who represent Scardina, said the court looked carefully at all the arguments and evidence from the trial.

“They just object to the idea of Ms. Scardina wanting a birthday cake that reflects her status as a transgender woman because they object to the existence of transgender people,” he said of Phillips and his shop.

In 2018, the U.S. Supreme Court ruled that the Colorado Civil Rights Commission had acted with anti-religious bias in enforcing the anti-discrimination law against Phillips after he refused to bake a cake celebrating the wedding of Charlie Craig and Dave Mullins in 2012. The justices called the commission unfairly dismissive of Phillips’ religious beliefs.

The high court did not rule then on the larger issue of whether a business can invoke religious objections to refuse service to LGBTQ people, but it has another chance to do so.