Is using the phrase”OK boomer” in the workplace inappropriate and a sign of ageism?

It depends on the situation and the frequency. Yes, “OK boomer” is an insult. It’s pretty mild, but the intention is to ridicule. The phrase is mainly used to respond to an older guy talking about “back in the old days” kind of stuff. Whether it’s offensive or not depends on your relationship with the person who uses the expression.
Another thing is how often people in the workplace call you that. If it’s a one-time thing meant as a joke, then it’s okay. However, it’s not okay if colleagues constantly comment about your age. It can even turn into harassment over time.