Former Chelsea winger Pat Nevin has told fans engaging in “offensive singing and behaviour” to “stay away” from the Premier League club.
Chelsea and the Metropolitan Police are investigating claims Manchester City’s Raheem Sterling was racially abused at Stamford Bridge earlier this month.
Anti-Semitic chants were then heard during the Stamford Bridge club’s Europa League tie with MOL Vidi.
Nevin said he found it “anger-inducing and downright embarrassing”.
Chelsea have suspended four fans while an investigation into the alleged racial abuse of Sterling takes place.
Nevin, a BBC football analyst, said recent wins over Brighton and Premier League rivals Manchester City meant “things should be hunky dory at the club” but they are “not by a long way”.
Writing in his regular column on the Chelsea website, the former Scotland international said: “It is getting a bit wearisome as the last lingering vestiges of an unreconstructed age still do not seem to get the message.
“As such they know what they are doing and seem not to care how badly this reflects on, and offends, the rest of us.”
Chelsea chairman Bruce Buck greeted Blues fans at the turnstiles before their Premier League win at Brighton.
Club officials said the atmosphere among the away support was positive, but fans were reminded they are under heavy scrutiny.
Addressing the fans, Nevin added: “I do not know the answer other than continuing to do and say the right things, all of us, including you the true fans whose name is being tarnished as well.
“I have however always had one strong belief. These people do not seem to really care about the club whatever they say.
“They certainly do not care above their political and social views which they use the club and its worldwide fame to advertise.
“If you cannot keep those emotions, views and outbursts away from following the team, for the sake of the club if you have any vague affection, just stay away from us all.
“Harsh as it may sound, we simply do not want you.”
Manager Maurizio Sarri – whose side face Bournemouth in the Carabao Cup quarter-finals in Wednesday – last week condemned “discrimination in any form”.