Suppose that depends on how you look at the item if you look at it as just a pattern or if you find the item makes you feel uncomfortable due to its “historical references”. If you don't have the same “feelings” as the original people it was intended for then its just a trinket rather than a “statement”. All things have a place somewhere in history whether we hold with the values or not, unfortunately you cannot just wipe out history nor turn back time to undo injustice.
There are plenty of what I personally think of as “horrible” people with their pictures on items etc I collect the patterns on the coins/tokens/medals not the people nor their “life values”.
In my eyes the choice has to be yours and how you feel you have a variety of options either keep it or bin it or bury it in the garden for somebody else to discover years from now or pass it on to somebody who collects it for a reason like a history museum.
Token - KKK - United States – Numista
It is very rare 
