Set tag color to white (or remove color)?

Would love to have a way to remove a color of a tag! The best I can do is turn it grey, but it doesn't match the ones that don't have any color.