My take: Not caring about what others think is equivalent to freeing oneself from the shackles of the human eye. Unfortunately however, being part of a society requires one to take part in many of society’s norms. This means you have to care.
The key here?
Take part in those norms enough so that you’re accepted by some. This will provide you the energy and courage to speak and live your life in a way where you have the the courage to be disliked.