Some women say that we shouldn't worry about men's opinion of our bodies, but I do