Face skin is not body skin
The skin on your face is thinner than the skin on your body, and it's exposed all day to sun, dust, sweat and air conditioning. Body soaps and many shower gels are made to remove a lot of oil from large areas of thicker skin. On the face, that can strip too much and leave it dry, tight or irritated.