Human brain waves are affected ever so slightly by magnetic fields, some people’s more than others. It might very well be that there’s some kind of subtle subconscious effect of the Earth’s magnetic field on our sense of orientation.