A head covering, in whatever form a person chooses, expresses feelings of awe and fear.
For this reason, the Sages would customarily wrap their heads in coverings at times of legal judgment or the speaking of words of Torah.
This is also the reason that most Jewish communities have the custom that during the silent Amida prayer the men cover their heads with a tallit (prayer shawl).
Doing so is a sign of submission to an authority, standing before that authority with awe and fear.
Conversely, when a man’s head is uncovered, it is a sign of irreverence and moral irresponsibility.
There is an Aramaic term, resh galei (uncovered head), which refers to this point.
This is why the phrase “with a high hand” is translated into Aramaic as “with an uncovered head,” with both positive (Ex.14:8) and negative (Num. 15:30) connotations.
A person removes layers of clothes when he feels free in his own home and covers himself when he feels he is being scrutinized from on high.
Rabbi Adin Steinsaltz