A central topic in the methodology of axiomatic mathematics concerns the issue of axiom choice: how do we choose the axioms as the starting points of mathematical reasoning? How can one justify the choice of an axiom candidate? On what grounds should new axioms be accepted? Finally, is the justification of new axioms structurally similar to the confirmation of hypotheses in the natural sciences? Such questions are particularly relevant in the context of set theory and other foundational fields, especially in light of the incompleteness and independence results since Gödel's work, which show that standard axiomatizations are insufficient to decide important statements. In the present talk, I will address these questions and survey different types of "regressive" strategies for axiom choice first discussed in the works of Russell, Zermelo, and Gödel. Their examples show that, quite similar to the case of hypothesis confirmation in the sciences, new axioms are often evaluated in terms of their deductive consequences.