Macrophages have a broad spectrum of immune-and non-immune-related tissue-supporting activities. The roots of this heterogeneity and versatility are only beginning to be understood. Here, we propose a conceptual framework for considering macrophage heterogeneity that organizes the factors shaping their identity within four cardinal points: (1) ontogeny and the view that adult macrophage populations comprise a defined mixture of cells that arise from either embryonic precursors or adult monocytes; (2) local factors unique to the niche of residence, evolving during development and aging; (3) inflammation status; and (4) the cumulative effect of time spent in a specific tissue that contributes to the resilient adaptation of macrophages to their dynamic environment. We will especially discuss how to transpose these concepts to cancer to better understand macrophage heterogeneity.