It has been widely accepted that the microstructure of bearing materials can significantly affect their rolling contact fatigue (RCF) lives. Hence, microlevel topological features of materials will be of significant importance to RCF investigation. In order to estimate the fatigue lives of bearing elements and account for the effects of topological randomness of the bearing materials, in this work, damage mechanics modeling approach is incorporated into a Voronoi finite element method recently developed by the authors. Contrary to most of the life models existing in the literature for estimating the RCF lives, the current model considers microcrack initiation, coalescence, and propagation stages. The proposed model relates the fatigue life to a damage parameter , which is a measure of the gradual material degradation under cyclic loading. In this investigation, 40 semi-infinite domains with different microstructural distributions are subjected to a moving Hertzian pressure. Using the fatigue damage model developed, the initiation and total lives of the 40 domains are obtained. Also, the effects of initial material flaws and inhomogeneous material properties (in the form of normal distribution of the elastic modulus) on the fatigue lives are investigated. It is observed that the fatigue lives calculated and their Weibull slopes are in good agreement with previous experimental and analytical results. It is noted that introducing inhomogeneous material properties and initial flaws within the domains decreases the fatigue lives and increases their scatters.