A finite element model (FEM) is used to investigate the effect of roughness on the frictional energy dissipation for an elastic contact subjected to simultaneous normal and tangential oscillations. Frictional energy losses are correlated against the maximum tangential load as a power-law where the exponents show the degree of nonlinearity. Individual asperity is shown to undergo similar stick–slip cycles during a loading period. Taller asperities are found to contribute significantly to the total energy dissipation and dominate the trends in the total energy dissipation. The authors' observations for spherical contacts are extended to the rough surface contact, which shows that power-law exponent depends on stick durations individual asperity contacts experience. A theoretical model for energy dissipation is then validated with the FEM, for both spherical and rough surface contacts. The model is used to study the influence of roughness parameters (asperity density, height distribution, and fractal dimension) on magnitude of energy dissipation and power-law exponents. Roughness parameters do not influence the power-law exponents. For a phase difference of π/2 between normal and tangential oscillations, the frictional energy dissipation shows quadratic dependence on the tangential fluctuation amplitude, irrespective of the roughness parameters. The magnitude of energy dissipation is governed by the real area of contact and, hence, depends on the surface roughness parameters. Larger real area of contact results in more energy under similar loading conditions.