A T-matrix method for scattering by particles with small-scale surface roughness is presented. The method combines group theory with a perturbation expansion approach. Group theory is found to reduce CPU-time by 4-6 orders of magnitude. The perturbation expansion extends the range of size parameters by a factor of 5 compared to non-perturbative methods. An application to optically hard particles shows that small-scale surface roughness changes scattering in side-and backscattering directions, and it impacts the single-scattering albedo. This can have important implications for interpreting remote sensing observations, and for the climate impact of mineral aerosols. (C) 2011 Optical Society of America