The tribological behaviors of polycrystalline diamonds (PCDs) against different materials is fundamental for their applications in abrasion and bearing industry. This paper systematically investigates the friction and wear of two types of PCDs: hot-filament-chemical-vapor-deposited (HFCVD) and high-pressure-high-temperature (HPHT) sintered, against steel GCr15 and ceramic Si3N4 balls. It is found that firstly the coefficient of friction (COF) increased with the roughness increasing; Secondly both the COF and the wear rate (k) of ceramic Si3N4 balls were generally lower than those of steel GCr15 balls, except for the HFCVD sample's COF; Thirdly the wear rate (k) increased with the PCD's grain size, accompanied by a decrease in residual cobalt content. The researchers observed transfer films of SiO2 and its hydrate at the friction interface Si3N4/diamond, which were found to smoothen the friction and wear. Additionally, carbon-based transfer films were identified at both GCr15/PCD and Si3N4/PCD interfaces, mainly catalyzed by the residual cobalt in PCDs, leading to the phase transformation from sp3 to sp2.