In order to study the mechanical behavior and acoustic emission characteristics of rock mass with holes and cracks, the uniaxial compression numerical test of red sandstone specimens with different crack dip angles was carried out by PFC software. The fracture characteristics, the whole process of stress and strain and the cumulative ringing count of acoustic emission were analyzed, and the damage evolution characteristics of rock samples were preliminarily discussed. The results show that the peak stress and elastic modulus of single fracture rock samples increase with the increase of fracture dip angle. The peak stress and elastic modulus of the three defect types of rock samples are the smallest when the fracture dip angle is 0°, and the largest when the dip angle is 90°. The rock samples with single fracture are mostly wing cracks developed at the tip of prefabricated cracks. For rock samples with cracks not penetrating holes, except that the crack dip angle is 90°, the instability failure modes of rock samples with other dip angles are all connected along the dip angle extension direction. The larger the fracture dip angle of single fracture rock sample is, the smaller the strain range of obvious emission is. The time when the acoustic emission signal occurs in the rock sample with the fracture not penetrating the hole is earlier than that of the other two types of rock samples. The three types of rock samples have the highest cumulative acoustic emission ringing count when the fracture dip angle is 90°.