Accurate identification of coal and gangue is essential for clean and efficient use of coal. Existing target detection algorithms are ineffective in detecting small-target and overlapping gangue, and contain complex network structure and large parameter volume, which cannot meet the demand of real-time detection of edge devices. To address the above problems, a lightweight detection and identification approach of coal gangue based on improved YOLOv5s is proposed. The depth-separable convolutions are used to replace ordinary convolutions, and the C3 (Concentrated-Comprehensive Convolution Block) Ghost module is constructed to replace all C3 modules in the YOLOv5s to reduce model computation and parameters. The CA (Coordinate Attention) attention mechanism is introduced to strengthen the attention to the target to be detected, suppress irrelevant background interference, and improve the detection accuracy of the model. The Focal- EIOU (Focal and Efficient Intersection Over Union) loss function was introduced to replace the original CIOU. Extensive experiments substantiated the proposed approach can effectively and quickly detect the small-target and overlapping coal gangue accurately, and the mAP (mean Average Presicion) reaches 97.7%. Compared with the original YOLOv5s, the proposed approach reduces the number of parameters and the amount of computation by 48.5% and 43%, respectively, under the premise of maintaining the same detection accuracy.