Abstract: In order to eliminate the adverse effect on and safety risks in coal cleaning operations produced by presence of tramp materials in raw coal, a smart tramp material recognition-localization-separation integrated system based on deep learning and machine vision is developed. A pixel-level recognition model based on semantic segmentation is likewise developed with its computional costs 8～9 times less than that of the standard convolution network model. The system is provided with a robotic arm which is capable of making recognition and selection of points for gripping of hard and light foreign materials and keeping clear of any interference, in an accurate manner, even under complex working environmental conditions. Result of test conducted with the system at Guobei Coal Preparation Plant shows that the system can work with a recognition rate up to 96.647%, an extraction success rate as high as 94.759% and a sorting rate of 91.640%. It well demonstrates that the use of the system can realize high-efficiency removal of foreign matters and lead to enhancement of the intelligent level in this respect.