Abstract: Aiming at the low accuracy of facial emotion recognition in public space, a method of facial emotion recognition, which was based on different receptive fields and two-stream convolution neural network, was proposed. Firstly, a facial expression dataset based on public spatial video was established. Then, a two-stream network was designed, for which a single face image with a size of 224×224 was input into convolutional neural network (CNN) to analyze the static characteristics of the image, and a 336×336 video sequence was input to CNN network, the extracted features were then sent to the long and short term memory network (LSTM) to analyze the local and global motion peculiarity. Finally, the softmax classifier was used to fuse the descriptors of the two channel to get the classification results. The results show that this method can effectively identify four typical facial emotions in public space by using the information features of different receptive fields, and the recognition accuracy reaches 88.89%.