Abstract: A new approach for hidden Markov model (HMM) training based on an improved maximum mutual information (MMI) criterion was presented and HMM parameter adjustment rules were induced. By adopting a more realistic MMI definition, discriminative information contained in the training data could be used to improve the performance of HMM and this method was also used in facial expression recognition. Facial expression feature vector flows were extracted by using the improved optical flow algorithm, and a hybrid classifier based on the improved HMM and BP neural network was designed. Experimental results show that the new method provides satisfactory recognition performance and the method is powerful for HMM parameter estimation.