Based on the industry panel data from 1990 to 2017, this paper uses Translog Stochastic Frontier model to measure the TFP of the service industry from the perspective of internal structural evolution, so as to reveal the phenomenon, degree and tendency of heterogeneity. Furthermore, the paper uses the conductivity variance decomposition model, VAR model and VECM model to investigate the contribution of TFP, and estimate the heterogeneity effect in the short term and the long run. The results show that: The level of TFP, technical progress and technical efficiency are gradually decreasing. The scale efficiency change is always very low, while the allocate efficiency change is quite low in the early stage, but increase gradually later. The heterogeneity of TFP and its decomposition in the production service and consumer service industries are declining, and the heterogeneity in the consumer service is lower than produce service. The heterogeneity in scale efficiency change is higher than the other parts, and the volatility of the heterogeneity in technical efficiency is the smallest. Service TFP heterogeneity is the main reason leading to differences in the development of the industries, and technical efficiency is key element which has significant effect in the long run.