The long-term mechanism of poverty alleviation should consider the existence of poverty risk in the future, that is to say, the vulnerability of poverty should be concerned. The continuous rise of the house price makes “house slaves” become a common phenomenon, and the generation of housing debt may have some impact on household poverty vulnerability. Based on the data of China Family Panel Studies (CFPS) in 2016, this paper discusses the relationship between the two. The empirical results show that, on average, the poverty vulnerability of households with housing debt is lower than those without housing debt, and the housing debt pressure measured by housing debt to asset ratio can alleviate the poverty vulnerability. However, further research shows that the mitigation effect is not continuous, but there is an optimal value of 137.5%, showing a “U” nonlinear relationship. That is to say, before the ratio of housing debt to assets reaches 137.5%, the housing debt mainly shows the net “wealth effect”, and there is the possibility of “making money from debt”; after that, the “housing slave effect” will dominate and stimulate the occurrence of “poverty due to debt”. In addition, the relationship between housing debt and poverty vulnerability will also vary according to family characteristics. In order to alleviate relative poverty, the efficiency of housing credit can be further improved.