Basically, if he decides he likes you (which seemingly has no pattern of criteria) you're safe. If he doesn't like you, you'll be targeted and bullied until you leave.
To be totally frank, the issue you have just addressed here is endemic within the management or middle management of even the best public colleges/high/middle schools and/or universities within China.

Yep, that's true. But I feel in a minor city or a county, where it's difficult to attract FTs, you'll be tolerated, even if you're disliked, until your contract has expired. You see, the last thing they want you to do is to piss off and leave them one FT short with little chance of filling the vacancy fast. Losing face is a big thing for Chinese people. So, I think it's better to be a big fish in a small pond than a small fish in a big pond, even though you're making less money in the sticks. At least you've got much more job security there.
By contrast, just compare the plight of Chinese teaching badly in the West. Targeting and bullying them for that is unthinkable. So long as the teaching isn't really bad, they'll be left in peace to fulfill their teaching contracts before being let go. No animosity whatever!