I think you'll like this, it's very simple but allows for interesting stuff to be layered on top. So, give your villages a small number, 1-6, to indicate their "health." This number, like character hit points is abstract and encompasses a lot of things- sanitation, safety, and economic and social wellbeing.
Each session the players visit a village roll a d6. If it's under this Health Score*, the village has grown a bit. If you are using these Easy Villages, I would say it's grown by one, 1 pip hut. Growth is beneficial to players because more huts means more opportunities for hirelings. And more 1 pip huts means more opportunity for food. You could tie more benefits in with a villages size too. (If you roll over the health score I would keep the village the same size to prevent it from fluctuating too much).
Now make certain things modify this village health. The terrain and climate it's located in could affect the starting score. If there is a monster lair with a day's travel -1. War or plague affecting the region -1. Tyrannical or insane reeve or headman -1. If these negative modifiers bring the health score into negatives the village will shrink by a hut every session the players visit. And this would be in addition to non-abstract losses, players killed by cultists or werewolves, for instance, will also be noted and reduce the village size. However, eliminating threats, bringing livestock, building mills or dovecotes can all give a bonus to the health score, making things better.
I mentioned this as an idea I had to my player on Friday, just as an aside really. And they got quite excited. They understood the simple premise and could see why doing things to help the locals would benefit them. And they immediately set out peppering me with ideas of possible improvements. I had to rein them in a little and make sure they knew I didn't intend for them to be able to turn these villages into town or cities, just help them become thriving villages. They were still excited.
This could be an easy way for the lord of a domain to affect how many troops he/she could draw from an area-- healthier villagers mean more serfs to carry pikes. You might have the villages affect each other, with a village of health 6+ giving all villages within ~ a days travel a +1 to their health. And vice versa, a struggling village could be a drag on all others nearby. I could make a list of simple improvements to show my players and make clear what the improvements would require. So maybe a dovecot would take a wagon load of stone and a wagon load of timber. That way the players could go about getting the supplies rather than just plopping down a set amount of gold. Serfs might be hired to cut roads or drain swampy areas.
Maybe players shouldn't know the actual score of a village unless they visit it or talk with someone that has just come from there recently. But the last known score could be put on the map so players can easily asses the state of the sandbox as they know it.
*I think Health Points is a better name for this than Hit Points but figured by putting the latter in the title you would immediately know what I was getting at.