Same thing happened to us.
It was our first day in Vietnam and went to walked then they from nowhere they showed up saying Ben tham market is closed. They said that the fee is for about 15 hundred per site and no limit on how long you want to stay on 1 site.
After our tour we went for a happy hour and asked us to write in their book. Saying all good things without what will happen next. We were shocked when they asked for 1.5 million VND each, and we end up paying them 3 million VND.
We learned our lesson and should have not trusted anyone who approached us nicely. It was just really frustrating. Just sharing for other travellers like us.