For some reason, you grow incredibly skeptical of this old man. Maybe the town isn't really destroyed... maybe you're not in this alternate dimension thing. So far, the wizard just SAID all this stuff and created this illusion!
You confront the wizard, "Lies!" You point accusingly to the wizard, "I don't know what you're up to, but it has got to stop!" You growl a bit, "Now return my town to the way it was!"