This project offers a web interface to evaluate chat bots, with optional mturk integration
Docs : https://cutelab.name/boteval/