Fix the /robots.txt response

[omad]

• Instead of newlines it was returning the text \n.
• Content type was set to application/json instead of text/plain.

I updated the test to check these as well.

---

📚 Documentation preview 📚: https://datacube-explorer--609.org.readthedocs.build/en/609/