Create WebCheckService to check robots.txt (#241)

[joelramilison]

Issue number: resolves #241

---

Checklist

• [x] Code Formatter (run prettier/spotlessApply)
• [ ] Code has unit tests? (If no explain in _otherinformation)
• [x] Builds on localhost
• [x] Builds/Runs in docker compose

Does this introduce a breaking change?

• [ ] Yes
• [x] No

Other information

This is scaffolding only.

I added an additional method as well as comments as to how I think the implementation could go.