search

civichackingagency / scangov

Government digital experience monitor
https://scangov.org
8 stars 2 forks source link

Update data: https://nv.gov #148

Open jdiarte opened 1 week ago

jdiarte commented 1 week ago

Domain: https://nv.gov

Hi, looks like your scan did not pick up our Sitemap or Robots. If your crawler is following the robots.txt, I just need to know what to allow so you can crawl the rest of the site successfully. Not sure if you are because you were able to find the Metadata. Regardless, let me know what needs to be done so that your crawler and view both the Sitemap and robots.

Sitemaps: https://nv.gov/sitemap.xml https://nv.gov/sitemap.html

Robots https://nv.gov/robots.txt

Narlotl commented 1 week ago

It's picking up both of them now. Your sitemap is still responding with a content type of text/html, but otherwise everything is working.

https://scangov.org/profile/?domain=nv.gov#sitemap https://scangov.org/profile/?domain=nv.gov#robots