WSNG-319: ability to accept a robots.txt file io as an argument so check does not have to download hte file from the domain #11

Closed pyellambalase closed 5 years ago