cisagov / trustymail

Scan domains and return data based on trustworthy email best practices
Creative Commons Zero v1.0 Universal
185 stars 31 forks source link

Fixes issue 132 (Public Suffix List (PSL) Updates Too Often) #133

Closed Matthew-Grayson closed 10 months ago

Matthew-Grayson commented 1 year ago

πŸ—£ Description

Added code to set the PSL file's last modified time to reflect time of last download.

πŸ’­ Motivation and context

PSL should only be downloaded once per day. The current code attempts to enforce that by only downloading a new PSL, when the local copy is older than 24 hours:

psl_age = datetime.now() - datetime.fromtimestamp(
    stat(PublicSuffixListFilename).st_mtime
)
if psl_age > timedelta(hours=24):
    updatePSL(PublicSuffixListFilename)

stat.st_mtime checks the last modified date of the local PSL, however, the last modified date will often be 2-4 days ago even if the PSL was downloaded minutes before.

Resolves #132.

πŸ§ͺ Testing

I ran Trustymail against an array of domains. The first domain was tested and then the code waited for a local PSL to be created before testing the remaining domains. I logged all STDOUT to see if the "PSL Updated" message appeared more than once.

βœ… Pre-approval checklist

βœ… Pre-merge checklist

βœ… Post-merge checklist

jsf9k commented 10 months ago

I also did some testing with these changes and verified that they function as expected.