search

18F / domain-scan

A lightweight pipeline, locally or in Lambda, for scanning things like HTTPS, third party service use, and web accessibility.
Other
371 stars 137 forks source link

Timstamps formatted differently between lambda and local executions #185

Closed jsf9k closed 6 years ago

jsf9k commented 6 years ago

If I run ./scan dhs.gov --scan=sslyze --lambda --debug --workers=400, I get the following output:

Domain,Base Domain,Scanned Hostname,Scanned Port,STARTTLS SMTP,SSLv2,SSLv3,TLSv1.0,TLSv1.1,TLSv1.2,Any Forward Secrecy,All Forward Secrecy,Weakest DH Group Size,Any RC4,All RC4,Any 3DES,Key Type,Key Length,Signature Algorithm,SHA-1 in Served Chain,SHA-1 in Constructed Chain,Not Before,Not After,Highest Served Issuer,Highest Constructed Issuer,Errors
dhs.gov,dhs.gov,www.dhs.gov,443,False,False,False,True,True,True,True,False,256,False,False,False,RSA,2048,sha256,False,False,2017-05-11T00:00:00,2018-08-10T23:59:59,GeoTrust Global CA,GeoTrust Global CA,

On the other hand, if I run ./scan dhs.gov --scan=sslyze --debug --workers=400, I instead get the output:

Domain,Base Domain,Scanned Hostname,Scanned Port,STARTTLS SMTP,SSLv2,SSLv3,TLSv1.0,TLSv1.1,TLSv1.2,Any Forward Secrecy,All Forward Secrecy,Weakest DH Group Size,Any RC4,All RC4,Any 3DES,Key Type,Key Length,Signature Algorithm,SHA-1 in Served Chain,SHA-1 in Constructed Chain,Not Before,Not After,Highest Served Issuer,Highest Constructed Issuer,Errors
dhs.gov,dhs.gov,www.dhs.gov,443,False,False,False,True,True,True,True,False,256,False,False,False,RSA,2048,sha256,False,False,2017-05-11 00:00:00,2018-08-10 23:59:59,GeoTrust Global CA,GeoTrust Global CA,

Note that in the lambda results the date and time are separated by a T character, while in the local results they are separated by a space character. The results should really be identical in the two cases.

konklone commented 6 years ago

Wow, good catch, @jsf9k. I'm going to guess it's related to a Python version/library difference between the Lambda environment and your local computer caused by relying on some implicit behavior somewhere instead of us explicitly specifying a timestamp format. But I will look into this and find out what's up!

jsf9k commented 6 years ago

@konklone I must have made a mistake when I tested before. The problem really is here after all, and it is fixed by changing this line. The reason is that datetime.__str__() is equivalent to datetime.isoformat(' ').

Pull request coming in a few minutes.