search

scrapinghub / exporters

Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations
BSD 3-Clause "New" or "Revised" License
40 stars 10 forks source link

Adds some log warnings for AWS S3 keys config checks. #339

Closed tsrdatatech closed 7 years ago

kalessin commented 7 years ago

I am not much convinced of this kind of checks. Considering that the probabilty of typing a wrong character is several orders bigger than giving a aws key bigger than the secret key, or that an space is just one of millions of possibilities of wrong key.

We need a better way to realize that aws keys are wrong, without checking those ultra specific possible mistakes. I.e. I would try to see if the aws error message from server when keys are wrong, provide better information that just can lead to infer that one possible cause is a wrong key.

kalessin commented 7 years ago

The most we can do to avoid some exceptions is by checking this:

http://docs.aws.amazon.com/IAM/latest/APIReference/API_AccessKey.html

access key id must match pattern \w+ and length can be from 16 to 32.

tsrdatatech commented 7 years ago

I agree with what you are saying. There is some better info with errors such as this from boto itself, which is where I originally saw this error. I will look into implementing something more along the lines in which you suggest. I am going to close this PR and redo another geared towards this other approach.