Added AWS credentials as required to enable the extension

scrapy-plugins / scrapy-dotpersistence

A scrapy extension to sync `.scrapy` folder to an S3 bucket

BSD 3-Clause "New" or "Revised" License

17 stars 8 forks source link

Added AWS credentials as required to enable the extension #11

Closed rennerocha closed 5 years ago

rennerocha commented 5 years ago

Extension will not enable if ADDONS_AWS_ACCESS_KEY_ID and ADDONS_AWS_SECRET_ACCESS_KEY are not defined in spider settings.

vshlapakov commented 5 years ago

@rennerocha Looks good to me, thanks, is it ready to be merged?

rennerocha commented 5 years ago

@rennerocha Looks good to me, thanks, is it ready to be merged?

Yes, it is!

joeharrison714 commented 3 years ago

@rennerocha May I ask why this was done? I am trying to use this plugin WITHOUT having to specify access/secret key so I can use the credential chain built-in to aws cli.

vshlapakov commented 3 years ago

I am trying to use this plugin WITHOUT having to specify access/secret key so I can use the credential chain built-in to aws cli.

@joeharrison714 It sounds reasonable. Using environment variables is the most common way to pass the settings, based on my experience, but definitely not the only one. I'd suggest to make the check optional: enabled by default and disabled if DOTSCRAPY_CHECK_ENVIRONMENT setting is set to False, would it work for you?