stevearc / pypicloud

S3-backed pypi server implementation
MIT License
504 stars 144 forks source link

S3 pre signed URLs inconsistently working #329

Open IdrisMiles opened 1 year ago

IdrisMiles commented 1 year ago

Hello, I'm testing this package out with an S3 backend storage, the server is also running on an EC2 instance with an IAM role that has full access to the S3 bucket. I'm relying on the instances IAM role rather than explicitly providing aws credentials.

However I'm finding when I do a pip install, I sometimes get a 403 error, and then after several minutes it starts to work, but after a few more minutes it stops working again. The cycle continues...

pip install command:

pip install --index-url http://<pypicloud server>/simple --trusted-host <pypicloud server> sparro-services

This is the error I get:

ERROR: Could not install requirement sparro-services from http://<pypicloud server>/api/package/sparro-services/sparro_services-2.0.0-py3-none-any.whl#sha256=bd70c243322381b7165e0c0eb056be20e4c598e0c8fce2d47263s89c4707fs9e because of HTTP error 403 Client Error: Forbidden for url: ...

When I follow the forbidden URL I get this message:

<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>

This is my server.ini:

[app:main]
use = egg:pypicloud

pyramid.reload_templates = False
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = en

pypi.default_read =
    everyone
pypi.default_write =
    authenticated

pypi.storage = s3
#storage.aws_access_key_id = 
#storage.aws_secret_access_key = 
storage.bucket = <pypi bucket>
storage.region_name = eu-west-2
#storage.redirect_urls = false

db.url = sqlite:///%(here)s/db.sqlite

...
# wsgi/logging config below

Any help would be much appreciated, this seems like a really useful setup just struggling to get it working consistently. Let me know if you need any more details.

IdrisMiles commented 1 year ago

I should add I'm running:

IdrisMiles commented 1 year ago

I'm now opting to use the production config ppc-make-config -p prod.ini And explicitly creating an IAM user and using its creds rather than relying on the EC2 instances IAM role. So far it seems to be a lot more stable, I've not faced any SignatureDoesNotMatch issues. Will keep monitoring and testing