Allowing S3 virtual paths and S3 aliases as data source - Githubissues

samtools / htslib

C library for high-throughput sequencing data formats

Other

803 stars 446 forks source link

Allowing S3 virtual paths and S3 aliases as data source #1756

Closed mapk-amazon closed 6 months ago

mapk-amazon commented 7 months ago

Related issue: https://github.com/samtools/samtools/issues/1984

The two main changes are:

In s3_open_v4 an additional hopen is called if a redirect (status code between 300 and 399) is detected.
In redirect_endpoint_callback the url is parsed and a new one constructed respecting the virtual path style in Amazon S3 links.

Some minor changes:

Saving the address style of S3 web links in s3_auth_data to allow the use in redirect_endpoint_callback
Replacing the hardcoded initial value {0, 0, NULL} by KS_INITIALIZE for various variables.

daviesrob commented 6 months ago

Thanks, I can confirm that this works in my tests.