ValueError: curlCode: 77, Problem with the SSL CA cert (path? access rights?)

[austinmw]

Hi, I'm attempting to test S3 datasets in a Jupyter notebook on an EC2 instance. I've configured the cli with aws configure and the following command lists files successfully:

url_path = 's3://data-sm/webdataset/dataloading_benchmarks/dogs/shards/'
!aws s3 ls {url_path}

However, the following results in an error:

from awsio.python.lib.io.s3.s3dataset import list_files
urls = list_files(url_path)

---

ValueError Traceback (most recent call last) Input In [216], in 7 # Errors 8 from awsio.python.lib.io.s3.s3dataset import list_files ----> 9 urls = list_files(url_path)

File ~/anaconda3/envs/py38/lib/python3.8/site-packages/awsio/python/lib/io/s3/s3dataset.py:97, in list_files(url) 94 """Returns a list of entries under the same prefix. 95 """ 96 handler = _pywrap_s3_io.S3Init() ---> 97 return handler.list_files(url)

ValueError: curlCode: 77, Problem with the SSL CA cert (path? access rights?) with address : 3.5.84.1 with address : 3.5.84.1

Any idea what's up with this error?

[waleedgondal]

I ran into the exactly same issue. Has there been any progress on this?

[austinmw]

I haven't found a solution.

[ydaiming]

@austinmw @waleedgondal

We're upstreaming the amazon-s3-plugin-for-pytorch into the torchdata package (https://github.com/pytorch/data/pull/318). We're dropping support for this plugin.

As for the ValueError: curlCode: 77, I think this issue may be relevant: https://github.com/curl/curl/issues/3793 Because the S3 plugin uses aws-sdk-cpp as its backend.

[austinmw]

@ydaiming Great idea to put in torchdata, thanks!

[ydaiming]

I would like to add an input to this issue. It's possible to resolve the issue with the following commands to add an additional certificate at the proper location for cURL.

mkdir -p /etc/pki/tls/certs && cp /etc/ssl/certs/ca-certificates.crt /etc/pki/tls/certs/ca-bundle.crt