Adding support for other Compatible S3 services

Problem

Hello, maintainers (@miararoy @igiloh-pinecone ). My name is Tomer Shalev.

It is highly desirable for organizations to use:

A more cost effective S3 provider such as Cloudflare R2
Or a self hosted S3 (such as minIO)

While I was playing with this library, i wanted to use my Cloudflare R2 buckets, but I couldn't because the current code does not support general http endpoints.

I made a small modification and now I can do the following:


fs = get_cloud_fs(
  endpoint=f'https://{ACCOUNT_ID}.r2.cloudflarestorage.com/{BUCKET}', 
  key=key, secret=secret, anon=False,
  config_kwargs={'s3': {'addressing_style': 'path'}}
)

url = f's3://my-pincecone-folder/metadata.json'
with fs.open(url, mode='rb') as f:
    c = f.read()

Solution

Easy modification, simply modify get_cloud_fs to take into consideration http endpoints. This will also help users to use a self hosted minIO server.

Type of Change

[ ] Bug fix (non-breaking change which fixes an issue)
[ x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ x] This change requires a documentation update
[ ] Infrastructure change (CI configs, etc)
[ ] Non-code change (docs, etc)
[ ] None of the above: (explain here)

Test Plan

This change is not meant to break previous functionality (but it should be tested).

Therefore:

Previous tests should not be affected (but they may fail)
I should consult with you about how and if you test integration between this library to cloud storage.

pinecone-io / pinecone-datasets