dirkpetersen / froster

Froster is a user-friendly archiving tool for teams that move data between Posix file systems and S3 like object storage systems such as AWS Glacier
Apache License 2.0
12 stars 1 forks source link

endpoint belongs in ~/.aws/config #89

Closed dirkpetersen closed 1 day ago

dirkpetersen commented 1 week ago

My ~/.aws/config has this entry:

[profile idrive]
region = us-or
s3 =
        endpoint_url = https://v2u8.or.idrivee2-42.com
        signature_version = s3v4

However when I select this profile for my IDrive provider it will only detect a previously configured region but not the endpoint_url

   default-dpcri
   aws-dpcri
   wasabi
 > idrive
   moin
   dipeit
   + Create new profile

*** SET REGION ***

[?] Choose your region:
   -- no region --
   + Create new region
 > us-or
   us-la
   us-va
   us-da
   us-ph
   us-ch
   us-sj
   us-mi
   eu-ie
   gb-ldn
   de-fra2

*** SET ENDPOINT ***

[?] Enter the IDrive endpoint:

instead I see that the endpoint is written to config.ini where it does not belong

[S3]
provider = IDrive
endpoint = https://v2u13.or.idrivee2-42.com

Instead , the selected region, endpoint and signature version should be written back to ~/.aws/config . All providers that require an endpoint url should also have signature_version = s3v4 set . endpoint and signature are indented by tab character

also I see that after config the AWS profile contains "output = json" . Do we know how this impacts the user? It may not be a change they want ?

dirkpetersen commented 1 week ago

To ensure that this works deterministically, it would be better to replace

            self.envrn['RCLONE_S3_ENDPOINT'] = cfg.endpoint
            self.envrn['RCLONE_S3_REGION'] = cfg.get_region(
                cfg.profile)

with

            self.envrn['RCLONE_S3_PROFILE'] = cfg.profile
dirkpetersen commented 2 days ago

@victormachadoperez, please hold off on this for now because it seems there was a little change in how AWS cli handles end points, they have now 'services' concept:

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-endpoints.html

I will test this some more and check how this is consistent with rclone, perhaps we leave the endpoint in config.ini if there is no consistency, let me check

victormachadoperez commented 1 day ago

Solved in froster v0.14.19

endpoints are now stored and read from the ~/.aws/config file