Add S3-Compatible support for External Download server

[0xCiBeR]

Is your feature request related to a problem? Please describe. No, but yes. I mean, it ultimatly solves or really improves server download issues.

Describe the solution you'd like The ability to specify, S3 Compatible storage(CLIENT_ID, SECRET_ID, S3 Endpoint) and that the server will keep resource files in sync with remote bucket providing a reliable multi region url for faster download speeds. This should be vendor non-specific, with this i mean it should be compatible with any vendor offering s3 compatible storage. Some examples are: AWS S3 Digital Ocean Spaces Backblaze

Describe alternatives you've considered The clear alternative is use cron and some s3 tool, like s3cmd. This is okay, if you have some level of understanding of whats going on, but it would be cool and would improve servers download speeds since the resource content would be region available and some providers even offer CDN, and for the average server owner who just starts, its much easier to just provide 3 variables and the server will do the rest.

Additional context I know MTA:SA is an oldish game, but improving compatibility with current tech will only improve adoptability for new comers.

[TheNormalnij]

Rclone?

[0xCiBeR]

Like i said, i'm not looking for a way to do it, since there are some, i'm talking about integrating it natively. Our server code is not up to date in some more recent technologies, and i know this is an old game, but there isn't any reason not to keep improving it :)

[qaisjp]

At a glance my thought process is this:

• Do any other game servers offer something like this?
• How do we select which cloud services to support?
• A web server is a pretty reasonable abstraction which has worked well, and (I think) scales pretty great with minimal additional complexity.

Is S3 on its own even the right option? At a glance Amazon Cloudfront provides CDN functionality, integrates with S3, and should work fine with what we have right now.

I'm assuming you want some sort of syncing between S3 and MTA's server scripts — what exactly are you looking for?

Instead of directly providing integrations with various cloud storage services, I'd prefer to provide better scripting support (not necessarily in Lua).

[0xCiBeR]

First, about the questions:

1. Not sure of any, though fastdl is offered on multiple games and this usually means manual jobs and scripts, thats what we currently have
2. I guess we can aim to support the bigger ones, AWS, GCP, Azure, but just with adding S3 support we can cover a broad range of services that rely on s3 compatible storage like: AWS S3, DigitalOcean, Backblaze, etc.
3. A public s3 bucket is ultimately a webserver serving static assets

On server boot, refresh, etc the server generates the files to upload to the external download server in http-client-files. It would be nice, that with 2 or 3 config options, the server automatically keeps an s3 bucket in sync.

Using cloudfront would be great, but i dont think that should be natively supported, since there is a lot of ways the get the files on a CDN, like Cloudflare. The scope of the issue is that the server natively supports keeping client files in sync on cloud providers for scalability, deliver-ability and geo-positioning.

So the end result would be:

2 or 3 config options to specify bucket name, access key and secret key

On boot, refresh and every event that updates the http-client-files folder, the server would trigger a sync to the bucket.

I think this is pretty trivial to integrate since we only would need to use some already created tool like the aws cli or s3cmd. I may be mistaken on the complexity, but if integrating this is more than trivial, then maybe it doesnt justify the work needed.

[Pirulax]

Isn't all this netcode stuff?