Open tropnikovvl opened 11 months ago
Hello,
Thanks for your issue. The reason I haven't made it was just: I never thought that would be useful. I can do it or you can do a merge request if you want. I cannot say when I will do it sorry.
Regards,
Oxyno-zeta
Hello @oxyno-zeta! Can you tell me if we can expect this feature in the foreseeable future? I noticed that in the latest release, you implemented multi-part uploading. Can you inform me whether it would be suitable for my particular use case?
I am trying to understand the impact of this new feature on traffic, disk usage, CPU and memory. Unfortunately, I haven't had the opportunity to test it yet. Thank you in advance.
Hello @tropnikovvl ,
Sorry for the delay, I have only few amount of time of projects those days...
Yes this should work. S3Manager allow to upload on memory and stream upload to S3 compatible servers.
Tell me if this isn't working :)
Best regards,
Hello @oxyno-zeta,
I tested your new feature and everything looks great.
Unfortunately, my case is still not running, since the data is first loaded inside the s3-proxy and only then into the s3 bucket. Instead of loading them directly into the s3 bucket, immediately bypassing the s3-proxy.
In my case, this slows down data loading in general, I have to pay for additional traffic and temporary storage for the s3-proxy.
In any case, I am very grateful for your product and hope that one day you will have time to implement signed upload links!
Hello @tropnikovvl ,
I'm sorry to ear that... I reflected a bit on this. I'm facing a issue:
That will imply that people will have to try an upload with a signature, catch the redirect and retry with another signature. I'm not sure that will work.
What do you think ? Am I clear ?
PS: Thanks for your return on this project !
@oxyno-zeta ,
I see it this way:
I may be wrong somewhere in the mechanics of the process, but judging by the AWS documentation, this should generally work
Hello @tropnikovvl ,
I'm still reflecting on this subject.
The workflow you proposed is supposing that you have a clever interface and a get and upload file API that s3-proxy don't have I'm afraid... For me, this project isn't the right place for such a workflow. It hasn't been designed for this.
I'm reflecting on another possibility:
With this, API call will also be possible.
What's "problematic" for me is that:
As you can see, I'm still searching :). If you have any comments on those lines, do not hesitate!
Hello @oxyno-zeta!
Unfortunately, I’m not sure that I can help you think about what to do correctly from the point of view of the project.
Your project as a whole works as required for me, but there are 2 nuances that are important to me.
I understand that this is open source and, in general, I have no right to demand or ask you to do anything. I just wanted to share my experience using the product and my opinion on how it could be improved.
Thank you!
Hello @tropnikovvl ,
You can ask using issues on my project ;) . I'm very happy to see people having ideas on those projects. I'm still searching but I'm not very enthusiastic with the solution proposed before.
By the way, the ability to check that a file with the same name and path exists before loading
is already possible. Check here: https://oxyno-zeta.github.io/s3-proxy/configuration/structure/#putactionconfigconfiguration with the field: allowOverride
Thank you !
Hello @oxyno-zeta!
Thank you very much for your responsiveness!
Perhaps you will succeed with the POST request. According to Google, this is possible.
In any case, I will be glad to hear any news!
Is your feature request related to a problem? Please describe. At the moment there is a limitation of signed URLs only for GET requests, can you tell me if there are plans to do this for PUT requests too? The fact is that I upload files larger than 100GB, sometimes 300GB each. And this is a very large load on the file system.
Describe the solution you'd like Uploading of files without overhead on the application and file system. Here's and here an approach to how you can achieve this.