Uploading exported data to s3 without storing at the local disk

awslabs / amazon-neptune-tools

Tools and utilities to enable loading data and building graph applications with Amazon Neptune.

Apache License 2.0

297 stars 151 forks source link

Uploading exported data to s3 without storing at the local disk #309

Open SergeVil opened 1 year ago

SergeVil commented 1 year ago

We are running the Neptune AWS Batch Fargate that has a local disk limitation 20Gb, while the memory is pretty big - 128Gb. Such a low disk volume limits us to export the large database (getting the OS error). We look for the option to upload the exported csv files right away from JVM memory to the S3 bucket. Thank you!

Cole-Greer commented 1 year ago

Hi @SergeVil, thanks for submitting this request. Just so you are aware, we have migrated Neptune-Export to a new repository. I have created a new issue in that repository to track things there.