jaydenseric / graphql-upload

Middleware and an Upload scalar to add support for GraphQL multipart requests (file uploads via queries and mutations) to various Node.js GraphQL servers.
https://npm.im/graphql-upload
MIT License
1.43k stars 131 forks source link

Stream Support #292

Closed j closed 2 years ago

j commented 2 years ago

I know there's a fork with stream support (https://github.com/flash-oss/graphql-upload-minimal), however, I'd much rather see less forking, and more collaboration now days, as I'd rather bring in a dependency with more popularity and usage than a fork into our organization.

With that said, I'd love to just use the temp filesystem, however, we're currently using Google Cloud Run, and the FS is based off the server's provisioned memory (which in itself is limited).

I've seen the "why's" (though it's a lot to understand so I didn't dig too much) into the reason of choosing fs-capacitor, however, due to this implementation, we can't just run our GCR instances with larger memory footprints to support uploads.

It'd be great if it was more pluggable, but I'm not sure how feasible that is. I'd love to just skip the FS and stream them right into Google Cloud Storage.

Just throwing this out there in hopes that it could eventually make it's way back into the creator's library.

jaydenseric commented 2 years ago

This appears to somewhat duplicate https://github.com/jaydenseric/graphql-upload/issues/129 , although feel free to correct me if Google Cloud Functions support would be different to Google Cloud Run support. More active serverless support discussion can be seen in the AWS Lambda issue: https://github.com/jaydenseric/graphql-upload/issues/155 .

Regarding your desire to skip buffering the uploads to disk, see https://github.com/jaydenseric/graphql-upload/issues/289#issuecomment-1013968708 .

I'll keep your sentiment in mind; I tried very hard for many months to avoid using the filesystem. I don't really have the availability to work on really tricky solutions like attempting to inspect the operation and tell if the same upload variable is used in multiple places or not to potentially skip buffering its upload to disk. Off the top of my head I can't even remember if that approach would be feasible due to all the other reasons buffering to disk was necessary.