Large file uploads to API failing with FileNotFoundError

[NiViggiano]

Querying the API (e.g. /api/validate/) with a file larger than (I believe) 2.5 MB results in an error similar to [Errno 2] No such file or directory: '/tmp/tmpz4g1gbzc.upload.json' in the [Function]FileUpload.objects.create() model calls.

I believe this has to do with Django storing sufficiently large files in a temp location, unlike small files which are stored in memory. It appears that these files are opened twice in most of the API routes, but have already been deleted by the second open. I have fixed this on my local instance by, in the helpers like license_validate_helper in /app/core.py, returning the permanently saved filename as part of the response. Then in the API routes like validate in /api/views.py, the query object is created using file=response['filename'] rather than trying to get the file again from request.FILES['file'].

If you agree with the issue and solution and would like me to send a PR then I will, I was just unsure about your stylistic best practices and if I was the only one experiencing this.

[goneall]

@NiViggiano - Thanks for reporting the issue. This is the first I've heard of the issue, but I would imagine others would hit this as well.

@BassCoder2808 @rtgdk - any feedback on the approach?

[BassCoder2808]

Hi @NiViggiano, Based on your explanation, it appears you have taken the following steps:

• In the license_validate_helper function located in /app/core.py, you modified the function to return the permanently saved filename as part of the response.
• In the API route handler (e.g., validate in /api/views.py), instead of trying to access the file again from request.FILES['file'], you're using the filename returned by the license_validate_helper function.

This solution aims to address the issue of files being deleted by the time they're accessed for the second time in certain API routes. This seems like a valid workaround @goneall , ensuring that the file is accessible throughout the necessary parts of your application's flow.

But @NiViggiano can you tell us the exact steps through which we can recreate this issue

[NiViggiano]

@BassCoder2808 Just use any sufficiently large file as input to any API route, e.g. curl -X POST -F "file=@large-spdx.json" -F "format=JSON" -H "Accept: application/json" http://localhost:8000/api/validate/. I'd suggest just putting several megabytes of lorem ipsum in the creator comments field of a sample SBOM you have.