openrelik / openrelik-server

The core backend server handling API requests and task management
Apache License 2.0
31 stars 3 forks source link

Uploading E01 Files Greater Than 15GB Freezes and Produces 401 Unauthorized Errors #38

Open dadamnmayne opened 3 weeks ago

dadamnmayne commented 3 weeks ago

Uploading E01 Files Greater Than 15GB Freezes and Produces 401 Unauthorized Errors

Description: When attempting to upload E01 files larger than 15GB to OpenRelik, the upload process intermittently freezes. We’ve observed that when this occurs, the application logs display a series of 401 unauthorized errors for the affected upload chunks.

image

Steps to Reproduce:

Upload an E01 file larger than 15GB. Observe that, after a certain point, the upload stops or freezes. Check the application logs, which show repeated 401 unauthorized errors for certain chunks of the upload.

Troubleshooting Attempts:

Logged out and logged back in to refresh session. Attempted to modify chunk size in files.py. Tested on both Chrome and Firefox, with similar results.

Expected Behavior: Files larger than 15GB should upload completely without interruptions or authorization issues across all chunks.

Observed Behavior: The upload freezes partway, and the logs indicate that chunks fail due to a 401 unauthorized error.

Additional Information

Browser Versions Tested: Chrome, Firefox File Types Affected: E01 Logs: image File Size: image

berggren commented 5 days ago

Sorry for the late reply.

Thanks for the report. I know what is going on here now. The default JWT token lifetime is 5minutes, and after it has to be refreshed. This is normally transparent for the user because we are refreshing automatically with a HTTP intercept in the Javascript API client.

The upload is not using the intercept due to the use of Axias directly, so the refresh doesn't happen in that case. I'll take a look at fixing this for uploads, but in the mean time you can up the lifetime in settings.toml:

jwt_cookie_access_expire_minutes = 5 # 5 minutes

Change thiss to something like:

jwt_cookie_access_expire_minutes = 720 # 12h

Don't forget to restart the openrelik-server container afterwards.