Open Wintermute68k opened 2 months ago
Acconut
commented
2 months ago That's odd and I am not able to reproduce it locally.
Wintermute68k
commented
2 months ago Okay, I'm doing the tests. Is this normal?
Happens for any file I pick.
I reloaded the JS client demo and tried with chunk size unset as you requested (what's a good chunk size?). One file uploaded normally, without any erratic progress bar behavior. Next file resulted in the above error again, and uploads don't complete anymore. Is that normal operation, would I have to handle that error in production, and is it frequent?
I emptied out /srv/tusd-data/data in the docker container but that did not fix the problem, so does not appear to be a free space problem.
Uploads to https://tusd.tusdemo.net/files/ seem to work fine.
https://tus.io/demo upload became stuck very early. Continued again after a couple of minutes. Network view in the browser did not show individual chunk requests in this client version, so I can't see what was happening when it became stuck.
Acconut
commented
2 months ago Okay, I'm doing the tests. Is this normal?
I have never seen this error before. So, no, this is not normal. Since this behavior does not appear when uploading to our demo server, it's likely that your server/container is misconfigured and causing errors. The progress bar is updated based on transfer information from the client and server. While the client says, I sent 100% of the data to the server, the server says I got an error and didn't save any data. This causes the progress bar to jump around while the client is retrying.
The logs from tusd should contain the exact errors that it's encountering. Can you put them here? If it's "too many open files" you might have to raise the limit for open file descriptors.
what's a good chunk size?
Best leave the option empty if you can. It's only needed in special situation and you don't have to set every option: https://github.com/tus/tus-js-client/blob/main/docs/api.md#chunksize
Wintermute68k
commented
2 months ago your server/container is misconfigured and causing errors.
Oh, that would be attributing just a 'lil too much credit to me. I use Docker Desktop, it pulled down tusproject/tusd:latest, and created a container where the only thing I set in the creation form were the container name "tusd" and the port "8080". Then I pointed my browser to localhost:8080 to see the greeting text, then I opened the JS client demo and pointed the upload to "http://localhost:8080/files", and I Googled for chunk size values and top result stated that 5MB (5242880 bytes) is the optimal size, then selected a file to upload, and the progress bar problem in this report started. The problem presented itself right away. While I'm open to the possibility that in the process of setting the container name and port number I have "misconfigured" the container, I think it is more likely that whatever misconfiguration exists is in this release of the official docker image...
The progress bar is updated based on transfer information from the client and server. While the client says, I sent 100% of the data to the server, the server says I got an error and didn't save any data. This causes the progress bar to jump around while the client is retrying.
Makes sense.
The logs from tusd should contain the exact errors that it's encountering. Can you put them here? If it's "too many open files" you might have to raise the limit for open file descriptors.
Okay. I hadn't touched anything yet so I'd have to look up how to do that, the file descriptors thing. I have included the complete log from the docker container since it seems to have gotten worse gradually, so I'm not sure there's an exact moment where it all broke. Hopefully you will find something useful in there. Here's the link: https://drive.google.com/file/d/10PgJgUUntacK3XPXgAHQwsJlwUgGzfzw/view?usp=sharing
Ideally I was hoping to use tusd in production so knowing what happened here would be important. This is my first experience with tus & tusd.
Best leave the option empty if you can. It's only needed in special situation and you don't have to set every option: https://github.com/tus/tus-js-client/blob/main/docs/api.md#chunksize
Thanks. I'll leave that option alone.
Acconut
commented
2 months ago The only error I can find in the logs is too many open files, but it shows up in many places (accepting new HTTP requests, invoking hooks, writing to files), so upload are likely not working properly at all.
This is the first time we have heard about this specific error with tusd, so I don't have concrete advice on how to fix this, but I would recommend you to look into increasing the limit of open file descriptors: https://stackoverflow.com/questions/29403185/docker-error-too-many-open-files
I hope this helps!
Describe the bug Manic behavior of the progress bar in example.
In the example screen recording shown, an upload of a 1.9GB file results in the progress bar jumping around wildly, at one point reaching about 40%, then falling back to like 15% again. A progress bar should gradually move in one direction without major fluctuations (ETA can fluctuate, but not progress!!!)
To Reproduce Use the JS browser client example.
Expected behavior A progress bar that moves from 0 to 100% in the positive direction.
Setup details Please provide following details, if applicable to your situation: