File transfer hangs at 99.9% and 100%

[AXDOOMER]

This is a new problem that I have in Linux Mint 17.2 (kernel: 3.19.0-21-generic). I never had this problem before because older versions didn't have the sequential file transfer tool, so I suspect this is the cause of the problem. I often copy files from any sizes and it does this. It hangs and I have to wait from 5 to 10 minutes. I am transferring from my SSD (~/Download) to my USB Flash Drive (CANADA).

In the case where this would be a kernel related bug, could the file transfer tool be updated so it could take the "hanging" factor into account when it transfers file? (so for example, it would tell the user that the file transfer tool is waiting for the drive to complete its operation)

It is a serious issue when the files transfer tools tells there's "0 seconds" left to the transfer and then the user has to wait 10 minutes. Also, when there is 1 or 0 second left, it should be written "# second" instead of "# seconds" (plural form).

[Vahan86]

@AXDOOMER, is this still an issue for you in Nemo 3.0.6 (Mint 18)?

[AXDOOMER]

I'm not running Mint anymore, but I'm sure this is still an issue. If you want to know if it's fixed, find the commit that patches this bug. As far as I know, no one worked on this bug to fix it.

There is a reason why it happens. The file is copied into a buffer. When the file is completely in the buffer, it displays 99.9%, but it's still being written to the USD flash drive which is slow. If the buffer and the file are huge, then it stays at 99,9% for a long time.

[cesc1989]

This is still a thing in Linux Mint 19.1

[leandrorebelo]

This bug happens in Linux Mint 19.1 :(

[UriNeri]

@AXDOOMER Did you find a distro without this bug ? (Still exist in Linux Mint 19)

[AXDOOMER]

@UriNeri As far as I know, this bug also happens on Xubuntu. I didn't notice it on other distros, which doesn't mean it doesn't exist elsewhere.

I have 16GB of RAM in my computer and if I copy a 4GB file to a slow USB flash drive, it copies at more than 100MB/s for the first 2GB, then it decreases to ~1MB/s for the remaining. When it reaches 99%, it hangs.

There must be a way for the distro to be tuned so it uses a smaller buffer, especially when copying to slow devices. Another possibility would be to find out the amount of data that was written to the disk, in order to get the real amount of data copied.

The best explanation for this bug is what I have read here: https://unix.stackexchange.com/a/181236

[UriNeri]

@AXDOOMER Thanks. I switched to Xubuntu and the Issue doesn't reoccur (or rather, I haven't encountered it yet). Overall, while missing some fancy features, Thunar is less buggy than nemo. No disrespect for the Devs, of course, who do wonderful work.

For the issue though, a dynamic control on buffer size would be optimal, and it would be independent of distro (as your link suggest, the cause is could be at the kernel level).

[shyaminayesh]

Issue is still here. Running nemo v4.4.2 on Arch linux with Kernel 5.5.9 & file copy process hang on 98%-100% when copyig to external drive.

[arievanden]

I've been experiencing the exact same issue, but with latest Manjaro KDE install. I'm loving it, except for the file copy lag issues. I first through it was a problem with copying to Samba share, but I tested a copy to USB thumb drive and the slowdown is exactly the same. When copying a large file like 4gig file, it starts to slow down around 50% of the way, then takes a steep drive in write speed until it hangs. If you let it be for a while, it will sometimes finish. But it hangs for a long time at 99%, then usually completes. File utilities like Dolphin will hang/lag until the copy is complete.

[ghost]

Still present in Manjaro 20.1 (Linux kernel 5.8.6-1) and Nemo 4.6.5-1. Tried to copy a file from one external drive (USB-C) to another (USB 3.0) - took about 2 minutes to copy it but another 5 minutes to actually go from 99% to 100% and finishing off.

[Jeremy7701]

When you output to a device from memory, you have no way of knowing how many bytes have completed their journey from memory buffer to final storage successfully. Even when nothing is apparently happening, the USB is still running at full speed. USB is notoriously slow when writing - perhaps half the read speed of the same USB. An average PC can easily buffer several GBytes of data. A "fix" could involve writing a small portion, waiting for it to complete before dribbling another small portion and so forth. You could tell the kernel to use smaller amounts of buffer space for all processes.... This would be wastefully slow. One solution would be to get a smaller computer.

[AXDOOMER]

Instead of buffering Gigabytes of data, the buffer could instead be, let's say, 256 Megabytes. This way it's not slow and wasteful, but it reduces the time you have to wait for your file to complete when it's at 99% completion.

[Jeremy7701]

Well, you could use spare memory as at present, but if you discover you are writing to a USB device you could issue the equivalent of a sync command (see man page) every 256MB to force completion of any outstanding output. This will reduce the already slow output performance of USB - quite possibly substantially. But it would allow your progress bar to be more active. Might be worth trying a loop in a script using split and sync to see how well a big file split into 256MB chunks can be written (with a sync after each chunk is written).

[dipenSan]

I still face this problem in Mint 20 Cinnamon.

[icarter09]

Nemo v4.8.2 Mint 20 64bit

I've tried to duplicated this bug several times with no luck. Attempts ranged from copying 4GBs to 15GBs from ~/Downloads to USB thumb drive. Did not experience any hang ups around the 99% marker nor did I see any slowing down around the 50% marker. I've also tested out smaller file sets and sizes as well, but still not able to reproduce the issue.

Let me know if I'm missing any steps or if anyone wants me to test other ideas out. Also let me know if you are still experiencing the issue and I'll help to get it figured out. Thanks everyone.

[AXDOOMER]

Nemo v4.8.2 Mint 20 64bit

I've tried to duplicated this bug several times with no luck. Attempts ranged from copying 4GBs to 15GBs from ~/Downloads to USB thumb drive. Did not experience any hang ups around the 99% marker nor did I see any slowing down around the 50% marker. I've also tested out smaller file sets and sizes as well, but still not able to reproduce the issue.

Let me know if I'm missing any steps or if anyone wants me to test other ideas out. Also let me know if you are still experiencing the issue and I'll help to get it figured out. Thanks everyone.

Use a slow USB device. Newer 3.0 USB flash drives are very fast, sometimes faster than a hard drive, so they are able to write all of the data before it gets accumulated in the system's cache.

[cesc1989]

Use a slow USB device. Newer 3.0 USB flash drives are very fast, sometimes faster than a hard drive, so they are able to write all of the data before it gets accumulated in the system's cache.

It's a fair point but let me add that I faced this issue in a not too old portable HDD with USB 3.0. In the very early days I'd see transfers going good but up to some point I started noticing the "hanging" at 99%. One day I couldn't wait any longer and unplugged the HDD. Never worked again.

So, the issue could be easier to see in slower devices but it happened in a relative new and fast one.

[GoldenGlimpse]

How can this still be a thing after all those years. What an extremely annoying bug that is!! I'm still having this issue on:

Distro: Linux Mint 20.1 Ulyssa Desktop: Cinnamon 4.8.5 Kernel: 5.4.0-60-generic x86_64 bits: 64 Base: Ubuntu 20.04 focal

[icarter09]

@GoldenGlimpse could you provide some steps to reproduce or what devices you are using when this issue occurs? Thanks.

[GoldenGlimpse]

open nemo 4.8.4 plugin a 32GB TDK LoR TF10 (fat32) copy 1.8gb movie.mp4 to stick few seconds ~140mb/s to 99% from /home/user/Downloads then wait 18min to complete

[Jeremy7701]

You need a much bigger file - 1.8GB could easily sit in various buffers. If you could try a directory of your favourite videos - hopefully bigger than your available RAM, then you might see some action.

[GoldenGlimpse]

I have 16GB of RAM in my computer and if I copy a 4GB file to a slow USB flash drive, it copies at more than 100MB/s for the first 2GB, then it decreases to ~1MB/s for the remaining. When it reaches 99%, it hangs.

...

I did the same process with a 4GB movie before. I would have the same explanation as the above qoute. With a 4gb movie.mkv it copied fast to 50% (2gb) then slowed down to 10mb/s and another 15 min stuck at 99% before finishing. I assume it's going to be the same thing with a big library? I also have 16GB of ram. The Linux Mint installation is with encryption and LVM.

[guidoguy22]

The same problem on Ubuntu 20.04. Seems like no one is trying to fix it even though it is pretty major bug.

[r4jk3]

I also confirm that problem is present on Ubuntu 22.04.

Copying hangs like this for more then 10 mins already ( not finished yet ):

I'm using SanDisk 128gb flash drive plugged into 2.1 port on laptop.

[Jeremy7701]

USB 2.0 interface speed is 480Mbps (60MB/s) - so you might expect a USB stick to take about a minute. However, speeds of:- Sequential write speed range = 3 -10 MB/s Sequential read speed range = 10 - 25 MB/s would be considered good or even 'excellent'. So 20 minutes to copy a big video shouldn't be considered unusual.

The problem is that the USB device does not give any feedback as to how much data has been accepted and permanently written to the USB device. Data sitting in buffers in RAM or even buffered by hardware cannot be considered as "written".

[zbandhan]

This is really an annoying issue. Not sure if it's a upstream or Nemo issue. Does anyone know if there is a temporary solution for it?

[Jeremy7701]

It's either a hardware "issue" or a lack of understanding on the part of the user. A "solution" would be to write the data in smallish blocks, then forcing data out to the USB before proceeding with the next block. The effective data rate would be hugely slower, though.