Feature request: Improve support for disconnecting USB/Network drives

[Dwedit]

Right now, MultiPar doesn't have good support for disconnecting a USB drive. It detects "device not ready", turns the progress bar red, then aborts everything.

Would it be hard to check every few seconds if the device has become available again, then resume the operation?

This would help with USB drives being removed, as well as Wifi signal failure for network drives.

[Yutaka-Sawada]

Would it be hard to check every few seconds if the device has become available again, then resume the operation?

Technically it may be possible. But, I need to know what error Windows OS returns at when. Did MultiPar save the error on MultiPar.log ? Please send the log file to me, or post here. If the error was not fatal and able to retry, I will consider the way.

[Dwedit]

I did not save any log files, but I do know Windows Error numbers.

You get ERROR_FILE_INVALID (1006) when a USB device is disconnected, and you get ERROR_UNEXP_NET_ERR (59) when network drive IO fails.

Another error code you could get is ERROR_INVALID_HANDLE (6), but to get that one, you need to use a third party tool such as "Unlocker" to force-close a file handle.

These are error codes you see from GetLastError.

I know that these apply to ReadFile, If you are using overlapped IO, ReadFile always succeeds (or returns ERROR_IO_PENDING), and the error codes apply to GetOverlappedResult instead.

In general, when you get one of those errors, you close the file handle, then try to reopen it. I haven't yet tested what error codes appear then. I also have not yet tested what error codes you get when file writes fail.

[Yutaka-Sawada]

At first, I need to decide which point I change. What action did you do at the error ? Such like, Creation or Reapir. Also, file access mode is unknown; sequential scan or random access. If you didn't change default setting, it might use sequential scan mode for external drive. But, it may use random access mode, when recovery data is large as compared to free RAM size. So, more information is helpful. At this time, I don't know the point the error happend.

In general, when you get one of those errors, you close the file handle, then try to reopen it.

Oh, I see. I may try with a USB memory on my PC.

[Dwedit]

In this case, I was trying to create PAR files for a very large disk image (2TB), and it was about 15% finished with reading the file.

I pressed the "Pause" button, waited a few seconds, unplugged the drive. I later plugged the drive back in again, then pressed the "Resume" button. But the "Resume" command did not work. The progress bar turned red, and I saw no sign of the program attempting to continue.

I'd assume the program was trying to use the same file handle. But the file handle was now in an invalid state, so any attempt to read from it would give ERROR_FILE_INVALID until the file handle was recreated.

Perhaps the Pause button should close all file handles to allow for orderly removal. That could help in cases where you know you are intending to remove the drive.

But there are other cases that result in drives unexpectedly being removed. Such as loose USB cables, or a wifi connection going down.

[Yutaka-Sawada]

I implement a method to retry after disconnected USB memory on my PC. I don't know that this works with other external drives. At this time, it works with creating PAR2 index file on a external drive. When it opens or reads a source file to calculate hash value, it will wait max 30 seconds and retry at some errors. ERROR_PATH_NOT_FOUND (3) ERROR_NOT_READY (21)

The waiting and retrying are done automatically. You may see the error number at Command-prompt. You cannot see any error on MultiPar GUI. If you save log, the error is written on the log file. You can Pause or Cancel progress while waiting.

I put the debug version (par2j_debug_2021-12-22.zip) in "MultiPar_sample" folder on OneDrive. You may test creating Index file. In the test, redundancy must be 0%, because this won't retry at creating recovery data. If this method works, I will try to add in a function of calculating recovery data.

[TokuNaNa]

Is this feature available in v1.3.2.7 beta?

I pressed the "Pause" button, waited a few seconds, unplugged the drive. I later plugged the drive back in again, then pressed the "Resume" button. But the "Resume" command did not work. The progress bar turned red, and I saw no sign of the program attempting to continue.

Progress bar reaches 100% and the log look good, but Index file is broken. MultiPar.log

[Yutaka-Sawada]

Is this feature available in v1.3.2.7 beta?

No, is was canceled. I didn't adopt the feature into public version. Though I put the sample code in md5_crc.c still, it's not tested. I don't implement a feature, while nobody test it. Big change requires strong mental power and supporter's aid. When no one is eager to request, it means that the feature has no worth to try. As I'm lazy, I don't do what I don't need to do.

[TokuNaNa]

@Yutaka-Sawada Ok. No problem!