File not found while name changes

[JamesOlvertone]

There are some situations (I dont know why) muwire looses files from the download list after a restart. The files are still in the inclomplete folder with name+hash.part and name+hash.pieces but that is not the point here. After you search again for this files and add them again to download they continue downloading where they stopped. So a 99% file is recognized again as 99% and only the missing 1% were downloaded and finished.

Now this situation:

1. There is a file shared from someone with only one source, he is the only sharer, no other sources of the file exist. Start download this file so that it appears in download list and exists in incomplete folder (.part and pieces) maybe there are some bytes downloded or not it doesnt matter.

2. Exit muwire

3. The Source/Sharer changes now filename, content is unchanged, file has same hash!

4 Start muwire again, the file disapeared in the download list.

5. Now you think a no problem search for the file again, muwire recognized it with the hash and it will continue the file but with the old name. No it doesn't. It generates a new name+hash.part and name+hash.pieces and even worse it never starts downloading this file maybe it throws an error (didn't check the logs yet).

It works if you skip stept 5 and do

6. Exit muwire, rename the incomplete file to the new one, restart muwire download succeeds.

muwire should recognize the file with the old name and continue here, it has the hash to identifiy it.

[zlatinb]

The first problem I want to address is why the file disappears from the download list at step 4. This should not happen. If the file is in the download list then the rest of the problems can be solved with searching for the file again because muwire will recognize the hash is the same and pick up from where it left.

Downloads in progress are persisted in a file called downloads.json in the MuWire settings directory. Can you see if that file gets corrupted or disappears somehow?

A new I2P release is due today and the 0.8.13 MuWire release is going to be today or tomorrow, so unless this is super-trivial to fix I will have to leave it for later.

[JamesOlvertone]

I have not the time right now. I will have an eye on that the next days.

[JamesOlvertone]

There are some of this files that appear in the download window, they do download, but their hash (that whats part of the filename+hash.part|pieces or "copy hash to clipboard when you right click it in the downloadlist) can not greped in the download.json !?

Other files' hash that are ok can be grepped in the download.json they appear as "hashRoot":"thehash=

[zlatinb]

This is very bad. If you restart MuWire those downloads will be lost, even if the .part and .pieces are still in the incompletes folder.

I have no idea what is causing this and I may have to rewrite the persistence of downloads in progress for the next version...

[zlatinb]

Actually no I'm wrong - if the entries do not have a hashRoot then they must have hashList. That means the full hash list has been fetched and is now persisted.

It is only a big problem if an entry does not appear in downloads.json at all.

[zlatinb]

The more I think of it the more likely it is I'm going to rewrite the persistence logic. Having a centralized file to keep track of all downloads is very unstable. I'll most likely move to a single json file per download and store that in the incompletes folder.

[JamesOlvertone]

Yes like emule did: .part-file + .met-file. Its easier to manage for the user when something goes wrong. You instantly see what files are damaged when some files are missing, you can temporarely move them out from the incompletefolder for easier testing or move to another version of muwire even on another machine ... I miss this a lot in muwire. I did this with emule a lot, and worked even cross versions from windows emule <--> aMule,...

It should be compatible with your new seed-daemon. There should be a function to forward a download to a seed-daemon, or at least some export and import functionality or it should work by moving from one incompletefolder to another.

[zlatinb]

Moving incomplete downloads between nodes is very advanced functionality but it will work. However I'm gonna keep the seedbox daemon out of this completely - it is designed only to upload (seed).

Anyway I'll keep this issue open and update it as I make progress on the code..

[Searinox]

Same happened when I was copying a new file into a shared folder:

java.io.FileNotFoundException: <PATH TO FILE> (The process cannot access the file because it is being used by another process)
    at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(Unknown Source)
    at com.muwire.core.files.FileHasher.hashFile(FileHasher.groovy:65)
    at com.muwire.core.files.FileHasher$hashFile.call(Unknown Source)
    at com.muwire.core.files.HasherService.processFile(HasherService.groovy:139)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
    at com.muwire.core.files.HasherService$_throttle_closure4.doCall(HasherService.groovy:111)
    at com.muwire.core.files.HasherService$_throttle_closure4.doCall(HasherService.groovy)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)

The best thing I can imagine doing here is wrapping this code in an exception and repeat trying until success or file no longer exists.

[zlatinb]

@Searinox this exception is actually unrelated to the original issue of losing downloads. I don't think I can just retry until it succeeds because this exception happens when a file cannot be found, which could happen if the file was genuinely deleted. Also I can't reliably check the error message because that sometimes get translated by the java virtual machine to the local language.

What I'll do is add the exception and only retry if the file actually exists. I'll tag the commit with this issue so you can see the exact logic.