Missing torrent after crash

[pimlie]

• [X] I have read CONTRIBUTING.rst

• [X] I have tried with the latest pre-release version and I still can reproduce the issue.

Tribler version/branch+revision:

$ git status On branch next Your branch is up-to-date with 'origin/next'. nothing to commit, working directory clean

Operating system and version:

Ubuntu 16.04

This morning I was cleaning up the finished torrents, I had only one torrent still downloading at ~5KB/s. After removing two torrents from the GUI, tribler crashed as mentioned in #3039 But after the restart the torrent that was still downloading is gone. Files still exists in the download folder but the torrent is not listed anymore.

Looking at the logs the only thing that is strange is that currently the log is filled with a lot of these messages. These messages are not showing before the crash, but after the crash they did.

WARNING:SearchCommunity:unable to find conversion to decode 0001fb04df93369587ec8fd9b74559186fa356cffda8f561f1730ce2b03f1ed74582504d684a078d0ece4700000000000000025ed0001e1e4f83b41b9b191783b41b9b1917c0a8010c1e4f4e2d7ace1e4f00611503623b515f7f125bc9748c39a3b2678d6c3092b7f6007831d126f308c63f242e549784a84ea16dc55932 in [<DefaultConversion 0000 [dispersy-dynamic-settings, dispersy-missing-identity, dispersy-missing-sequence, dispersy-missing-message, dispersy-puncture-request, dispersy-undo-other, dispersy-signature-response, dispersy-destroy-community, dispersy-introduction-response, dispersy-identity, dispersy-signature-request, dispersy-authorize, dispersy-puncture, dispersy-missing-proof, dispersy-revoke, dispersy-undo-own, dispersy-introduction-request]>, <SearchConversion 0002 [dispersy-missing-message, torrent-collect-request, dispersy-signature-response, search-response, dispersy-missing-identity, torrent-collect-response, dispersy-puncture-request, dispersy-undo-other, dispersy-dynamic-settings, dispersy-undo-own, dispersy-introduction-request, dispersy-missing-sequence, dispersy-destroy-community, dispersy-introduction-response, dispersy-identity, dispersy-signature-request, dispersy-revoke, dispersy-puncture, torrent-request, search-request, dispersy-authorize, dispersy-missing-proof, torrent]>]

[pimlie]

This behaviour seems to be consistent. If you add a new magnet link it is first only visible under tab 'ALL' with the name Unknown. Due to the anonimity layer it can take quite a while before a torrent file is retrieved and the torrent is listed under Downloading. When tribler crashes then it always 'forgets' all torrents that are still Unknown and you have to add them manually again.

Besides the bug part, maybe it would also be nice to add a tab that lists just these Unknown torrents as sometimes these torrents can stay Unknown for a couple of days.

[Griffon26]

I'm not entirely certain if it's the same problem, but when I close tribler while some torrents are still "building circuits", they will all have been removed from the list when I start tribler the next time. This is particularly troublesome because even torrents that had already started downloading will start again in "building circuits" after a restart.

[Captain-Coder]

I've observed this problem too in development work. Specifically it generates the following log line:

LibtorrentDownloadImpl:841 Resume data failed to save: <class 'Tribler.Core.exceptions.SaveResumeDataError'>: torrent has no metadata

@devos50, hope that helps :)

[qstokkink]

According to the documentation we should disable the torrent_handle::save_info_dict flag to save the torrent without its metadata.

Update: n.v.m. even though the above approach wont save metadata, the torrent still needs to have it.

[pimlie]

@devos50 @qstokkink this issue appears not to be fully resolved.

Currently my Tribler app is unable to connect to the tribler network. Not sure why, sometimes it just cant for a couple of days. This means that all torrents have the status Building circuits.

If I add a new magnet link, it will be visible in the ALL tab with name 'Unknown' but if I restart tribler that torrent / magnet link is not listed anymore.

# git status
On branch next
Your branch is up-to-date with 'origin/next'.

nothing to commit, working tree clean

# git log
[[33mcommit 517a1cfd665aba3d9d4efd22d369ceb4964cabde[[m[[33m ([[m[[1;34mgrafted[[m[[33m, [[m[[1;36mHEAD -> [[m[[1;32mnext[[m[[33m, [[m[[1;31morigin/next[[m[[33m)[[m
Author: Martijn de Vos
Date:   Fri Aug 25 16:28:45 2017 +0200

    Merge pull request #3062 from qstokkink/various_fixes

    Various fixes for Tribler 7

[qstokkink]

@pimlie Hmm that seems like the perfect storm. I guess it's stuck in the twilight zone of fetching the metadata for the magnet link, which we require for adding a download, which in this case does not get saved then.

The fix for this would be to extract and add the infohash of the magnetlink before fetching the metadata.

[qstokkink]

@pimlie do you have an easy way to test this? I believe #3069 should resolve this, but setting up your scenario is kind of difficult to do programmatically.

Actually, nevermind, by simply quickly hard-killing the Tribler process I verified this myself.

[pimlie]

@qstokkink If you make sure that docker cannot connect to the internet I would believe you can replicate the issue (but I didnt test this). E.g if you run Tribler in docker you could probably run something like iptables -t nat -D POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE so the traffic from your docker containers is not masqueraded. Afterward run the same command but with -A instead of -D