wont get passed while (not handle.has_metadata()): using libtorrent 2.0

[macordalfy]

Please provide the following information

libtorrent version (or branch): 2.0

platform/architecture: linux / centos / python

compiler and compiler version: gcc+ 14 / boost 1.74

please describe what symptom you see, what you would expect to see instead and how to reproduce it. I have the following code, and it will always gets stuck on the has_metadata, it wont get passed it ... even after 30min .. nothing

settings_pack = {
                'connection_speed': 20, 
                'enable_dht': 0, 
                'listen_interfaces': '0.0.0.0:63255', 
                'peer_fingerprint': lt.generate_fingerprint('qB', 3, 3, 5, 0), 
                'user_agent': 'qBittorrent/3.3.5', 
                'dht_bootstrap_nodes': 'router.bittorrent.com:6881,router.utorrent.com:6881,router.bitcomet.com:6881'
            }
            print('starting')
            ses = lt.session(settings_pack, flags=0)            

            params = {
                'save_path': 'to_download/',
                'storage_mode': lt.storage_mode_t(2)
            }

            handle = lt.add_magnet_uri(ses, content_results['magnet'], params)
            handle.set_max_connections(-1)

            print(str(datetime.now())+' downloading metadata...')
            while (not handle.has_metadata()):
                  time.sleep(1)

[arvidn]

so, perhaps the magnet link points to an invalid torrent or a torrent that have no peers.

You can trouble shoot with this: http://libtorrent.org/img/troubleshooting.png

[wiserain]

Happened same thing for me.

I've been using handle.has_metadata() method for years. It was okay with v2.0.0, but no response from has_metadata() in this release v2.0.1.

I am playing with various environment like alpine 3.10-3.12, ubuntu 19.10-20.10 and python 2 and 3. At first, I thought it could be build error, but they are all the same.

Build ... OK test.py in bindings/python ... OK simple_client.py in bindings/python ... confirmed file downloading! which means no problem with connections and peers/seeders?

Is there anything I can check further?

Thanks!

P.S. No problem with v1.2.11.

[arvidn]

your program cannot rely on has_metadata() ever returning true when adding a torrent via magnet link. That's your main problem.

Now, if there is a swarm at the info-hash, and you can connect to a peer and that peer supports the metadata extension and it can send you the metadata, then you will (eventually) receive it. There is no guarantee that that will happen though.

So, there may be a secondary problem causing something to fail in that chain of events (bootstrapping DHT or contacting tracker, finding peers, connecting to peers, peers supporting the metadata extension). You can enable logging alerts, and print them to a file to see details about what's going on with all of those things.

You may want to take a look at this trouble shooting guide: http://libtorrent.org/img/troubleshooting.png

[wiserain]

Thanks for the quick reply. I got your point and will dig into the chain of events with an assist of the trouble shooting guide you referred and logging alerts. I may have to change my python logic to get metadata from magnet uri.

One thing that I don't understand is that it was okay with previous releases. Again I've been tracking has_metadata() behavior since v1.2.3. It wasn't that reliable as you mentioned and thus I have to set timeout and retries to tackle intermittent failure. However, in this release, it never gets no matter how long I wait or how many times I tried. What do you think making this change?

And is there any proper/reliable way to get metadata from magnet uri?

[wiserain]

From v2.0.1, adding extension 'ut_metadata' to a session is required to get metadata. Specifically, I had to add session.add_extension('ut_metadata') at the end of the following codes,

# session
settings = {
    # basics
    'listen_interfaces': '0.0.0.0:6881',
    # dht
    'enable_dht': True,
    'use_dht_as_fallback': True,
    'dht_bootstrap_nodes': 'router.bittorrent.com:6881,dht.transmissionbt.com:6881,router.utorrent.com:6881,127.0.0.1:6881',
    'enable_lsd': True,
    'enable_upnp': True,
    'enable_natpmp': True,
    'announce_to_all_tiers': True,
    'announce_to_all_trackers': True,
    'aio_threads': 4*2,
    'checking_mem_usage': 1024*2,
}
session = lt.session(settings)

One question here. Is it possible including add_extension() into settings_pack dict?

Thanks!

[arvidn]

ut_metadata is supposed to be added by default. I will take a look at why it wasn't in this case.

[arvidn]

I notice that you pass in flags=0 to the session constructor. I believe that's what's disabling the adding of default plugins, such as ut_metadata and ut_pex. I take it that wasn't your intention with setting flags o zero. What was it?

[wiserain]

@arvidn I am not the one opened this issue and didn't give any additional arguments other than a single settings_pack dict like I commented yesterday.

[arvidn]

@wiserain thanks! would you mind trying this patch? https://github.com/arvidn/libtorrent/pull/5772

[wiserain]

@arvidn Confirmed working without adding extensions explicitly. Thanks.