search

lbryio / lbry-sdk

The LBRY SDK for building decentralized, censorship resistant, monetized, digital content apps.
https://lbry.com
MIT License
7.19k stars 482 forks source link

file not saved / not fully downloaded if streaming + calling get #2958

Open tzarebczan opened 4 years ago

tzarebczan commented 4 years ago

I've seen a few instances where save_files + blobs are on (and passed to get), but when the file is streamed from the app, the SDK will only download the requested blobs and not save the file. Doesn't happen each time. See below for a recent log where it didn't happen, but there was a strange stopping inactive download:

2020-05-18 09:54:55,396 INFO     lbry.file.file_manager:198: starting download for lbry://@sushianddesserts#f/chocolate-bonbons-with-brownie-almonds#f
2020-05-18 09:54:55,397 INFO     lbry.stream.managed_stream:147: start downloader for stream (sd hash: fa4a15dc1040b8652ad17c61a9221258ad72d0356d00d898106d9a9605c9c9f8fb035f526fc537bef0af98da5c68a729)
2020-05-18 09:54:57,629 INFO     lbry.blob_exchange.client:159: downloaded fa4a15dc from 51.81.56.48:5567 at 0.110000MB/s
2020-05-18 09:54:57,630 INFO     lbry.stream.downloader:82: downloaded sd blob fa4a15dc1040b8652ad17c61a9221258ad72d0356d00d898106d9a9605c9c9f8fb035f526fc537bef0af98da5c68a729
2020-05-18 09:54:57,635 INFO     lbry.stream.downloader:91: loaded stream manifest fa4a15dc1040b8652ad17c61a9221258ad72d0356d00d898106d9a9605c9c9f8fb035f526fc537bef0af98da5c68a729
2020-05-18 09:54:57,636 INFO     lbry.stream.downloader:110: added head blob to peer search for stream fa4a15dc1040b8652ad17c61a9221258ad72d0356d00d898106d9a9605c9c9f8fb035f526fc537bef0af98da5c68a729
2020-05-18 09:54:57,713 INFO     lbry.file.file_manager:213: paid fee of 10.0 for lbry://@sushianddesserts#f/chocolate-bonbons-with-brownie-almonds#f
2020-05-18 09:54:58,668 INFO     lbry.stream.managed_stream:378: stopping inactive download for lbry://None#None (fa4a15...)
2020-05-18 09:54:59,634 INFO     lbry.stream.managed_stream:147: start downloader for stream (sd hash: fa4a15dc1040b8652ad17c61a9221258ad72d0356d00d898106d9a9605c9c9f8fb035f526fc537bef0af98da5c68a729)
2020-05-18 09:54:59,635 INFO     lbry.stream.downloader:103: searching for peers for stream fa4a15dc1040b8652ad17c61a9221258ad72d0356d00d898106d9a9605c9c9f8fb035f526fc537bef0af98da5c68a729
2020-05-18 09:54:59,639 INFO     lbry.stream.downloader:110: added head blob to peer search for stream fa4a15dc1040b8652ad17c61a9221258ad72d0356d00d898106d9a9605c9c9f8fb035f526fc537bef0af98da5c68a729
2020-05-18 09:54:59,658 INFO     lbry.stream.managed_stream:247: save file for lbry://chocolate-bonbons-with-brownie-almonds#faaea585300f2bff85d2b417c9c9658ed152fd2c (sd hash fa4a15...) -> C:\Users\SoloDolo\Downloads\chocolate-bonbons-with-brownie.mp4
tzarebczan commented 4 years ago

This also happens with non-paid content and looks random. Pretty sure it's related to both streaming and downloading the file at the same time, but it does work most of the time. 

2020-06-29 14:36:53,797 INFO     lbry.file.file_manager:198: starting download for lbry://@Mettaverse#5/live-meditation-music-deep-relaxation#2
2020-06-29 14:36:53,797 INFO     lbry.stream.managed_stream:151: start downloader for stream (sd hash: 411e34a664e07ee79db06af261afb2eb571227117da063895bba17207f25fcc22a199792843eb42671cc6fb1b37cc1a3)
2020-06-29 14:36:53,820 INFO     lbry.stream.downloader:103: searching for peers for stream 411e34a664e07ee79db06af261afb2eb571227117da063895bba17207f25fcc22a199792843eb42671cc6fb1b37cc1a3
2020-06-29 14:36:55,949 INFO     lbry.blob_exchange.client:159: downloaded 411e34a6 from 51.81.56.48:5567 at 3.150000MB/s
2020-06-29 14:36:55,949 INFO     lbry.stream.downloader:82: downloaded sd blob 411e34a664e07ee79db06af261afb2eb571227117da063895bba17207f25fcc22a199792843eb42671cc6fb1b37cc1a3
2020-06-29 14:36:55,977 INFO     lbry.stream.downloader:91: loaded stream manifest 411e34a664e07ee79db06af261afb2eb571227117da063895bba17207f25fcc22a199792843eb42671cc6fb1b37cc1a3
2020-06-29 14:36:55,977 INFO     lbry.stream.downloader:110: added head blob to peer search for stream 411e34a664e07ee79db06af261afb2eb571227117da063895bba17207f25fcc22a199792843eb42671cc6fb1b37cc1a3
2020-06-29 14:36:56,600 INFO     lbry.stream.managed_stream:251: save file for lbry://live-meditation-music-deep-relaxation#22dd9fcec24e061221a71d6eb02a206717ed12ef (sd hash 411e34...) -> D:\downloads\live-meditation-music-deep.mp4
2020-06-29 14:36:56,852 INFO     lbry.blob_exchange.client:159: downloaded 7ffc3eff from 51.81.56.48:5567 at 8.350000MB/s
2020-06-29 14:36:56,864 INFO     lbry.stream.managed_stream:259: write blob 1/1763
2020-06-29 14:36:56,996 INFO     lbry.stream.managed_stream:202: stream file to browser for lbry://live-meditation-music-deep-relaxation#22dd9fcec24e061221a71d6eb02a206717ed12ef (sd hash 411e34...)
2020-06-29 14:36:57,006 INFO     lbry.stream.managed_stream:229: sent browser blob 1/1763
2020-06-29 14:36:57,191 INFO     lbry.blob_exchange.client:159: downloaded c4fed634 from 51.81.56.48:5567 at 6.460000MB/s
2020-06-29 14:36:57,204 INFO     lbry.stream.managed_stream:259: write blob 2/1763
2020-06-29 14:36:57,213 INFO     lbry.stream.managed_stream:229: sent browser blob 2/1763
2020-06-29 14:36:57,510 INFO     lbry.blob_exchange.client:159: downloaded e491c8f4 from 51.81.56.48:5567 at 7.090000MB/s
2020-06-29 14:36:57,523 INFO     lbry.stream.managed_stream:259: write blob 3/1763
2020-06-29 14:36:57,532 INFO     lbry.stream.managed_stream:229: sent browser blob 3/1763
2020-06-29 14:36:57,760 INFO     lbry.blob_exchange.client:159: downloaded 552aea2e from 51.81.56.48:5567 at 9.580000MB/s
2020-06-29 14:36:57,772 INFO     lbry.stream.managed_stream:259: write blob 4/1763
2020-06-29 14:36:57,780 INFO     lbry.stream.managed_stream:229: sent browser blob 4/1763
2020-06-29 14:36:58,127 INFO     lbry.blob_exchange.client:159: downloaded dff6e664 from 51.81.56.48:5567 at 6.080000MB/s
2020-06-29 14:36:58,139 INFO     lbry.stream.managed_stream:259: write blob 5/1763
2020-06-29 14:36:58,148 INFO     lbry.stream.managed_stream:229: sent browser blob 5/1763
2020-06-29 14:36:58,467 INFO     lbry.blob_exchange.client:159: downloaded 092cbaf3 from 51.81.56.48:5567 at 6.610000MB/s
2020-06-29 14:36:58,479 INFO     lbry.stream.managed_stream:259: write blob 6/1763
2020-06-29 14:36:58,488 INFO     lbry.stream.managed_stream:229: sent browser blob 6/1763
2020-06-29 14:36:58,846 INFO     lbry.blob_exchange.client:159: downloaded 55a51c03 from 51.81.56.48:5567 at 5.920000MB/s
2020-06-29 14:36:58,858 INFO     lbry.stream.managed_stream:259: write blob 7/1763
2020-06-29 14:36:58,867 INFO     lbry.stream.managed_stream:229: sent browser blob 7/1763
2020-06-29 14:36:59,200 INFO     lbry.blob_exchange.client:159: downloaded e049d3a2 from 51.81.56.48:5567 at 6.320000MB/s
2020-06-29 14:36:59,213 INFO     lbry.stream.managed_stream:259: write blob 8/1763
2020-06-29 14:36:59,222 INFO     lbry.stream.managed_stream:229: sent browser blob 8/1763
2020-06-29 14:36:59,551 INFO     lbry.blob_exchange.client:159: downloaded 41850136 from 51.81.56.48:5567 at 6.430000MB/s
2020-06-29 14:36:59,563 INFO     lbry.stream.managed_stream:259: write blob 9/1763
2020-06-29 14:36:59,571 INFO     lbry.stream.managed_stream:229: sent browser blob 9/1763
2020-06-29 14:36:59,931 INFO     lbry.blob_exchange.client:159: downloaded 206667ed from 51.81.56.48:5567 at 5.970000MB/s
2020-06-29 14:36:59,943 INFO     lbry.stream.managed_stream:259: write blob 10/1763
2020-06-29 14:36:59,952 INFO     lbry.stream.managed_stream:229: sent browser blob 10/1763
2020-06-29 14:37:00,319 INFO     lbry.blob_exchange.client:159: downloaded 22b109c5 from 51.81.56.48:5567 at 5.740000MB/s
2020-06-29 14:37:00,330 INFO     lbry.stream.managed_stream:259: write blob 11/1763
2020-06-29 14:37:00,339 INFO     lbry.stream.managed_stream:229: sent browser blob 11/1763
2020-06-29 14:37:00,599 INFO     lbry.blob_exchange.client:159: downloaded ee25170b from 51.81.56.48:5567 at 8.120000MB/s
2020-06-29 14:37:00,612 INFO     lbry.stream.managed_stream:259: write blob 12/1763
2020-06-29 14:37:00,620 INFO     lbry.stream.managed_stream:229: sent browser blob 12/1763
2020-06-29 14:37:00,624 INFO     lbry.stream.managed_stream:202: stream file to browser for lbry://live-meditation-music-deep-relaxation#22dd9fcec24e061221a71d6eb02a206717ed12ef (sd hash 411e34...)
2020-06-29 14:37:00,995 INFO     lbry.blob_exchange.client:159: downloaded 586e1c7a from 51.81.56.48:5567 at 6.310000MB/s
2020-06-29 14:37:01,009 INFO     lbry.stream.managed_stream:229: sent browser blob 14/1763
2020-06-29 14:37:01,295 INFO     lbry.blob_exchange.client:159: downloaded 3b2219aa from 51.81.56.48:5567 at 7.560000MB/s
2020-06-29 14:37:01,312 INFO     lbry.stream.managed_stream:229: sent browser blob 15/1763
2020-06-29 14:37:01,650 INFO     lbry.blob_exchange.client:159: downloaded 2924bb62 from 51.81.56.48:5567 at 6.310000MB/s
2020-06-29 14:37:01,671 INFO     lbry.stream.managed_stream:229: sent browser blob 16/1763
2020-06-29 14:37:02,007 INFO     lbry.blob_exchange.client:159: downloaded b1045963 from 51.81.56.48:5567 at 6.280000MB/s
2020-06-29 14:37:02,019 INFO     lbry.stream.managed_stream:229: sent browser blob 17/1763
2020-06-29 14:37:02,322 INFO     lbry.blob_exchange.client:159: downloaded aa6ee192 from 51.81.56.48:5567 at 6.970000MB/s
2020-06-29 14:37:02,335 INFO     lbry.stream.managed_stream:229: sent browser blob 18/1763
2020-06-29 14:37:02,664 INFO     lbry.blob_exchange.client:159: downloaded 8f93f162 from 51.81.56.48:5567 at 6.400000MB/s
2020-06-29 14:37:04,084 INFO     lbry.dht.protocol.iterative_find:379: blob peer search bottomed out
2020-06-29 14:37:13,510 INFO     lbry.stream.managed_stream:229: sent browser blob 19/1763
2020-06-29 14:37:13,823 INFO     lbry.blob_exchange.client:159: downloaded 82b1d23d from 51.81.56.48:5567 at 6.740000MB/s
2020-06-29 14:37:13,835 INFO     lbry.stream.managed_stream:229: sent browser blob 20/1763
2020-06-29 14:37:14,059 INFO     lbry.blob_exchange.client:159: downloaded f1316957 from 51.81.56.48:5567 at 9.410000MB/s
2020-06-29 14:37:31,202 INFO     lbry.dht.protocol.iterative_find:379: blob peer search bottomed out
tzarebczan commented 4 years ago

Worked when I downloaded via get directly, but not when triggered from app + using streaming.