search

Ezwen / bandcamp-collection-downloader

A command-line tool to automatically download all releases purchased with a Bandcamp account. The official page of the project is https://framagit.org/Ezwen/bandcamp-collection-downloader, while here this is just a mirror hosted on Github.
GNU Affero General Public License v3.0
246 stars 25 forks source link

"Connection reset" error when file downloaded #29

Closed Ponywka closed 2 years ago

Ponywka commented 2 years ago

OS: Windows 11 21H2 (22000.194) Java version: OpenJDK 11.0.2 / OpenJDK 17.0.1 / Oracle JDK 17 CLI command: java -jar bandcamp-collection-downloader.jar -j 1 -r 100 -f mp3-320 -d bandcamp320 ponywka Album in queue: https://mirroredreality.bandcamp.com/track/pony-girl-mirroredreality-remix

When the file has been downloaded completely, the program gives an error and tries to download the file again.

WindowsTerminal_s2TLTZ6WTf

Md5Checker_fBu08pujck

Ezwen commented 2 years ago

Interesting, so what you are saying is that even if the downloaded file is complete (as shown by the MD5 hashes), it keeps retrying to download it again?

Strange. Can you easily reproduce the bug?

Ponywka commented 2 years ago

Interesting, so what you are saying is that even if the downloaded file is complete (as shown by the MD5 hashes), it keeps retrying to download it again?

Yep

Can you easily reproduce the bug?

https://20101010.xyz/public/bandcamp-collection-downloader-2021-11-30-19-58-12.mp4

Ezwen commented 2 years ago

I bought the track for 50 cents just to test, and everything went fine. So it's not specific to this item, and is something related to your setup, I suppose. I wonder what this could be.

Ponywka commented 2 years ago

Interesting... 🤔

Ponywka commented 2 years ago

OS: Arch Linux Java version:

openjdk version "17" 2021-09-14
OpenJDK Runtime Environment (build 17+35)
OpenJDK 64-Bit Server VM (build 17+35, mixed mode)

Same problem https://asciinema.org/a/Cne8yYoxpQc8LhPqC45CmiNQI

Ezwen commented 2 years ago

I've added an option --debug to display the complete error stack trace, and a test build with this new feature can be downloaded here: https://framagit.org/Ezwen/bandcamp-collection-downloader/-/jobs/1521019/artifacts/raw/build/libs/bandcamp-collection-downloader.jar?inline=false

Can you try again with this test build, and report back here the complete error stack trace?

Ponywka commented 2 years ago 
Error while trying: "java.net.SocketException: Connection reset".

java.net.SocketException: Connection reset
        at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323)
        at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
        at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
        at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
        at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
        at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
        at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
        at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
        at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500)
        at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421)
        at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
        at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
        at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
        at bandcampcollectiondownloader.util.RealIO.downloadFile(RealIO.kt:44)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadRelease(BandcampCollectionDownloader.kt:303)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader.access$downloadRelease(BandcampCollectionDownloader.kt:17)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:248)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:245)
        at bandcampcollectiondownloader.util.Util.retry(Util.kt:66)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:245)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:119)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:132)
        at bandcampcollectiondownloader.cli.Command.call(Command.kt:94)
        at bandcampcollectiondownloader.cli.Command.call(Command.kt:14)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)
Ponywka commented 2 years ago

Also in Arch Linux: https://20101010.xyz/public/bcd-2021-12-08-14-59-33.mp4 00:00 - System info 00:32 - Start downloading & Crashes when downloading singles 01:40 - Downloading albums without chashes

Error while trying: "java.net.SocketException: Connection reset".                                                                 
java.net.SocketException: Connection reset
    at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323)
    at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
    at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
    at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
    at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
    at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
    at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
    at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500)
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421)
    at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
    at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
    at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
    at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
    at bandcampcollectiondownloader.util.RealIO.downloadFile(RealIO.kt:44)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadRelease(BandcampCollectionDownloader.kt:303)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.access$downloadRelease(BandcampCollectionDownloader.kt:17)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:248)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:245)
    at bandcampcollectiondownloader.util.Util.retry(Util.kt:66)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:245)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:119)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:132)
    at bandcampcollectiondownloader.cli.Command.call(Command.kt:94)
    at bandcampcollectiondownloader.cli.Command.call(Command.kt:14)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
    at picocli.CommandLine.access$1300(CommandLine.java:145)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
    at picocli.CommandLine.execute(CommandLine.java:2078)
    at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)

Could not download item: Could not perform task after 0 retries.                                                                  
bandcampcollectiondownloader.core.BandCampDownloaderError: Could not perform task after 0 retries.
    at bandcampcollectiondownloader.util.Util.retry(Util.kt:75)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:245)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:119)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:132)
    at bandcampcollectiondownloader.cli.Command.call(Command.kt:94)
    at bandcampcollectiondownloader.cli.Command.call(Command.kt:14)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
    at picocli.CommandLine.access$1300(CommandLine.java:145)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
    at picocli.CommandLine.execute(CommandLine.java:2078)
    at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)
Ezwen commented 2 years ago

Great thanks. So the problem strangely arises when trying to download the cover image for the track.

I've added yet more debug printing, to have a better look at what is the URL of the cover file it tries do download. Can you try again with this new test build here, using --debug once more: https://framagit.org/Ezwen/bandcamp-collection-downloader/-/jobs/1521424/artifacts/raw/build/libs/bandcamp-collection-downloader.jar?inline=false

Ponywka commented 2 years ago

As I understand it, in Russia (at least at the Internet provider Rostelecom and Tele2), the domain of pictures at Bandcamp is blocked. This makes sense because usually the Bandcamp page looks like this for me: firefox_jCzQlFNVUN

Managing item 1/93                                                                                                                
Found release "Pony Girl (MirroredReality Remix)" (2018) by MirroredReality (Bandcamp ID: r152688695).                            
Starting the download of "Pony Girl (MirroredReality Remix)" (2018) by MirroredReality.                                           
[DEBUG] Starting download of: https://p4.bcbits.com/download/track/12c24612d74aee5a3ae64fa9a7c39976d/mp3-320/3850187924?id=3850187924&sig=2f115bb4dd67221746821bd767fd2ea4&sitem_id=152688695&token=1639579443_aa3ad6ca4549c315d25ad2df66e7ac3f55b4fcd4
Progress: 0.00 %                                                                                                                  
...                                                                                              
Progress: 99.99 %                                                                                                                 

[DEBUG] This is a single track release: fetching the cover                                                                        
[DEBUG] Starting download of: https://f4.bcbits.com/img/a637921479_10                                                             
Error while trying: "java.net.SocketException: Connection reset".                                                                 
[DEBUG] [DEBUG]java.net.SocketException: Connection reset
    at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323)
    at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
    at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
    at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
    at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
    at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
    at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
    at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500)
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421)
    at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
    at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
    at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
    at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
    at bandcampcollectiondownloader.util.RealIO.downloadFile(RealIO.kt:43)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadRelease(BandcampCollectionDownloader.kt:315)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.access$downloadRelease(BandcampCollectionDownloader.kt:18)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:258)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:255)
    at bandcampcollectiondownloader.util.Util.retry(Util.kt:35)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:255)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:131)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:142)
    at bandcampcollectiondownloader.cli.Command.call(Command.kt:96)
    at bandcampcollectiondownloader.cli.Command.call(Command.kt:13)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
    at picocli.CommandLine.access$1300(CommandLine.java:145)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
    at picocli.CommandLine.execute(CommandLine.java:2078)
    at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)

Could not download item: Could not perform task after 0 retries.                                                                  
[DEBUG] [DEBUG]bandcampcollectiondownloader.core.BandCampDownloaderError: Could not perform task after 0 retries.
    at bandcampcollectiondownloader.util.Util.retry(Util.kt:42)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:255)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:131)
    at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:142)
    at bandcampcollectiondownloader.cli.Command.call(Command.kt:96)
    at bandcampcollectiondownloader.cli.Command.call(Command.kt:13)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
    at picocli.CommandLine.access$1300(CommandLine.java:145)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
    at picocli.CommandLine.execute(CommandLine.java:2078)
    at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)
Ezwen commented 2 years ago

Ah I see, I did not know that! Well at least now we have nailed the problem :)

Since it will be difficult to detect programmatically whether the problem is a (temporary) real network error, or a (permanent) proper blocking from the ISP, instead we can consider adding a new flag in the app to choose to not download the covers for single-track releases. WDYT?

Ponywka commented 2 years ago

Good idea, but I have this problem already 1 year (no pictures on Bandcamp). And yep, it is blocked: https://isitblockedinrussia.com/?host=f4.bcbits.com

Ezwen commented 2 years ago

Here is a new test build with a new option --no-covers-single-track : https://framagit.org/Ezwen/bandcamp-collection-downloader/-/jobs/1521643/artifacts/raw/build/libs/bandcamp-collection-downloader.jar?inline=false

Can you try with this option to check if it fixes your problem? I'm not 100% sure I will keep it exactly this way for the next release, but at least we will be sure that this is the right track.

Ponywka commented 2 years ago

Yay! Now it works like I want. Thank you!

Ezwen commented 2 years ago

Fantastic, thanks for your patience and helping finding out the problem! Now the tool should be a little more russia-proof :grin: . I'll probably keep the option like this for the next release, but just in case I decide to rename it, please do check the release notes when/if you will download the next stable version.