Closed Ponywka closed 2 years ago
Interesting, so what you are saying is that even if the downloaded file is complete (as shown by the MD5 hashes), it keeps retrying to download it again?
Strange. Can you easily reproduce the bug?
Interesting, so what you are saying is that even if the downloaded file is complete (as shown by the MD5 hashes), it keeps retrying to download it again?
Yep
Can you easily reproduce the bug?
https://20101010.xyz/public/bandcamp-collection-downloader-2021-11-30-19-58-12.mp4
I bought the track for 50 cents just to test, and everything went fine. So it's not specific to this item, and is something related to your setup, I suppose. I wonder what this could be.
Interesting... 🤔
OS: Arch Linux Java version:
openjdk version "17" 2021-09-14
OpenJDK Runtime Environment (build 17+35)
OpenJDK 64-Bit Server VM (build 17+35, mixed mode)
Same problem https://asciinema.org/a/Cne8yYoxpQc8LhPqC45CmiNQI
I've added an option --debug
to display the complete error stack trace, and a test build with this new feature can be downloaded here: https://framagit.org/Ezwen/bandcamp-collection-downloader/-/jobs/1521019/artifacts/raw/build/libs/bandcamp-collection-downloader.jar?inline=false
Can you try again with this test build, and report back here the complete error stack trace?
Error while trying: "java.net.SocketException: Connection reset".
java.net.SocketException: Connection reset
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421)
at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
at bandcampcollectiondownloader.util.RealIO.downloadFile(RealIO.kt:44)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadRelease(BandcampCollectionDownloader.kt:303)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.access$downloadRelease(BandcampCollectionDownloader.kt:17)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:248)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:245)
at bandcampcollectiondownloader.util.Util.retry(Util.kt:66)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:245)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:119)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:132)
at bandcampcollectiondownloader.cli.Command.call(Command.kt:94)
at bandcampcollectiondownloader.cli.Command.call(Command.kt:14)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)
Also in Arch Linux: https://20101010.xyz/public/bcd-2021-12-08-14-59-33.mp4 00:00 - System info 00:32 - Start downloading & Crashes when downloading singles 01:40 - Downloading albums without chashes
Error while trying: "java.net.SocketException: Connection reset".
java.net.SocketException: Connection reset
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421)
at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
at bandcampcollectiondownloader.util.RealIO.downloadFile(RealIO.kt:44)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadRelease(BandcampCollectionDownloader.kt:303)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.access$downloadRelease(BandcampCollectionDownloader.kt:17)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:248)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:245)
at bandcampcollectiondownloader.util.Util.retry(Util.kt:66)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:245)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:119)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:132)
at bandcampcollectiondownloader.cli.Command.call(Command.kt:94)
at bandcampcollectiondownloader.cli.Command.call(Command.kt:14)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)
Could not download item: Could not perform task after 0 retries.
bandcampcollectiondownloader.core.BandCampDownloaderError: Could not perform task after 0 retries.
at bandcampcollectiondownloader.util.Util.retry(Util.kt:75)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:245)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:119)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:132)
at bandcampcollectiondownloader.cli.Command.call(Command.kt:94)
at bandcampcollectiondownloader.cli.Command.call(Command.kt:14)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)
Great thanks. So the problem strangely arises when trying to download the cover image for the track.
I've added yet more debug printing, to have a better look at what is the URL of the cover file it tries do download. Can you try again with this new test build here, using --debug
once more: https://framagit.org/Ezwen/bandcamp-collection-downloader/-/jobs/1521424/artifacts/raw/build/libs/bandcamp-collection-downloader.jar?inline=false
As I understand it, in Russia (at least at the Internet provider Rostelecom and Tele2), the domain of pictures at Bandcamp is blocked. This makes sense because usually the Bandcamp page looks like this for me:
Managing item 1/93
Found release "Pony Girl (MirroredReality Remix)" (2018) by MirroredReality (Bandcamp ID: r152688695).
Starting the download of "Pony Girl (MirroredReality Remix)" (2018) by MirroredReality.
[DEBUG] Starting download of: https://p4.bcbits.com/download/track/12c24612d74aee5a3ae64fa9a7c39976d/mp3-320/3850187924?id=3850187924&sig=2f115bb4dd67221746821bd767fd2ea4&sitem_id=152688695&token=1639579443_aa3ad6ca4549c315d25ad2df66e7ac3f55b4fcd4
Progress: 0.00 %
...
Progress: 99.99 %
[DEBUG] This is a single track release: fetching the cover
[DEBUG] Starting download of: https://f4.bcbits.com/img/a637921479_10
Error while trying: "java.net.SocketException: Connection reset".
[DEBUG] [DEBUG]java.net.SocketException: Connection reset
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421)
at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
at bandcampcollectiondownloader.util.RealIO.downloadFile(RealIO.kt:43)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadRelease(BandcampCollectionDownloader.kt:315)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.access$downloadRelease(BandcampCollectionDownloader.kt:18)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:258)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:255)
at bandcampcollectiondownloader.util.Util.retry(Util.kt:35)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:255)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:131)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:142)
at bandcampcollectiondownloader.cli.Command.call(Command.kt:96)
at bandcampcollectiondownloader.cli.Command.call(Command.kt:13)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)
Could not download item: Could not perform task after 0 retries.
[DEBUG] [DEBUG]bandcampcollectiondownloader.core.BandCampDownloaderError: Could not perform task after 0 retries.
at bandcampcollectiondownloader.util.Util.retry(Util.kt:42)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:255)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:131)
at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:142)
at bandcampcollectiondownloader.cli.Command.call(Command.kt:96)
at bandcampcollectiondownloader.cli.Command.call(Command.kt:13)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)
Ah I see, I did not know that! Well at least now we have nailed the problem :)
Since it will be difficult to detect programmatically whether the problem is a (temporary) real network error, or a (permanent) proper blocking from the ISP, instead we can consider adding a new flag in the app to choose to not download the covers for single-track releases. WDYT?
Good idea, but I have this problem already 1 year (no pictures on Bandcamp). And yep, it is blocked: https://isitblockedinrussia.com/?host=f4.bcbits.com
Here is a new test build with a new option --no-covers-single-track
: https://framagit.org/Ezwen/bandcamp-collection-downloader/-/jobs/1521643/artifacts/raw/build/libs/bandcamp-collection-downloader.jar?inline=false
Can you try with this option to check if it fixes your problem? I'm not 100% sure I will keep it exactly this way for the next release, but at least we will be sure that this is the right track.
Yay! Now it works like I want. Thank you!
Fantastic, thanks for your patience and helping finding out the problem! Now the tool should be a little more russia-proof :grin: . I'll probably keep the option like this for the next release, but just in case I decide to rename it, please do check the release notes when/if you will download the next stable version.
OS: Windows 11 21H2 (22000.194) Java version: OpenJDK 11.0.2 / OpenJDK 17.0.1 / Oracle JDK 17 CLI command:
java -jar bandcamp-collection-downloader.jar -j 1 -r 100 -f mp3-320 -d bandcamp320 ponywka
Album in queue: https://mirroredreality.bandcamp.com/track/pony-girl-mirroredreality-remixWhen the file has been downloaded completely, the program gives an error and tries to download the file again.