defvs / monsterutilities

Browse, stream and download Monstercat Songs
GNU General Public License v3.0
39 stars 6 forks source link

Downloading everything? #32

Open Chaphasilor opened 8 months ago

Chaphasilor commented 8 months ago

Hi! Thanks for providing this awesome tool :)

I've been playing around with the tool for a bit now, but can't seem to ever load all albums/tracks at once in the downloads tab. Only a small selection of the whole catalog is shown, but the selection is not consistent. The most I was ever able to select with smart select was ~300 items.
I've set the date filter to "before <tomorrow's date>", so that should include everything.

My current approach is to just download everything that is shown, then cut the CID string and paste it again to re-fetch releases, and then download the next batch (checking the option for skipping existing downloads, which doesn't seem to be reliable either).

Am I doing something wrong? Is there any way to tell which releases I'm still missing without manually checking?

defvs commented 8 months ago

Hi. I haven't been maintaining this tool very much recently. I'll have a peek.

Chaphasilor commented 8 months ago

That would be wonderful. Let me know if I can assist!

defvs commented 8 months ago

I'll possibly need some logs, but frankly it's been so long since I touched the software I don't even know the flags to get them anymore. Just give me a few days, maybe the issue also happens on my side.

xeruf commented 7 months ago

btw just chiming in as the original developer using it to also download everything back then I now have a qobuz subscription and download everything with https://github.com/vitiko98/qobuz-dl - a bit more expensive but access to a much wider catalog still at pristine quality unlike most services

Chaphasilor commented 7 months ago

Interesting, thanks for the tip! I think I've got everything now, at least I can't seem to find more items. Figured out the "Exclude already downloaded releases" option simply takes a few seconds to uncheck everything that's already there. I still don't get consistent listings, so maybe I'm still missing things, but there's no way to tell for me. In total, I have about 264 gigs now, which is definitely plenty! :D

EarlSven commented 7 months ago

The catalog tab seems to list everything (e.g. filtering by before 2015 shows all the compilations up to 020, and I assume all the singles etc..) just seems like the downloader tab is not getting all the same data somehow.

codicusmaximus commented 5 months ago

I have figured out that the reason for this is running out of heap space. If you launch the application with java -jar -Xmx2048m MonsterUtilities-dev273-b2c2c04.jar it will be able to obtain all info about all tracks.

--edit-- this proved to be incorrect, see below.

codicusmaximus commented 5 months ago

It worked once for me with my previous mentioned change, but, unfortunately it looks like it's more than just the memory -- The parallel connections is causing Monstercat to block or rate-limit. The "fiber" setting uses hundreds of connections at once. Even the "dialup" connection option uses 5.

I rebuilt the application with it forcing only 2 connections max, will see if this works:

        DIALUP(2, "Dial-up (100 kb/s)"),
        ADSL(2, "ADSL / 3G (10 Mb/s)"),
        CABLE(2, "Cable / 4G (100 Mb/s)"),
        FIBER(2, "Fiber (300+ Mb/s)");

--edit--: confirmed, now everything lists (memory change above not needed)

jako800 commented 1 month ago

It worked once for me with my previous mentioned change, but, unfortunately it looks like it's more than just the memory -- The parallel connections is causing Monstercat to block or rate-limit. The "fiber" setting uses hundreds of connections at once. Even the "dialup" connection option uses 5.

I rebuilt the application with it forcing only 2 connections max, will see if this works:

      DIALUP(2, "Dial-up (100 kb/s)"),
      ADSL(2, "ADSL / 3G (10 Mb/s)"),
      CABLE(2, "Cable / 4G (100 Mb/s)"),
      FIBER(2, "Fiber (300+ Mb/s)");

--edit--: confirmed, now everything lists (memory change above not needed)

Sorry to resurrect this topic. I know nothing about Java, how would I go about implementing this fix for myself?

defvs commented 1 month ago

@jako800 Clone the code (or download it through GitHub as a .zip), find these lines and apply the modification, then open a terminal / command prompt in the base folder and run ./gradlew.bat run. Should start the app. Note that you need the JDK (find it at adoptium.net)