har-nick / troupetent

A Kotlin Multiplatform Bandcamp client
7 stars 1 forks source link

re: A separate library containing just the API client #3

Closed YoshiRulz closed 1 year ago

YoshiRulz commented 1 year ago

Hi o/ I was about to start on a Bandcamp scraper in Kotlin and figured I'd search for one first. Is the readme accurate w.r.t. how many (few) features are working? I'm not sure there are features beyond search and artist/album enumeration, but those are the only ones I'm interested in atm. I've already hacked together a function to get stream URIs.

At any rate I'd be happy to contribute if you could publish a separate artifact without the GUI or anything. My experience with Kotlin/Android is very limited, I mostly use Kotlin/JS in the browser and Kotlin/JVM on desktop.

I take it you've reverse-engineered their internal API, but I'm interested in whether you're parsing HTML anywhere, and if so, how. For my project I've ported jsoup to Kotlin/JVM and have made good headway on swapping out the parts tied to the JVM. But recently I'm considering starting over and instead making an abstraction over jsoup, DOMParser.parseFromString, and lexbor or something for Native.

har-nick commented 1 year ago

Hi! Thanks for your questions! I'll answer them piecemeal just for simplicity's sake.

I'm not sure there are features beyond search and artist/album enumeration

In the context of Troupetent or the requirements of your scraper? If the former, I intend for Troupetent to be a full-fledged media player, if my sanity allows. Sorry if I've misunderstood your question here.

At any rate I'd be happy to contribute if you could publish a separate artifact without the GUI or anything.

Publishing Troupetent's functions as a library has been in the back of my mind for a while, and I'll probably do it over the next couple of weeks, if only to decouple it from Troupetent itself.

I take it you've reverse-engineered their internal API

I did try doing this with their Android application, but I had concerns with regards to copyright and any legal issues that may arise from it. I also came across various obfuscated token generation methods that concatenate at the very end to be added to at least the login request. It seemed like too much work for what could be dubious gains.

At the moment I'm using Wireshark to record network requests from a desktop browser to their API and replicating what's necessary where possible.

I'm interested in whether you're parsing HTML anywhere

I'm trying to avoid parsing HTML as much as possible, so at the moment there are only two contexts this occurs - once when getting a user's profile picture (the ID of it is random on the client-side), and (in future) parsing download links.

As a side note, anything of note I find usually gets thrown in one of the wiki pages in that repo. It's not true documentation, more like blog posts on the fun stuff I find with Bandcamp's service.

YoshiRulz commented 1 year ago

I'm not sure there are features beyond search and artist/album enumeration

In the context of Troupetent or the requirements of your scraper? If the former, I intend for Troupetent to be a full-fledged media player, if my sanity allows. Sorry if I've misunderstood your question here.

No worries, I meant features of the Bandcamp web app. It's rather bare-bones compared to YouTube or Spotify.

[...] I'll probably do it over the next couple of weeks [...]

I'm looking forward to it.

edit 2023-04-21: Just remembered NewPipe can interface with Bandcamp too. https://github.com/TeamNewPipe/NewPipeExtractor/tree/dev/extractor/src/main/java/org/schabi/newpipe/extractor/services/bandcamp

har-nick commented 1 year ago

I'm a liiiitle late on this, but I've created a small library for this feature - BandKit.

YoshiRulz commented 1 year ago

Oh I ended up making my own based on my bespoke platform abstraction and kitchen sink libraries. I was just about done when my hard drive failed... Still, can't hurt to have 2, so I'll see if I can take any ideas from yours once I'm back up and running.