Scrobble to .txt file

[Bujiraso]

Is your feature request related to a problem, or particular use cases? Please describe. I'd like to be able to easily scan over my all-time historic listen data, but all networked scrobbling solutions such as Last.fm, Libre.fm, Listenbrainz, seem to have paginated data and do not necessarily make it easy to bulk export all listen data (e.g. last.fm doesn't have bulk export (you need to use various web pages), Libre.fm has manual email based export).

I love that Pano supports custom servers, and tried looking into running my own custom GNU FM or Listenbrainz, but both are exceptionally heavy for what I'm looking for.

Describe the solution you'd like I'd like Pano Scrobbler to log its scrobbles to a flat file. Any format is good but I already use the the last.fm style seen here if that's of interest.

Additional info

• I would imagine it would go under Services in the Configure page and let me choose a folder, that way I can sync it trivially via SyncThing. File name can be user or application decided -- not important.
• No desire for importing. This is just to permit downstream apps and users to be able to more easily read the scrobble data Pano Scrobbler has seen (i.e. export only).
• No concern for any higher precision or accuracy than is already used with last.fm -- I don't see incorrect scrobbles so I'd just be looking for it to effectively log to a file.

Cheers!

[Bujiraso]

I've made a small amount of headway on this today. I got the app building, read over the code to see where to splice in the new service, and did some light front-end work.

The Onboarding login flow is currently a UX :cold_face:-moment where the user has to type a path. It'll take more work to get a file picker.

Configure looks like this

[kawaiiDango]

I think this eventually becomes a big undertaking and much more than just logging to a text file. The UI has screens for scrobble history, charts and track info where the user would expect to see their locally stored history, stats and scrobble counts or at least something. A text file is no longer suitable for that. One would use a local database, define all the data structures and schemas and maybe also have logging to a text file in parallel. Then the UI has options to love, edit or delete a track and the user would expect the text file to be edited by those actions. Manually going through the text file containing 1000's of scrobbles isn't a common use case. So the user may want the data in json or csv instead. This is open ended.

[Bujiraso]

I'm definitely seeing this complexity while working today and agree! :worried:

I wager it's a lot easier to try and splice a Stuff.log to a scrobble file somewhere before the Scrobbler type is determined instead of wiring it in like this. The user doesn't necessarily even need to file pick it if it's in a sane enough place (e.g. /sdcard/panoscrobbler)

Please feel open to give me tips or take this one on to make it very bite-sized -- I'm definitely looking for the minimal viable to get a small output file on the SD card that can be synced, as I agree with you that the use-case isn't large enough to want all these changes and functionality.

I'm mostly just looking for a flat text log which gets appended to with new scrobbles that sits outside the hidden app files! :yum:

[kawaiiDango]

It depends on why you specifically want a text file and not a nice UI. You can take a look at https://github.com/krateng/maloja which is very easy to self host (check their demo server), is compatible with Pano Scrobbler, makes your entire data searchable, so you are not limited by pagination, and also has an option to export everything to a text file.

[Bujiraso]

It's actually just for archival and longevity purposes without the need to run an entirely separate server.

Over the past 20 years I've gone through the now-defunct iTunes (with its last-play-only internal DB), libre.fm seems to be waning, last.fm, as well, honestly.

Correspondingly, I've gotten a few other clients logging scrobbles properly to a tab-delimited or comma-separated format. Pano Scrobbler is the one last spot where I love the client (it lets me scrobble anything on my phone), but I still need to regularly interface with a server (last.fm) instead of just setting up a one-time set-and-forget sync for a text file.

I use the following scrobble info as an example of my other data.

#ARTIST #ALBUM #TITLE #TRACKNUM #LENGTH #RATING #TIMESTAMP #MUSICBRAINZ_TRACKID
The Mars Volta  De-Loused in the Comatorium Son et lumière  1   95  L   1689620654  fadaa416-b683-4eda-947f-dcb6d72e07aa
The Mars Volta  De-Loused in the Comatorium Inertiatic ESP  2   263 L   1689620750  aaaf7e32-a6c6-4143-a873-10bd9d00412b

When I want to view the data I use LibreOffice Calc which can open it as a CSV nicely and I can perform whichever searches or aggregate functions I like in there. So, truly I don't want or need any UI.

I hope that clarifies the interest :slightly_smiling_face:

[kawaiiDango]

I am going to implement this, the real question is, if the user has chosen to scrobble only to file, what does the user see on the main screen? It shouldn't be blank.

[Bujiraso]

:tada: good news to hear!

A great question I was wondering, too, when proto-typing. Got any initial thoughts?

I was thinking the standard option is to use either the same metadata providers already in use, or implement others (discogs, musicbrainz, freedb, etc.) and read from the file and present about the same thing that already shows for today. If the file is clean, it would be like a new user who just signed up to last.fm -- modeling what pano-scrobbler does in the case for a cleanly minted last.fm account (or one that has only been around for a day or two with a few scrobbles) could set the tone here.

As to reading the file for historical input, it is luckily a machine-parsing-first file format, and the information should generally be expected to come straight from good sources (the player, pano-scrobbler), so it could be fairly tersely edge-guarded against errors introduced from tampering on the external side by a user who might be mucking around with files more easily than they do app data guarded by the Android system. Sort of a "hey, I tried, but you mucked up the format" or just show the lines that parse, all seem acceptable as it's not critical to do anything right if the file format is borked.

[kawaiiDango]

Reading the file and displaying data especially when the file has become very large can be very slow and demanding. When you choose a file from the system document picker, you are not actually dealing with a file. You are dealing with what Android calls a document. A lot of file operations are unavailable for a document.

A relevant operation involves seeking to the end of the file and reading it backwards line by line to fetch the last n scrobbles to show to the user. This is possible with a file but not with a document, unless you read the entire file from the beginning and take the last n lines every time.

The other way is to store the file in the app's private storage where you cannot directly access it. So you cannot sync it with Syncthing and stuff. That file can then be opened as a File(), and can be read backwards fast. The app would have a share button which would temporarily allow sharing/opening that file with other apps. I don't think this is what you are looking for.

Writing to the file in append mode should be fast, so I will only be supporting line appendable formats like CSV and JSONL (not JSON)

[Bujiraso]

Sounds reasonable. The tab-separated value that's default to scrobbling (shown above) allows easy append-only transactions.

It's still perhaps a question as to what to show if the file scrobbling is the only thing available, but I'm not highly prizing the rich historical views of pano-scrobbler for this use case.