search

SHOEGAZEssb / Last.fm-Scrubbler-WPF

Manual Last.fm scrobbling for when a service (or you!) failed to scrobble.
GNU General Public License v3.0
420 stars 50 forks source link

Import YouTube history to Last.fm? #87

Closed davidmickelsen closed 5 years ago

davidmickelsen commented 5 years ago

Hi Tim, is it possible to use the Last.fm Scrubbler to import YouTube history to Last.fm? Google gives me access to my full YouTube history, but I can't figure out how to get it into the Scrubbler.

SHOEGAZEssb commented 5 years ago

Hi, can you provide me with some more info? Hwere did you get the history, what format is it in etc?

davidmickelsen commented 5 years ago

Hi Tim, thanks for your reply. Since I posted my original question, I actually found a method that I think may work. When I posted the question, I was just viewing my YouTube history in the "History" tab of the site, which is more or less in a playlist format, so I could see it all there, but didn't really have any way of accessing the data other than playing back the entire history video by video and scrobbling the tracks one by one, which was certainly not viable. However, last night I found a way to export my YouTube history to a JSON file, at https://takeout.google.com/settings/takeout, using this method:

https://www.youtube.com/watch?v=zlzzO1e6dws

I haven't received the file yet, as apparently it can take several days for the archive to be created depending on the file size (I'm guessing in my case this is going to be quite a large file). Once the archive is ready, I'll receive an email with a link to download it. If I have any further questions at that point, I'll let you know; if you have any other input in the meantime, that would be welcome as well.

Thanks and best regards, David

On Sat, Aug 10, 2019 at 4:08 AM Tim Stadler notifications@github.com wrote:

Hi, can you provide me with some more info? Hwere did you get the history, what format is it in etc?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/coczero/Last.fm-Scrubbler-WPF/issues/87?email_source=notifications&email_token=AM3VIAUWOWPESRH2HW6JKLDQDZSNNA5CNFSM4IKYFLZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4AHXAQ#issuecomment-520125314, or mute the thread https://github.com/notifications/unsubscribe-auth/AM3VIAS3Z34EBLB2LTQBZATQDZSNNANCNFSM4IKYFLZA .

SHOEGAZEssb commented 5 years ago

Could you provide me with an excerpt of the json file when you get it?

davidmickelsen commented 5 years ago

Hi Tim, thanks again for your reply. I received the file last night; it didn't end up being too massive after all, just 3.4 MB. Below is an exerpt:

},{ "header": "YouTube", "title": "Watched King Gizzard \u0026 The Lizard Wizard - Nonagon Infinity", "titleUrl": "https://www.youtube.com/watch?v\u003dudyYluSPBbc", "subtitles": [{ "name": "t3h Sn34kyN1nj4", "url": "https://www.youtube.com/channel/UCSHZyljV6HdiTPHXH03Ebpg" }], "time": "2019-08-05T03:34:00.633Z", "products": ["YouTube"] },{ "header": "YouTube", "title": "Watched King Gizzard and the Lizard Wizard - Billabong Valley", "titleUrl": "https://www.youtube.com/watch?v\u003dIt7LM0bPi48", "subtitles": [{ "name": "Quentin Mackenzie", "url": "https://www.youtube.com/channel/UCYlP3CSGtoChL9mta4ajG1A" }], "time": "2019-08-05T00:04:43.555Z", "products": ["YouTube"] },{ "header": "YouTube", "title": "Watched King Gizzard \u0026 The Lizard Wizard - Gamma Knife", "titleUrl": "https://www.youtube.com/watch?v\u003dnC7ii3Ir-no", "subtitles": [{ "name": "Flightless Records", "url": "https://www.youtube.com/channel/UCmbnRwU2AFYuDCcXZFpQMNQ" }], "time": "2019-08-04T20:20:04.259Z", "products": ["YouTube"] },{ "header": "YouTube", "title": "Watched Κing Gizzard and The Lizard Wizard - Big Fig Wasp", "titleUrl": "https://www.youtube.com/watch?v\u003deCkMxcEf7jI", "subtitles": [{ "name": "Michael Petrelis", "url": "https://www.youtube.com/channel/UCfn1jYfXPuMYTqUmm2TYaBg" }], "time": "2019-08-04T20:15:02.946Z", "products": ["YouTube"]

I can send you the file itself as well, if you'd like to take a look.

Do you think it will be possible to make this work?

Best, David

SHOEGAZEssb commented 5 years ago

Hmm, that is really a pain to parse, I don't think the effort would be worth it to be honest :(

davidmickelsen commented 5 years ago

Hi Tim,

Thanks for your reply. I was afraid of that. There's no way to use "Find and Replace" or something to make it workable?

Best, David

SHOEGAZEssb commented 5 years ago

Hi David,

the parsing for this is quite complex, since you dont have Artist and track title in seperate fields. If they were it would be easy, but for this case there would be many special cases that'd need to be considered and the whole thing would take a long time to implement.

davidmickelsen commented 5 years ago

Ah, I see. Yeah, that makes sense. Well, thanks for your help anyway.

Just so I can see it, could you show me a few lines of what an ideally-parsed JSON file would look like for loading into the Scrubbler?

SHOEGAZEssb commented 5 years ago

Something like this:

[ { "artistName" : "King Gizzard & The Lizard Wizard", "trackName" : "Robot Stop", "albumName": "Nanogon Infinity", "time" : "2018-02-28 00:30:44" }, { "artistName" : "King Gizzard & The Lizard Wizard", "trackName" : "Big Fig Wasp", "albumName": "Nanogon Infinity", "time" : "2018-02-28 00:30:44" }, ]

You could also do csv which is basically just King Gizzard & The Lizard Wizard, Robot Stop, Nanogon Infinity, 2018-02-28 00:30:44

davidmickelsen commented 5 years ago

Thanks, Tim.

Best, David

MSOB7YY commented 2 years ago

Sooo for anyone wanting this

here is a workaround, just follow the simple steps :)

  1. extract youtube history in json format using takeout.google.com
  2. open the downloaded file using any text editor (if ur on android, i recommend u NMM Editor as it can deal with huge files without problems.)
  3. search for .*https://www.youtube.com.*\r?\n and apply RegEx then hit replace all (this will remove all the lines containing youtube links)
  4. go to JSON-to-CSV (or any converter app), upload your file and tweak these options in second step: • disable Include header in first row • Output Field Separator: Other ", " • date timestamp: YYYY-MM-DD hh:mm:ss [this hasn't worked with me tho :( but since last.fm doesn't even accept scrobbles older than 2 weeks, then i guess it ain't a problem]
  5. Press Convert then Download Result
  6. open the downloaded file
  7. search for the following and replace all with nothing, so that they be removed: YouTube", "Watched [note: empty space at the end] , "", " , "YouTube", "YouTube watch history" - (lyrics) or (lyrics) [empty space at the start]
  8. replace - [empty space at the start and the end] with ", " [this separates the artist and title as last.fm requires them separated]
  9. import the csv file into the app or universal scrobbler
  10. Done :D

IMPORTANT NOTES:

• u can try with few entries first to check that everything's working well. • the uploader will be submitted as the Album name • YouTube History DOESN'T include the rewatches done using the replay button, so you have to take that in mind, u can increase the number maybe twice or depending on how many times u think u have replayed it. • as u all know, submitting scrobbles in bulk would mess up the weekly and monthly records, they will no longer be accurate, so make sure u accept that before proceeding.

and thats it, happy scrobbling :)

menggatot commented 1 year ago

Sooo for anyone wanting this

here is a workaround, just follow the simple steps :)

  1. extract youtube history in json format using takeout.google.com
  2. open the downloaded file using any text editor (if ur on android, i recommend u NMM Editor as it can deal with huge files without problems.)
  3. search for .*https://www.youtube.com.*\r?\n and apply RegEx then hit replace all (this will remove all the lines containing youtube links)
  4. go to JSON-to-CSV (or any converter app), upload your file and tweak these options in second step: • disable Include header in first row • Output Field Separator: Other ", " • date timestamp: YYYY-MM-DD hh:mm:ss [this hasn't worked with me tho :( but since last.fm doesn't even accept scrobbles older than 2 weeks, then i guess it ain't a problem]
  5. Press Convert then Download Result
  6. open the downloaded file
  7. search for the following and replace all with nothing, so that they be removed: YouTube", "Watched [note: empty space at the end] , "", " , "YouTube", "YouTube watch history" - (lyrics) or (lyrics) [empty space at the start]
  8. replace - [empty space at the start and the end] with ", " [this separates the artist and title as last.fm requires them separated]
  9. import the csv file into the app or universal scrobbler
  10. Done :D

IMPORTANT NOTES:

• u can try with few entries first to check that everything's working well. • the uploader will be submitted as the Album name • YouTube History DOESN'T include the rewatches done using the replay button, so you have to take that in mind, u can increase the number maybe twice or depending on how many times u think u have replayed it. • as u all know, submitting scrobbles in bulk would mess up the weekly and monthly records, they will no longer be accurate, so make sure u accept that before proceeding.

and thats it, happy scrobbling :)

thank you for the guide.

Hi Tim, thanks for your reply. Since I posted my original question, I actually found a method that I think may work. When I posted the question, I was just viewing my YouTube history in the "History" tab of the site, which is more or less in a playlist format, so I could see it all there, but didn't really have any way of accessing the data other than playing back the entire history video by video and scrobbling the tracks one by one, which was certainly not viable. However, last night I found a way to export my YouTube history to a JSON file, at https://takeout.google.com/settings/takeout, using this method: https://www.youtube.com/watch?v=zlzzO1e6dws I haven't received the file yet, as apparently it can take several days for the archive to be created depending on the file size (I'm guessing in my case this is going to be quite a large file). Once the archive is ready, I'll receive an email with a link to download it. If I have any further questions at that point, I'll let you know; if you have any other input in the meantime, that would be welcome as well. Thanks and best regards, David … On Sat, Aug 10, 2019 at 4:08 AM Tim Stadler @.***> wrote: Hi, can you provide me with some more info? Hwere did you get the history, what format is it in etc? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#87>, or mute the thread https://github.com/notifications/unsubscribe-auth/AM3VIAS3Z34EBLB2LTQBZATQDZSNNANCNFSM4IKYFLZA .

thank you for the youtube link and the google takeout link.

here I'm writing a python script to convert the youtube-history.html file we get from google takeout into a usable CSV file that can be parse into universal scrobbler