search

LBBO / netflix-migrate

A command-line tool to migrate data to and from Netflix profiles
MIT License
266 stars 32 forks source link

It's not setting the viewing history #31

Open persocon opened 5 years ago

persocon commented 5 years ago

I can see that the history is being saved when I export but not being imported

lalmachado commented 5 years ago

@persocon currently, the import function is only able to import the rating history, but that will hopefully change soon. However, you now already have your data and once the functionality is added you will be able to import your old viewing history too, even if you don't have any access to the old account anymore.

rdeforest commented 5 years ago

Anything we can do to speed things along? Do you have a Patreon we can contribute to or something?

LBBO commented 5 years ago

Thank you for your interest in this project and for offering your help! Sadly, what is holding up the progress is not a lack of money but rather my lack of time. However, I do have a plan for moving things along, that I am slowly but seadily working on:

  1. Refactor node-netflix2. netflix-migrate just provides a CLI interface for node-netflix2, which then actually performs all the necessary requests. IMO, the code is kind of a mess and I would like to move it to ES6 etc. Thas is what I'm currently working on.
  2. Find a "quick and dirty" way to implement this feature. My first idea would be to visit the netflix page and pretend to watch a video without actually downloading the video (in order to not look like a DOS attack). A good way to help would be to figure out, if and how exactly that would be possible.
  3. Figure out how to send just the API calls. Last time I checked, they were encrypted and I pretty much gave up there because I had more important things to do. The plan would be to figure out where on netflix' website the encryption for this request happens, and hopefully figure out what the data looks like before encryption and how exactly it's encrypted.

If you (or anyone else, for that matter) would like to contribute to any of those steps, please feel free to do so! I'm also welcome to new ideas on how to tackle this matter.

zamarax commented 4 years ago

Any update on this feature or has it been abandoned?

LBBO commented 4 years ago

Nope, I'm sorry to say there has been no update so far. There has been some more research into the API calls but I wasn't able to figure out anything important. Currently, it looks like the quick and dirty version is the way to go, but I won't have the time to implement it until at least february.

I'm really sorry that this project is making such little progress. If anybody really needs this feature urgently, please feel free to create a pull request!

zamarax commented 4 years ago

Nope, I'm sorry to say there has been no update so far. There has been some more research into the API calls but I wasn't able to figure out anything important. Currently, it looks like the quick and dirty version is the way to go, but I won't have the time to implement it until at least february.

I'm really sorry that this project is making such little progress. If anybody really needs this feature urgently, please feel free to create a pull request!

No worries, thanks for the update!

dlong500 commented 4 years ago

@LBBO have you seen any evidence of an undocumented way to set the watched status without actually streaming the video? I know you haven't figured out how to send the encrypted calls, but was just curious if you had found anything that led you to believe this might be possible without actually streaming the video.

LBBO commented 4 years ago

No, I haven't seen any explicit evidence, but I strongly assume it is possible.

When I record my network traffic while manipulating the video progress (seeking, play, pause) there are generally a few different kinds of requests that are fired:

  1. The video is buffered. Those requests could be responsible for updating the viewing history, but since Netflix preloads a portion of the video, I don't think that's the case.
  2. Requests to [...]/personalization/cl2: these requests contain a ton of information in a JSON object. However, that object has a property type: 'CompactConsolidatedLoggingEnvelope', so this information might just be logged and that's it. Manipulating those requests might be my next attempt at figuring this out, though.
  3. Requests to [...]/router. Those are the encrypted requests I already talked about.
  4. Requests to [...]/pathEvaluator. I really don't think they are relevant to this feature because I don't see them transporting any relevant information.

Those last two types aren't sent after I seek to a specific video position, so I assume they at least aren't the only way of manipulating the viewing history (if they even do that at all). The only requests that are fired while watching a video (that has previously been buffered enough to not have to load any portion of the video while playing) are those to [...]/cl2, so that might be the solution.

If anyone wants to look into this some more: feel free! Otherwise, it might take me a while.

allysanx commented 3 years ago

here's a simple browser-based example for e.g. Chrome:

import.html ```js

Netflix Importer

```
  1. export your file as netflixData.js in the same folder as import.html
  2. add let _netflixData = to the front of netflixData.js
  3. ensure you have the right profile selected
  4. open import.html in Chrome or other browsers that might work
danypacf commented 3 years ago

here's a simple browser-based example for e.g. Chrome:

import.html

  1. export your file as netflixData.js in the same folder as import.html
  2. add let _netflixData = to the front of netflixData.js
  3. ensure you have the right profile selected
  4. open import.html in Chrome or other browsers that might work

You have to open the netflixData.js file and paste this let _netflixData = text at the beginning.

Anyway, this method does not work the way I want it to. It should work from the old series to the new series, not from the most recent ones. That is, from below upwards.

Redsandro commented 3 years ago

It should work from the old series to the new series, not from the most recent ones. That is, from below upwards.

- const movies = _netflixData.viewingHistory
+ const movies = _netflixData.viewingHistory.reverse()
AndiZandi commented 2 years ago

The solution from allysanx above works beautifully as of writing this. Make sure to also follow the tips from danypacf and redsandro above (adjusting generated json file to make it a valid and usable js file AND add the reverse() to the import.html)

But I have some tips for future users/visitors:

a) make sure that ur browser isnt blocking the opening of new tabs (press allow or something on the popup from ur browser) b) make sure to allow autoplay of audio AND video (in firefox -> settings -> search for autoplay -> adjust to allow for audio AND video c) based on the amount of entries in ur exported json (visible as the tab title of the opened import.html, e.g. 0112/1614) maybe adjust the watchseconds variable in import.html before starting the session. In my case there were 1614 entries, each 20 seconds = ~9 hours. I reduced the watchseconds to 10, and it still worked perfectly fine, and i reduced the needed time from 9 to 4.5hours. (If your pc is fast enough to open the new tab and start the netflix video in an appropiate amount of time to actually get it playing a bit before the next video is started. So if ur pc is slow, stick to higher watchseconds.) d) open the view history of the new profile you are importing to and make sure that the watched stuff is getting added, to see if everything is working as intended

Usernameistakensad commented 2 years ago

The solution from allysanx above works beautifully as of writing this. Make sure to also follow the tips from danypacf and redsandro above (adjusting generated json file to make it a valid and usable js file AND add the reverse() to the import.html)

But I have some tips for future users/visitors:

a) make sure that ur browser isnt blocking the opening of new tabs (press allow or something on the popup from ur browser) b) make sure to allow autoplay of audio AND video (in firefox -> settings -> search for autoplay -> adjust to allow for audio AND video c) based on the amount of entries in ur exported json (visible as the tab title of the opened import.html, e.g. 0112/1614) maybe adjust the watchseconds variable in import.html before starting the session. In my case there were 1614 entries, each 20 seconds = ~9 hours. I reduced the watchseconds to 10, and it still worked perfectly fine, and i reduced the needed time from 9 to 4.5hours. (If your pc is fast enough to open the new tab and start the netflix video in an appropiate amount of time to actually get it playing a bit before the next video is started. So if ur pc is slow, stick to higher watchseconds.) d) open the view history of the new profile you are importing to and make sure that the watched stuff is getting added, to see if everything is working as intended

Hey! If you got time could you please make a video guide on this?

TechRemarker commented 2 years ago

Got this working after taking in all the tidbit notes people mentioned here, except of my 3,700 shows, Netflix only shows continue watching for about 9 of them, most which seem to be series. So seems 20 seconds is not longer enough any more to trigger it's been watched. I did a couple quick tests and when I got to 1 minute and 35 seconds it counted it. I'll have to try more to find the exact amount, and perhaps it various based on the length of the movie/show, or varies based on the length of intro credits for the show? Setting a much longer time will resolve that but if you have a large number like mine it would take weeks. Or perhaps could update it to have it open new tabs every few seconds have lots of tabs open and I could manually just close them after a couple of minutes, unless if multiple videos are playing it doesn't work. Need to do more testing.

Redsandro commented 2 years ago

have it open new tabs every few seconds have lots of tabs open

It probably won't work; you can only have 2 (or 4 with Netflix Premium) tabs open at the same time.

I'm not subscribed to Netflix anymore, but someone should try to use their legal right under GDPR Article 16: Right to rectification [which is] the right to have incomplete personal data completed, including by means of providing a supplementary statement.

Just email them your watch list and demand they supplement the watch list in your account. I'm curious what would happen.

Best case scenario they get so tired from manually doing those imports that they will automate it so that Netflix will finally have proper import/export functionality.

LBBO commented 1 year ago

Hey everyone! Sorry for not helping out with this issue for so long, I was just really busy and, quite frankly, fed up with playing catchup with changes to Netflix's API or DOM structure or whatever every couple of months.

However, there's some good news! In case anyone hasn't heard, Netflix now officially supports account migration, including viewing history! For more details, see their official documentation. Since that feature implements exactly what this project was aiming to do, I will officially archive this repository now (see #73). Feel free to reach out if you have any further questions!