search

damienhaynes / TraktRater

TraktRater is a tool written in C# to help users transfer user episode, show and movie user ratings and watchlists from multiple media database sites around the web.
655 stars 37 forks source link

Movielens support #118

Open MrDemocracy opened 4 years ago

MrDemocracy commented 4 years ago

Could you add support for Movielens?

I don't know if there's an api that could allow automatic syncing and Movielens doesn't have public profiles, so there's probably no way of doing web scraping without authentication, but manually importing the csv files is fine too. The activity logs csv can be used to get the time and date of when a rating or wishlist entry was added. However if you change a rating, the log will contain multiple times/dates for the same movie so that might be a bit difficult to implement. I also noticed that a few ratings were missing from the activity logs (4/198), so the time/date can't always be grabbed from the logs. Movielens has a 5-star rating system that allows half-star rating.

Ratings:

movie_id,imdb_id,tmdb_id,rating,average_rating,title
1,0114709,862,3.5,3.8906,Toy Story (1995)

Wishlist:

movie_id,imdb_id,tmdb_id,average_rating,title
1,0114709,862,3.8906,Toy Story (1995)

Activity logs:

datetime,login_id,action_type,log_json
2020-08-23 18:30:00.0,Tlqr4ty,rating,"{""movieId"":1,""rating"":3.5,""action"":""ADD"",""pred"":3.3187699801952673}"
2020-08-23 18:30:05.0,Tlqr4ty,user-list,"{""listId"":1,""movieId"":1,""action"":""ADD""}"

Thank you for making this tool, I especially love the Trakt maintenance feature 😃

damienhaynes commented 4 years ago

Hi @MrDemocracy, I've signed up and will take a look.

damienhaynes commented 4 years ago

Hi @MrDemocracy,

I have started adding support for this and have attached a build to test. Note that it only supports the 'Ratings' and 'Activity' CSV files, ignore the Tags and Wishlist textboxes for now.

Providing the Activity log is optional and when provided I use it to get the Date/Time a movie was rated (which is also used for the watch date when inserting into your watched history if that setting is enabled in options).

TraktRater_v2.3.10.0.exe.zip

MrDemocracy commented 4 years ago

Thank for taking a look at this :) I tried syncing from Movielens, but it instantly fails without syncing anything to trakt. Here's the logs: logs.zip There's two log files because I tried with and without the Activity CSV.

damienhaynes commented 4 years ago

Thanks @MrDemocracy, it must be something specific to your CSV files. Is there any chance I can get a copy? Im working with a pretty small data set so I may have missed a few things.

damienhaynes commented 4 years ago

I suspect your have a null MovieID or some value not expected.

MrDemocracy commented 4 years ago

movielens-ratings.csv: https://pastebin.com/raw/2NAVzEXM Not seeing "null" anywhere, but there are a few with / and " that could be causing issues.

damienhaynes commented 4 years ago

You're right, your ratings csv is fine...I just tested it and parsed fine: 2020-10-21 12:01:03.651 [INFO] [][01]: Found 199 ratings in Movie Lens Rating CSV file

damienhaynes commented 4 years ago

@MrDemocracy, what regional settings are you using? Possibly a number formatting issue.

damienhaynes commented 4 years ago

Here is an update with a possible fix (hopefully will log the error line this time if still a problem) TraktRater_v2.3.10.1.exe.zip

MrDemocracy commented 4 years ago

Awesome! 😃 That fixed the issue. All the ratings was synced to trakt now 😊 Thank you! Here's the log file: https://pastebin.com/dl/usMKbnPu 7 of the movies were imported with the current time instead since they were missing from movielens-logs.csv, however that's not an issue with TraktRater.

damienhaynes commented 4 years ago

Awesome, good to hear.

I'll now focus on the wishlist and the tags csv, if you have any ideas let me know but I was thinking of creating a custom list 'MovieLens: Wishlist' or similar and add items to that list. I guess the same could be done for movies that are tagged.

MrDemocracy commented 4 years ago

Can you make it so it adds movies from the movielens wishlist to the default trakt wishlist or is that not possible/preferable? How would tags be synced from movielens to trakt? I haven't seen tags used on trakt the same way it's used on movielens, and making a list for each tag added to a move would probably be a bit excessive since you can add multiple tags to one movie.

damienhaynes commented 4 years ago

Do you mean trakt watchlist? I can add it to that easily enough. I just wasn't sure if they had similar meaning but to keep things simple can make it add to the watchlist unless someone wants that to be optional.

Also in the spirit of keeping things simple I will remove the tags import unless people really want it.

MrDemocracy commented 4 years ago

Yes, I meant the trakt watchlist 😅 I'm pretty sure they're supposed to be used for the same thing, so personally I'd prefer if it was imported to the default trakt watchlist.

damienhaynes commented 4 years ago

Here is what is hopefully the final version with support to import the wishlist.

NB: I can't use the dates from the activity log as trakt API does not have support for date/time when inserting into watchlist. TraktRater_v2.3.10.2.exe.zip