search

lukearran / TvTimeToTrakt

A Python script to import TV Time data into Trakt.TV
MIT License
147 stars 17 forks source link

Error using the script on GDPR data received on 20 June 2024 #56

Closed unseensholar closed 4 months ago

unseensholar commented 4 months ago

Getting an error while using 'Import Watch History for TV Shows from TV Time'.

└─$ python TimeToTrakt.py
>> What do you want to do?
    1) Import Watch History for TV Shows from TV Time
    2) Import Watched Movies from TV Time
    3) Do both 1 and 2 (default)
    4) Exit
Enter your menu selection: 1
Please go here and authorize, [removed]
Paste the Code returned here: [removed]
2024-06-22 07:49:07 [INFO] :: Processing watched shows.
Traceback (most recent call last):
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 170, in <module>
    start()
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 154, in start
    process_watched_shows(SHOWS_AND_MOVIES_PATH)
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 94, in process_watched_shows
    if row["episode_number"] == "":  # if not an episode entry
KeyError: 'episode_number'

image

Also getting an error while using the 'Import Watched Movies from TV Time' option.


2024-06-22 07:52:07 [INFO] :: (17.91%) - Processing 'Bullet Train'
2024-06-22 07:52:07 [WARNING] :: Bullet Train already in watchlist
Traceback (most recent call last):
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 170, in <module>
    start()
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 159, in start
    process_watched_movies()
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 112, in process_watched_movies
    movie = TVTimeMovie(row)
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/searcher.py", line 149, in __init__
    release_date = datetime.strptime(
  File "/usr/lib/python3.10/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/lib/python3.10/_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data '' does not match format '%Y-%m-%d %H:%M:%S'

image

AdClarky commented 4 months ago

Looks like they've changed the data format again 😭. Would you mind sending a sample of your "tracking-prod-records.csv" and "tracking-prod-records-v2.csv" here? I can fix the second error in a moment.

unseensholar commented 4 months ago

Sure. Thanks.

sample_tracking-prod-records.csv sample_tracking-prod-records-v2.csv

unseensholar commented 4 months ago

The TV Time support did mention that their GDPR export process was undergoing maintenance when I send in the request a week ago.

AdClarky commented 4 months ago

Have you removed anything or is that how the data was sent? I received my data 2 weeks ago and those files for me are 100x bigger than yours. It's quite hard to debug without access to more information, are you okay sending your entire export?

unseensholar commented 4 months ago

I just send you the first 10 columns of each file. I could send the whole thing if you want.

AdClarky commented 4 months ago

oh yes please, there's a lot of data in each file that isn't used so the entire file would be helpful thank you.

unseensholar commented 4 months ago

I've uploaded the files to a repository and sent you an invite. https://github.com/unseensholar/tvtime_GDPR_data

AdClarky commented 4 months ago

@unseensholar could you try #59 see if that works for you? I tested it with no problems but didn't run through the entire data.

unseensholar commented 4 months ago

@unseensholar could you try #59 see if that works for you? I tested it with no problems but didn't run through the entire data.

I don't see any change. Am I doing it wrong?


┌──(kali㉿kali)-[~/Documents/TVTime]
└─$ git clone https://github.com/AdClarky/TvTimeToTrakt.git     
Cloning into 'TvTimeToTrakt'...
remote: Enumerating objects: 171, done.
remote: Counting objects: 100% (92/92), done.
remote: Compressing objects: 100% (45/45), done.
remote: Total 171 (delta 63), reused 66 (delta 47), pack-reused 79
Receiving objects: 100% (171/171), 68.83 KiB | 1.23 MiB/s, done.
Resolving deltas: 100% (96/96), done.

┌──(kali㉿kali)-[~/Documents/TVTime]
└─$ cd TvTimeToTrakt                                            

┌──(kali㉿kali)-[~/Documents/TVTime/TvTimeToTrakt]
└─$ python TimeToTrakt.py                                  
2024-06-22 10:50:44 [INFO] :: config.json not found prompting user for input
Enter your Trakt.tv username: [removed]
Enter you Client id: [removed]
Enter your Client secret: [removed]
Enter your GDPR workspace path: /home/kali/Documents/TVTime/tv-time-personal-data/
>> What do you want to do?
    1) Import Watch History for TV Shows from TV Time
    2) Import Watched Movies from TV Time
    3) Do both 1 and 2 (default)
    4) Exit
Enter your menu selection: 1
Please go here and authorize, [removed]
Paste the Code returned here: [removed]
2024-06-22 10:51:51 [INFO] :: Processing watched shows.
Traceback (most recent call last):
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 170, in <module>
    start()
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 154, in start
    process_watched_shows(SHOWS_AND_MOVIES_PATH)
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 94, in process_watched_shows
    if row["episode_number"] == "":  # if not an episode entry
KeyError: 'episode_number'


┌──(kali㉿kali)-[~/Documents/TVTime/TvTimeToTrakt]
└─$ python TimeToTrakt.py
>> What do you want to do?
    1) Import Watch History for TV Shows from TV Time
    2) Import Watched Movies from TV Time
    3) Do both 1 and 2 (default)
    4) Exit
Enter your menu selection: 2
Please go here and authorize, [removed]
Paste the Code returned here: [removed]
2024-06-22 10:52:34 [INFO] :: Processing movies.
2024-06-22 10:52:36 [INFO] :: (0.00%) - Processing 'The Batman'
2024-06-22 10:52:37 [INFO] :: 'The Batman' added to watchlist
.........................
2024-06-22 10:53:08 [INFO] :: Skipping 'Tenet' to avoid redundant watchlist entry.
2024-06-22 10:53:09 [INFO] :: (17.91%) - Processing 'Bullet Train'
2024-06-22 10:53:09 [INFO] :: 'Bullet Train' added to watchlist
Traceback (most recent call last):
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 170, in <module>
    start()
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 159, in start
    process_watched_movies()
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 112, in process_watched_movies
    movie = TVTimeMovie(row)
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/searcher.py", line 149, in __init__
    release_date = datetime.strptime(
  File "/usr/lib/python3.10/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/lib/python3.10/_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data '' does not match format '%Y-%m-%d %H:%M:%S'
AdClarky commented 4 months ago

It's not the main branch on my fork. Try git checkout betterConfig then run it again. @unseensholar

unseensholar commented 4 months ago

betterConfig

Seems it broke something else.

┌──(kali㉿kali)-[~/Documents/TVTime/TvTimeToTrakt]
└─$ python TimeToTrakt.py                                                   
Traceback (most recent call last):
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 12, in <module>
    from processor import TVShowProcessor, MovieProcessor
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/processor.py", line 12, in <module>
    from searcher import TVShowSearcher, MovieSearcher, TraktTVShow, TraktMovie, TraktItem, TVTimeItem, TVTimeTVShow, \
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/searcher.py", line 22, in <module>
    DATE_TIME_FORMAT = json.load(f)["DATE_FORMAT"]
KeyError: 'DATE_FORMAT'
AdClarky commented 4 months ago

I updated how the config file works, have you updated yours? Sorry I should've mentioned that, the updated versions is in the README

unseensholar commented 4 months ago

I updated how the config file works, have you updated yours? Sorry I should've mentioned that, the updated versions is in the README

........I did not. My bad.

unseensholar commented 4 months ago

I updated how the config file works, have you updated yours? Sorry I should've mentioned that, the updated versions is in the README

Still got issues:

Changed the config to:

{
    "CLIENT_ID": "[removed]",
    "CLIENT_SECRET": "[removed]",
    "MOVIE_DATA_PATH": "/home/kali/Documents/TVTime/tv-time-personal-data/tracking-prod-records.csv",
    "SHOW_DATA_PATH": "/home/kali/Documents/TVTime/tv-time-personal-data/tracking-prod-records-v2.csv",
    "TRAKT_USERNAME": "[removed]",
    "DATE_FORMAT": "%d/%m/%Y %H:%M"
}

Getting the error:

2024-06-22 11:14:00 [INFO] :: Processing both watched shows and movies.
Traceback (most recent call last):
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 173, in <module>
    start()
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 166, in start
    process_watched_shows()
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/TimeToTrakt.py", line 103, in process_watched_shows
    tv_time_show = TVTimeTVShow(row)
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/searcher.py", line 113, in __init__
    super().__init__(row["series_name"], row["created_at"])
  File "/home/kali/Documents/TVTime/TvTimeToTrakt/searcher.py", line 106, in __init__
    self.date_watched = datetime.strptime(
  File "/usr/lib/python3.10/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/lib/python3.10/_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data '2020-07-23 06:19:45' does not match format '%d/%m/%Y %H:%M'
AdClarky commented 4 months ago

Interesting, in the data sample you shared with me all the dates are in the format "%d/%m/%Y %H:%M". I think maybe you opened the file in a program which changed the date format? Could you check if all your dates are in this format?

Try adjusting the date format in the config to "%Y-%m-%d %H:%M:%S" and see if that works.

unseensholar commented 4 months ago

Interesting, in the data sample you shared with me all the dates are in the format "%d/%m/%Y %H:%M". I think maybe you opened the file in a program which changed the date format? Could you check if all your dates are in this format?

Try adjusting the date format in the config to "%Y-%m-%d %H:%M:%S" and see if that works.

Thank you so much. That seems to be exactly what had happened. Well, actually, I had opened the files I shared to you before, and it seems the date format was changed while I saved and uploaded it into the repository.............. Really am sorry about that..... It seems this, '%Y-%m-%d %H:%M:%S', was the original format. Might want to change that in the commit.

Thanks again for all your help.

unseensholar commented 4 months ago

The script is working great now!