FreeTubeApp / FreeTube

An Open Source YouTube app for privacy
https://freetubeapp.io/
GNU Affero General Public License v3.0
13.66k stars 864 forks source link

[Bug]: Unable to import playlist #3261

Closed octodi closed 1 year ago

octodi commented 1 year ago

Guidelines

Describe the bug

I have written a python script to create a playlist database by fetching videoID from a csv and getting other details from Invidious:

import csv
import requests
from datetime import datetime
from tqdm import tqdm

invidious_api = 'https://inv.riverside.rocks/api/v1/videos/'

with open('watch_later.csv', 'r') as csvfile:
    reader = csv.reader(csvfile)
    video_ids = [row[0] for row in reader]

video_data = []

for video_id in tqdm(video_ids, desc='Fetching video details', unit='videos'):
    response = requests.get(invidious_api + video_id)
    if response.status_code == 200:
        video = response.json()
        video_data.append({"videoId":video["videoId"],
                           "title":video["title"],
                           "author":video["author"],
                           "authorId":video["authorId"],
                           "published":"",
                           "viewCount":video["viewCount"],
                           "lengthSeconds":video["lengthSeconds"],
                           "timeAdded":int(datetime.now().replace(second=0, microsecond=0).timestamp()) * 1000,
                           "isLive":"false",
                           "paid":"false",
                           "type":"video"})
    else:
        print(f'Failed to fetch video data for video ID {video_id}. Status code: {response.status_code}')
        print(response.text)

playlist_data = {"playlistName":"Favorites",
                 "videos":video_data,
                 "_id":"avPMWbHn5lzwaGjo"}

with open('playlist_data.txt', 'w') as outfile:
    outfile.write(str([playlist_data]))

But I was unable to import the playlist.db from FreeTube>Data Settings>Import Playlists

Expected Behavior

Output of the program looks like :

[{"playlistName":"Favorites","videos":[{"videoId":"biFCXf8927M","title":"Log File Poisoning and Windows Privilege Escalation | HackTheBox Bart","author":"Motasem Hamdan","authorId":"UCNSdU_1ehXtGclimTVckHmQ","published":"","viewCount":215,"lengthSeconds":3178,"timeAdded":1678031317894,"isLive":false,"paid":false,"type":"video"},{"videoId":"oaJCatpcqBc","title":"Lab-2 Memlabs || Memory Forensics || Digital Forensics","author":"The Cyber Expert","authorId":"UCWd8wa-OOyeBSqBZyiGW99g","published":"","viewCount":211,"lengthSeconds":1150,"timeAdded":1678031352054,"isLive":false,"paid":false,"type":"video"}],"_id":"avPMWbHn5lzwaGjo"}]

I have also changed the file name from text to db before importing

Issue Labels

inconsistent behavior

FreeTube Version

0.18.0-2

Operating System Version

Arch Linux

Installation Method

AUR

Primary API used

Local API

Last Known Working FreeTube Version (If Any)

No response

Additional Information

No response

Nightly Build

octodi commented 1 year ago

I tried exporting playlist from the app itself and tried reimporting it after modifying the timeAdded field. But I was unable to do so, I think the problem is with timeAdded field. Modifying data in _id field doesn't creates the same issue.

MarmadileManteater commented 1 year ago

In the future, I would recommend pasting your python code either into code formatting or into a gist because, without indentation, python can be indecipherable.

  with open('playlist_data.txt', 'w') as outfile:
    outfile.write(str([playlist_data]))

This should be serialised using json.dumps instead of str. Also, the output file should have the file type .db.

This is what I came up with after debugging this script a little locally. I haven't changed a whole lot, but I just thought I would paste it in here with code formatting if it may be helpful.

import csv
import requests
from datetime import datetime
from tqdm import tqdm
import json

invidious_api = 'https://inv.riverside.rocks/api/v1/videos/'

with open('watch_later.csv', 'r') as csvfile:
  reader = csv.reader(csvfile)
  video_ids = [row[0] for row in reader]
  video_data = []
  for video_id in tqdm(video_ids, desc='Fetching video details', unit='videos'):
    response = requests.get(invidious_api + video_id)
    if response.status_code == 200:
      video = response.json()
      video_data.append({"videoId":video["videoId"],
      "title":video["title"],
      "author":video["author"],
      "authorId":video["authorId"],
      "published":"",
      "description":"",
      "viewCount":video["viewCount"],
      "lengthSeconds":video["lengthSeconds"],
      "timeAdded":int(datetime.now().replace(second=0, microsecond=0).timestamp()) * 1000,
      "isLive":False,
      "paid":False,
      "type":"video"})
    else:
      print(f'Failed to fetch video data for video ID {video_id}. Status code: {response.status_code}')
      print(response.text)
  playlist_data = {"playlistName":"Favorites",
  "videos":video_data,
  "_id":"avPMWbHn5lzwaGjo"}
  with open('playlist_data.db', 'w') as outfile:
    outfile.write(json.dumps([playlist_data]))
octodi commented 1 year ago

In the future, I would recommend pasting your python code either into code formatting or into a gist because, without indentation, python can be indecipherable.

  with open('playlist_data.txt', 'w') as outfile:
    outfile.write(str([playlist_data]))

This should be serialised using json.dumps instead of str. Also, the output file should have the file type .db.

This is what I came up with after debugging this script a little locally. I haven't changed a whole lot, but I just thought I would paste it in here with code formatting if it may be helpful.

import csv
import requests
from datetime import datetime
from tqdm import tqdm
import json

invidious_api = 'https://inv.riverside.rocks/api/v1/videos/'

with open('watch_later.csv', 'r') as csvfile:
  reader = csv.reader(csvfile)
  video_ids = [row[0] for row in reader]
  video_data = []
  for video_id in tqdm(video_ids, desc='Fetching video details', unit='videos'):
    response = requests.get(invidious_api + video_id)
    if response.status_code == 200:
      video = response.json()
      video_data.append({"videoId":video["videoId"],
      "title":video["title"],
      "author":video["author"],
      "authorId":video["authorId"],
      "published":"",
      "description":"",
      "viewCount":video["viewCount"],
      "lengthSeconds":video["lengthSeconds"],
      "timeAdded":int(datetime.now().replace(second=0, microsecond=0).timestamp()) * 1000,
      "isLive":False,
      "paid":False,
      "type":"video"})
    else:
      print(f'Failed to fetch video data for video ID {video_id}. Status code: {response.status_code}')
      print(response.text)
  playlist_data = {"playlistName":"Favorites",
  "videos":video_data,
  "_id":"avPMWbHn5lzwaGjo"}
  with open('playlist_data.db', 'w') as outfile:
    outfile.write(json.dumps([playlist_data]))

Thanks for your help 😊, serialising it through json.dumps solved the issue. Yes I was previously changing the file extension to .db . I am new to github, sorry for the inconvenience caused due to formatting.