search

woefe / ytcc

Command line tool to keep track of your favorite playlists on YouTube and many other places.
GNU General Public License v3.0
178 stars 20 forks source link

Migration from 1.8.4 to 2.0.1 fails... (migration script) #80

Closed pghpete closed 3 years ago

pghpete commented 3 years ago

While following the migration instructions to migrate using the migration script, it fails...

The install using pip throws some errors but appears to function over all...

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting git+https://github.com/woefe/ytcc.git
  Cloning https://github.com/woefe/ytcc.git to /tmp/pip-req-build-y2z0ogcc
Requirement already satisfied: click in /usr/lib/python3/dist-packages (from ytcc==2.0.1) (7.0)
Collecting wcwidth (from ytcc==2.0.1)
  Downloading https://files.pythonhosted.org/packages/59/7c/e39aca596badaf1b78e8f547c807b04dae603a433d3e7a7e04d67f2ef3e5/wcwidth-0.2.5-py2.py3-none-any.whl
Requirement already satisfied: youtube_dl in /usr/local/lib/python3.7/dist-packages (from ytcc==2.0.1) (2020.12.12)
Building wheels for collected packages: ytcc
  Running setup.py bdist_wheel for ytcc ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-req-build-y2z0ogcc/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-aioub81k --python-tag cp37:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for ytcc
  Running setup.py clean for ytcc
Failed to build ytcc
Installing collected packages: wcwidth, ytcc
  Found existing installation: ytcc 1.8.4
    Uninstalling ytcc-1.8.4:
      Successfully uninstalled ytcc-1.8.4
  Running setup.py install for ytcc ... done
Successfully installed wcwidth-0.2.5 ytcc-2.0.1

Version appears to have been updated properly...

[pete@omv: src]$ ytcc --version
ytcc, version 2.0.1

Copyright (C) 2015-2020  Wolfgang Popp
This program comes with ABSOLUTELY NO WARRANTY; This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU General
Public Licence for details.

The migrate.py script is then run using...

python3 migrate.py --olddb ~/.local/share/ytcc/ytcc.db.v1 --newdb ~/.local/share/ytcc/ytcc.d

This also appears to complete successfully without any errors. However, when I go to run ytcc and list subscriptions... I get this...

[pete@omv: src]$ ytcc subscriptions
ERROR: This version of ytcc is not compatible with the older database versions. See https://github.com/woefe/ytcc/blob/master/doc/migrate.md for more details.

Environment, configuration and version information

[pete@omv: src]$ cat /etc/os-release 
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
[pete@omv: src]$ uname -a
Linux omv.zhome 5.4.79-v7l+ #1373 SMP Mon Nov 23 13:27:40 GMT 2020 armv7l GNU/Linux

Bug report output

ytcc bug-report
---ytcc version---
2.0.1

---youtube-dl version---
2020.12.12

---SQLite version---
SQLite system library version: 3.27.2
Python module version: 2.6.0

---python version---
3.7.3 (default, Jul 25 2020, 13:03:44) 
[GCC 8.3.0]

---mpv version---
mpv 0.29.1 Copyright © 2000-2018 mpv/MPlayer/mplayer2 projects
 built on UNKNOWN
ffmpeg library versions:
   libavutil       56.14.100 (runtime 56.22.100)
   libavcodec      58.18.100 (runtime 58.35.100)
   libavformat     58.12.100 (runtime 58.20.100)
   libswscale      5.1.100 (runtime 5.3.100)
   libavfilter     7.16.100 (runtime 7.40.101)
   libswresample   3.1.100 (runtime 3.3.100)
ffmpeg version: 4.1.6-1~deb10u1+rpt1

---config dump---
[ytcc]
download_dir = ~/Downloads
mpv_flags = --really-quiet --ytdl --ytdl-format=bestvideo[height<=?1080]+bestaudio/best
order_by = playlists:asc, publish_date:desc
video_attrs = id, title, publish_date, duration, playlists
playlist_attrs = name, url, tags
db_path = ~/.local/share/ytcc/ytcc.db
date_format = %Y-%m-%d
max_update_fail = 5
max_update_backlog = 20
age_limit = 0

[tui]
alphabet = sdfervghnuiojkl
default_action = play_video

[theme]
prompt_download_audio = 2
prompt_download_video = 4
prompt_play_audio = 2
prompt_play_video = 4
prompt_mark_watched = 1
table_alternate_background = 245

[youtube_dl]
format = bestvideo[height<=?1080]+bestaudio/best
output_template = %(title)s.%(ext)s
ratelimit = 0
retries = 0
subtitles = off
thumbnail = true
skip_live_stream = true
merge_output_format = mkv
pghpete commented 3 years ago

I may be reverting back to an older version though... If I can't replicate this same functionality in my cron script...

#!/bin/bash

# ytcc.get.sh: update/get videos from youtube subscriber list
# requires: youtube-dl, ytcc, minidlna

# output directory
output_dir="/export/dlna"

# Update videos list, from subscribers
/usr/local/bin/ytcc -u

# get new videos from the last 3 days that were not already downloaded
/usr/local/bin/ytcc -d -s $(date -d "-3 day" +%Y-%m-%d) -p $output_dir -g

# Redownload, even watched from last 24h
#/usr/local/bin/ytcc -n -d -s $(date -d "-1 day" +%Y-%m-%d) -p $output_dir -g
woefe commented 3 years ago

You're not installing it from pypi.org? What's the pip command you used to install it?

The migration script from master should be working with the current master. The script at tag v2.0.1 should work with 2.0.1. Also, check ytcc --loglevel debug subscriptions and post the error message if possible

woefe commented 3 years ago

And regarding the pip error: you might be missing the wheel package.

pghpete commented 3 years ago

Installed the wheel package but still no dice.

A DB migration attempt...

pete@omv: src]$ python3 migrate.py --olddb ~/.local/share/ytcc/ytcc.db.v1 --newdb ~/.local/share/ytcc/ytcc.db
Wm. Walker Co. is already subscribed.
Adding videos for Wm. Walker Co.
Traceback (most recent call last):
  File "migrate.py", line 123, in <module>
    core_v2.database.add_videos(videos, ytcc.Playlist(c_name, url))
  File "/usr/local/lib/python3.7/dist-packages/ytcc/database.py", line 258, in add_videos
    for video in videos:
  File "migrate.py", line 120, in <genexpr>
    for yt_videoid, title, description, publish_date, watched
TypeError: __init__() got an unexpected keyword argument 'watch_date'

ytcc version

[pete@omv: src]$ ytcc --version
ytcc, version 2.0.1

a ytcc subscriptions run with debug


[pete@omv: src]$ ytcc --loglevel debug subscriptions
[1608698736.744235] [MainProcess/MainThread] ytcc.config.DEBUG: Trying to read config from following locations: ['/etc/ytcc/ytcc.conf', '/home/pete/.config/ytcc/ytcc.conf', '/home/pete/.ytcc.conf']
[1608698736.747499] [MainProcess/MainThread] ytcc.config.DEBUG: Config was read from following locations: ['/home/pete/.config/ytcc/ytcc.conf']
[1608698736.749665] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA foreign_keys = ON;
[1608698736.749979] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA USER_VERSION;
[1608698736.750940] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT p.id AS id, p.name AS name, p.url AS url, t.name AS tag FROM playlist AS p LEFT OUTER JOIN tag AS t ON p.id = t.playlist;
 name           │ url                                                             │ tags 
────────────────┼─────────────────────────────────────────────────────────────────┼──────
 Wm. Walker Co. │ https://www.youtube.com/channel/UCievvwx_-UU-rP28103rUCw/videos │
pghpete commented 3 years ago

I don't want to trouble you anymore though if the feature which allowed you to download by date is no longer in ytcc. Is it? or am I missing something? Specifically...

Update subscription available videos...
usr/local/src/ytcc/ytcc.py -u
# get new videos from the last 3 days, not already downloaded
/usr/local/src/ytcc/ytcc.py -d -s $(date -d "-3 day" +%Y-%m-%d) -p $output_dir -g

oh and I think I forgot to mention I believe I was just using "pip3 install ytcc"

woefe commented 3 years ago

I release version 2.1.0 yesterday. Does that fix your migration problem? Make sure you also get the latest migration script.

woefe commented 3 years ago

I don't want to trouble you anymore though if the feature which allowed you to download by date is no longer in ytcc. Is it? or am I missing something?

No, it's still there: ytcc ls --since 2021-01-28 | ytcc download