ytcc list / ytcc ls not returning any data after initial subscribe and update

[pghpete]

This may totally be user error and if it is I apologize in advance. That being said, I seem to be having issues, on a fresh install via pip, getting "ytcc list" or "ytcc ls" to return anything at all.

My basic goal is to use this command every X days to download new videos from youtube. ytcc ls --since $(date -d "-3 day" +%Y-%m-%d) | ytcc download

Describe the bug No output from "ytcc list" or "ytcc ls" after fresh install, ytcc subscribe , ytcc update

To Reproduce Steps to reproduce the behavior:

1. pip install ytcc
2. ytcc subscribe
3. ytcc update
4. ytcc list (returns nothing)
5. ytcc ls (returns nothing)

Expected behavior Output from ytcc ls and/or ytcc list

Environment, configuration and version information

ytcc --version
ytcc, version 2.5.0

---ytcc version---
2.5.0

---youtube-dl version---
2021.06.06

---SQLite version---
SQLite system library version: 3.26.0
Python module version: 2.6.0

---python version---
3.9.2 (default, Mar  5 2021, 01:49:45) 
[GCC 8.4.1 20200928 (Red Hat 8.4.1-1)]

---mpv version---
mpv is not installed

---config dump---
[ytcc]
download_dir = /archive2/dlna
download_subdirs = false
mpv_flags = --really-quiet --ytdl --ytdl-format=bestvideo[height<=?1080]+bestaudio/best
order_by = playlists:asc, publish_date:desc
video_attrs = id, title, publish_date, duration, playlists
playlist_attrs = name, url, tags, reverse
db_path = ~/.local/share/ytcc/ytcc.db
date_format = %Y-%m-%d
max_update_fail = 5
max_update_backlog = 20
age_limit = 0

[tui]
alphabet = sdfervghnuiojkl
default_action = play_video

[theme]
prompt_download_audio = 2
prompt_download_video = 4
prompt_play_audio = 2
prompt_play_video = 4
prompt_mark_watched = 1
table_alternate_background = 245

[youtube_dl]
format = bestvideo[height<=?1080]+bestaudio/best
output_template = %(title)s.%(ext)s
ratelimit = 0
retries = 0
subtitles = off
thumbnail = true
skip_live_stream = true
merge_output_format = mkv
max_duration = 0
restrict_filenames = false

Debug log (optional)

ytcc --loglevel debug ls
[1634318311.493418] [MainProcess/MainThread] ytcc.config.DEBUG: Trying to read config from following locations: ['/etc/ytcc/ytcc.conf', '/home/pete/.config/ytcc/ytcc.conf', '/home/pete/.ytcc.conf']
[1634318311.493765] [MainProcess/MainThread] ytcc.config.DEBUG: Config was read from following locations: ['/home/pete/.config/ytcc/ytcc.conf']
[1634318311.495631] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA foreign_keys = ON;
[1634318311.495694] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA user_version;
[1634318311.495954] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT v.id AS id, v.title AS title, v.url AS url, v.description AS description, v.duration AS duration, v.publish_date AS publish_date, v.watch_date AS watch_date, v.thumbnail_url AS thumbnail_url, v.extractor_hash AS extractor_hash, p.name AS playlist_name, p.url AS playlist_url, p.reverse AS playlist_reverse FROM video AS v JOIN content c ON v.id = c.video_id JOIN playlist p ON p.id = c.playlist_id LEFT JOIN tag AS t ON p.id = t.playlist WHERE v.publish_date > 190800.0 AND v.publish_date < 32536789200.0 AND v.watch_date IS NULL ORDER BY playlist_name ASC, publish_date DESC

 ytcc --loglevel debug list
[1634319824.417430] [MainProcess/MainThread] ytcc.config.DEBUG: Trying to read config from following locations: ['/etc/ytcc/ytcc.conf', '/home/pete/.config/ytcc/ytcc.conf', '/home/pete/.ytcc.conf']
[1634319824.417840] [MainProcess/MainThread] ytcc.config.DEBUG: Config was read from following locations: ['/home/pete/.config/ytcc/ytcc.conf']
[1634319824.419817] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA foreign_keys = ON;
[1634319824.419881] [MainProcess/MainThread] ytcc.database.DEBUG: PRAGMA user_version;
[1634319824.420147] [MainProcess/MainThread] ytcc.database.DEBUG: SELECT v.id AS id, v.title AS title, v.url AS url, v.description AS description, v.duration AS duration, v.publish_date AS publish_date, v.watch_date AS watch_date, v.thumbnail_url AS thumbnail_url, v.extractor_hash AS extractor_hash, p.name AS playlist_name, p.url AS playlist_url, p.reverse AS playlist_reverse FROM video AS v JOIN content c ON v.id = c.video_id JOIN playlist p ON p.id = c.playlist_id LEFT JOIN tag AS t ON p.id = t.playlist WHERE v.publish_date > 190800.0 AND v.publish_date < 32536789200.0 AND v.watch_date IS NULL ORDER BY playlist_name ASC, publish_date DESC
 id │ title │ publish_date │ duration │ playlists 
────┼───────┼──────────────┼──────────┼───────────

The database appears to be populated and when I browse the 'Videos' table with sqlitebrowser it does show several hundred entries.

Any help would be greatly appreciated. I would also like to know if it is possible to donate/contribute to the project in some way. From a donation to documentation, I'd love to help.

Regards Pete

[pghpete]

Not sure if this helps, but here is the output from a subscribe...

ytcc subscribe "DIY Perks" https://www.youtube.com/channel/UCUQo7nzH1sXVpzL92VesANw
INFO: Performing update check on 10 playlist items
INFO: Checking playlist 'DIY Perks'...
WARNING: Publication date of video 'Renewable Energy Projects' is unknown
WARNING: Duration of video 'Renewable Energy Projects' is unknown
INFO: Processed video 'Renewable Energy Projects'
...

[woefe]

This problem is related to youtube-dl, which hasn't seen any updates in months and is starting to break on more and more websites. Ytcc supports yt-dlp since version 2.5.0. Install yt-dlp and ytcc will prefer it over youtube-dl.

[pghpete]

yt-dlp has been on my system during all of this...

yt-dlp==2021.10.10

and it would effect a "ytcc list" and "ytcc ls" ? I thought I ought to at least see available videos to download that match my date/since criteria.

[woefe]

According to the ytcc bug-report output you posted above yt-dlp is not available. The Python module of yt-dlp needs to be available, because ytcc hooks into internals of yt-dlp or youtube-dl. Installing yt-dlp from the binary bundle with sudo curl -L ... -o /usr/local/bin/yt-dlp or similar will not work.

And ytcc list is affected, because the publication date is unknown (as shown in the warnings you posted above). When listing videos ytcc uses a default date range of several hundred years, which leads to empty results if the publication date is null. The same is true if you provide your own date range

[pghpete]

Ah! Makes sense. I think I see why yt-dlp wasn't available or at least wasn't seen by ytcc... I was attempting to use ytcc under a user account on this particular system when yt-dlp was installed I thought I used sudo python3 -m pip install yt-dlp ... but just running pip install yt-dlp as the user in question then running a ytcc update did the trick.

ytcc list now returns data.

Thank you very much for your help with this and your time on the project in general. Please let me know if I can donate time or donate financially to this project or to yourself in general.

Regards!