Videos in a playlist uploaded on the same day are not sorted by time

[duckpuppy]

Describe the bug If a playlist/subscription has multiple videos uploaded on the same day, they are not sorted by time when using the publish_date ordering.

To Reproduce Subscribe to a channel that publishes multiple videos per day and update. Set the order_by value in ytcc.conf to publish_date:asc. Execute ytcc list -p <channel>

Expected behavior Videos are listed in the strict order they were published.

Environment, configuration and version information

---ytcc version---
2.0.1

---youtube-dl version---
2020.12.09

---SQLite version---
SQLite system library version: 3.34.0
Python module version: 2.6.0

---python version---
3.9.0 (default, Dec  6 2020, 18:02:34)
[Clang 12.0.0 (clang-1200.0.32.27)]

---mpv version---
mpv 0.33.0 Copyright © 2000-2020 mpv/MPlayer/mplayer2 projects
 built on Sun Nov 22 19:54:23 GMT 2020
FFmpeg library versions:
   libavutil       56.51.100
   libavcodec      58.91.100
   libavformat     58.45.100
   libswscale      5.7.100
   libavfilter     7.85.100
   libswresample   3.7.100
FFmpeg version: 4.3.1

---config dump---
[ytcc]
download_dir = ~/Downloads
mpv_flags = --really-quiet --ytdl --ytdl-format=bestvideo[height<=?1080]+bestaudio/best
order_by = publish_date:asc
video_attrs = id, title, publish_date, duration, playlists
playlist_attrs = name, url, tags
db_path = ~/.local/share/ytcc/ytcc.db
date_format = %Y-%m-%d
max_update_fail = 5
max_update_backlog = 20
age_limit = 0

[tui]
alphabet = sdfervghnuiojkl
default_action = play_video

[theme]
prompt_download_audio = 2
prompt_download_video = 4
prompt_play_audio = 2
prompt_play_video = 4
prompt_mark_watched = 1
table_alternate_background = 19

[youtube_dl]
format = bestvideo[height<=?1080]+bestaudio/best
output_template = %(title)s.%(ext)s
ratelimit = 0
retries = 0
subtitles = off
thumbnail = true
skip_live_stream = true
merge_output_format = mkv

Additional context Example output from a channel below. I used the YouTube Data API to retrieve two of the videos (Part 225 and 224). 224's publishedAt property was one hour before 225, but in the list below all the videos for each day are sorted in descending order by time even though the ordering in the config file is publish_date:asc.

 id   │ title                                                             │ publish_date │ duration │ playlists
──────┼───────────────────────────────────────────────────────────────────┼──────────────┼──────────┼───────────────────
 56   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 202 │ 2020-11-24   │   32:21  │ Aavak After Hours
 57   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 201 │ 2020-11-24   │   34:11  │ Aavak After Hours
 58   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 200 │ 2020-11-24   │   32:04  │ Aavak After Hours
 59   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 199 │ 2020-11-24   │   31:35  │ Aavak After Hours
 60   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 198 │ 2020-11-24   │   30:15  │ Aavak After Hours
 49   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 209 │ 2020-11-25   │   29:11  │ Aavak After Hours
 50   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 208 │ 2020-11-25   │   29:14  │ Aavak After Hours
 51   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 207 │ 2020-11-25   │   31:01  │ Aavak After Hours
 52   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 206 │ 2020-11-25   │   35:05  │ Aavak After Hours
 53   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 205 │ 2020-11-25   │   28:30  │ Aavak After Hours
 54   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 204 │ 2020-11-25   │   31:08  │ Aavak After Hours
 55   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 203 │ 2020-11-25   │   30:21  │ Aavak After Hours
 41   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 217 │ 2020-12-03   │   31:41  │ Aavak After Hours
 42   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 216 │ 2020-12-03   │   36:35  │ Aavak After Hours
 43   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 215 │ 2020-12-03   │   32:30  │ Aavak After Hours
 44   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 214 │ 2020-12-03   │   32:47  │ Aavak After Hours
 45   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 213 │ 2020-12-03   │   29:07  │ Aavak After Hours
 46   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 212 │ 2020-12-03   │   31:52  │ Aavak After Hours
 47   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 211 │ 2020-12-03   │   31:05  │ Aavak After Hours
 48   │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 210 │ 2020-12-03   │   25:10  │ Aavak After Hours
 2451 │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 225 │ 2020-12-09   │   40:52  │ Aavak After Hours
 2452 │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 224 │ 2020-12-09   │   32:56  │ Aavak After Hours
 2453 │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 223 │ 2020-12-09   │   32:24  │ Aavak After Hours
 2454 │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 222 │ 2020-12-09   │   33:54  │ Aavak After Hours
 2455 │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 221 │ 2020-12-09   │   30:13  │ Aavak After Hours
 2456 │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 220 │ 2020-12-09   │   30:04  │ Aavak After Hours
 2457 │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 219 │ 2020-12-09   │   31:27  │ Aavak After Hours
 2458 │ Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 218 │ 2020-12-09   │   32:34  │ Aavak After Hours

Partial responses for each video from the API:

{
  "kind": "youtube#videoListResponse",
  "etag": "xTwOH37rQ4erFVC3YQJ2CpE6SWw",
  "items": [
    {
      "kind": "youtube#video",
      "etag": "PXp1Up0QqBrdbhND8bSr8y6P5wc",
      "id": "OGUUo5V0CuQ",
      "snippet": {
        "publishedAt": "2020-12-10T04:00:02Z",
        "channelId": "UCk7U8vOfzutkuZ-coGdpXrw",
        "title": "Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 224",

{
  "kind": "youtube#videoListResponse",
  "etag": "zD1fJud1ccfJIQQm_SUjtzzuD9I",
  "items": [
    {
      "kind": "youtube#video",
      "etag": "BEj4UbeWNkTdXVe3hnzt9N7Nsuw",
      "id": "WQj-OJ6Pwqs",
      "snippet": {
        "publishedAt": "2020-12-10T05:00:21Z",
        "channelId": "UCk7U8vOfzutkuZ-coGdpXrw",
        "title": "Oxygen Not Included: Oassise – Let’s Play Stream Archive Part 225",

[Glandos]

See https://github.com/ytdl-org/youtube-dl/issues/1706

This is not possible because youtube-dl doesn't have the time. Maybe a sort according to name should be done then.

[duckpuppy]

That would work in that exact case, but only because the uploader used the exact same title each time. A pure string sort on most other channels would fail since it's common to name those types of videos "Catchy title just for this episode to get your attention - Game Name Part X".

I know it's not your fault - it's certainly youtube-dl's fault for ignoring that issue for this long - but I think it does make ytcc unusable for my purposes. It's a great piece of software otherwise.

[pukkandan]

I know it's not your fault - it's certainly youtube-dl's fault for ignoring that issue for this long

It is youtube's "fault" for not providing this data... This info is available only through the data API which ytdl obv cannot use, and not in the innertube API

Can't ytcc somehow save the order in which the video is extracted? ytdl/p always extracts videos in the order the youtube page provides them. So when you do yt-dlp "https://www.youtube.com/channel/...", it will be in the correct newest to oldest

[woefe]

Can't ytcc somehow save the order in which the video is extracted?

Ytcc actually does that. Video IDs in ytcc (scoped by playlist) are increasing in the order they are extracted from ytdl. But that does not help as a reliable workaround. Sorting by order_by = playlists:asc, publish_date:desc, id:asc will list in correct order, assuming the playlist on YouTube is sorted oldest to newest. You'll run into problems, though, if the playlist is sorted any different than ytcc's pulish_date sorter setting, or if you ytcc update in between same-day video releases.