Some <entry> elements from YT rss feed don't parse

[Carravan]

Some entries from YT rss feed seem to fail parsing or are discarded. The videos can be added manually and are recorded as expected (based on similar, past cases for which I didn't open an issue). I checked trace-level logs but the two video ID never appeared. The filter I'm applying should match everything. Running 0.2.2 in Docker v20.10.21 on Windows

[2023-01-07T06:32:50Z INFO  hoshinova] hoshinova v0.2.2 (23435f3)
[2023-01-07T06:32:50Z DEBUG hoshinova] Git hash: 23435f3334ac948707543763c6b973b39cb8d904
[2023-01-07T06:32:50Z DEBUG hoshinova] Built on: 2022-11-05T04:06:50.280737947+00:00

Config

[ytarchive]
executable_path = "ytarchive"
working_directory = "temp"
args = ["--vp9", "--thumbnail", "--add-metadata", "--threads", "4", "--output", "%(upload_date)s %(title)s [%(channel)s] (%(id)s)", "--cookies", "/app/youtube.com_cookies.txt", "--retry-stream", "120"]
quality = "best"
delay_start = "3s"

[scraper.rss]
poll_interval = "30s"

[notifier.discord]
webhook_url = "hook url here"
notify_on = ["recording", "done", "failed"]

# Coming soon, a web interface to view and manage tasks.
# Optional, remove this section to disable.
[webserver]
bind_address = "0.0.0.0:1104"

[[channel]]
id = "UCu1INzefw3R7M9-3QEHYmMQ"
name = "Nene Amano"
filters = [".*"]
outpath = "/app/kawaii/Nene Amano"

Exerpt from https://www.youtube.com/feeds/videos.xml?channel_id=UCu1INzefw3R7M9-3QEHYmMQ

<entry>
<id>yt:video:zjUlh5lVFDs</id>
<yt:videoId>zjUlh5lVFDs</yt:videoId>
<yt:channelId>UCu1INzefw3R7M9-3QEHYmMQ</yt:channelId>
<title>【ASMR/KU100】WIFE Becomes Your Bunny Servant [Onee-san Voice/$10,000 Mic]</title>
<link rel="alternate" href="https://www.youtube.com/watch?v=zjUlh5lVFDs"/>
<author>
<name>Nene Amano Ch. 天野寧々【kawaii】</name>
<uri>https://www.youtube.com/channel/UCu1INzefw3R7M9-3QEHYmMQ</uri>
</author>
<published>2023-01-05T02:07:53+00:00</published>
<updated>2023-01-05T22:13:45+00:00</updated>
<media:group>
<media:title>【ASMR/KU100】WIFE Becomes Your Bunny Servant [Onee-san Voice/$10,000 Mic]</media:title>
<media:content url="https://www.youtube.com/v/zjUlh5lVFDs?version=3" type="application/x-shockwave-flash" width="640" height="390"/>
<media:thumbnail url="https://i3.ytimg.com/vi/zjUlh5lVFDs/hqdefault.jpg" width="480" height="360"/>
<media:description>#VTuber #ENVTuber #nenetan Using over $10,000 of audio equipment. The KU100 ASMR mic! (Whispering/Oil Massage/Heartbeat/Ear Cleaning) ☁️Become a memeber and support Tenshi!!☁️ https://www.youtube.com/channel/UCu1INzefw3R7M9-3QEHYmMQ/join ☁️Donation Link!☁️ https://streamlabs.com/Neneamano ☁️1st Anniversary Celebration Merch☁️ https://productionkawaii.booth.pm/items/3816918 ☁️Twitter☁️ https://twitter.com/amanene_kawaii Thumbnail by https://twitter.com/nakanomaru925 ✰┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈✰ ଘ(❁•̀ 3 •́)━☆*✲⋆[Chat Rules]☆*✲⋆ Let’s make this a fun and enjoyable place for everyone! ε(*´・ω・)з If rules are not followed, your comment may get deleted/muted. 1. No spoilers! They are my worst enemy! 2. Please be nice to other people watching. Don’t spam or troll. 3. Please understand that I try to respond to every comment, but it’s not possible for me to ​engage with every comment. Don’t take it personally if I don’t read your comment out loud! 4. If you see spam or trolling, don’t engage with it. Just block, report, and ignore it. Tenshi will take care of getting rid of them. ଘ(˵╹-╹)━☆ 5. Please keep the conversation relevant to the stream content. Asking me questions is ok, just make sure they are appropriate! 6. Please don’t cross the line with overly inappropriate/sensitive/sexual comments 7. I will only listen to backseating when I need help, and will ask for help if needed. Please keep in mind I may not listen to advice because I want to try for myself. 8. Please don’t talk about other streamers unless I engage the conversation myself. 9. I don’t do shout outs/saying name requests outside of superchats/donations. ✰┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈✰ kawaii members: 【Reina Sun 🍣 レイナ・サン】 https://www.youtube.com/channel/UCKJexadCeNo3lu0U20skmNg Twitter: https://twitter.com/reina_kawaiii 【Charlotte Suzu 🩸 シャーロット・スズ】 https://www.youtube.com/channel/UCoAw3SML_09dF-7yhQU8wFQ Twitter: https://twitter.com/charsuzu_kawaii 【Isla Coleman 👸 アイラ・コールマン】 https://www.youtube.com/channel/UCuaDidGk4HNLOnQE9wzMMeQ Twitter: https://twitter.com/isla_kawaii Kawaii Gen 2 Members: 【Lua Asuka Ch. 飛鳥瑠藍】 【Shee Icho Ch. 銀杏しい】 【Namiji Freesia Ch. ナミジ・フリージア】 Kawaii Gen 3 Members: 【Oceane Otoishi Ch. 音衣詩おしあん 】 【Aletta Sky Ch. アレッタ・スカイ】 【Miryu Kotofuji Ch. 琴藤みりゅう】 【Peony Aeria Ch. ピオニー・エリア】 【Sava Safari Ch. サヴァ・サファリ】 ✰┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈✰ Contact ／お問い合わせ 【プロダクション「kawaii」HP】 https://production-kawaii.com/en/home-2/ 【「kawaii」公式Twitter】 https://twitter.com/kawaiiOfficial5​✧┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈✧ I'm your angel, you're my angel! It's Nene! VTuber from production kawaii (Japan) who does chatting, gaming, and ASMR! I use a 3dio mic and a KU100 to do binaural ASMR. I can guarantee you will enjoy my waifu/wife ASMR. I can speak Tenshi, English, and Japanese! I also love to sing and rap, and have a 3D model coming in January!! Idol Tenshi coming soon! I enjoy games like Elden Ring, Dragon Age, FGO, Fate/Stay Night, Umineko, and more! I'm a big anime/manga fan who likes series such as Vinland Saga, JJBA, Berserk, Steins;Gate, chainsawman, FMA fullmetal alchemist, and more. I'm a big fan of VTubers such as Hololive. HololiveEN and HoliveJP are inspirations. I was inspired by Shirogane Noel, Mori Calliope, Gawr Gura, Ceres Fauna, Ouro Kronii, and more. VShojo and Nijisanji inspired me as well through art, digitalart, singing, game review, let's play, gaming and more. I quit my job to become a virtual animegirl and virtualyoutuber and will continue to fulfill my dream and lots of livestreaming to bring you joy. I usually stream every day! (and post lots of meme anime meme, animememe, anime funny).</media:description>
<media:community>
<media:starRating count="102" average="5.00" min="1" max="5"/>
<media:statistics views="0"/>
</media:community>
</media:group>
</entry>
<entry>
<id>yt:video:XHIrqwyMD6Q</id>
<yt:videoId>XHIrqwyMD6Q</yt:videoId>
<yt:channelId>UCu1INzefw3R7M9-3QEHYmMQ</yt:channelId>
<title>I’m Going to Destroy Amelia’s Entire Family [Bloodborne]</title>
<link rel="alternate" href="https://www.youtube.com/watch?v=XHIrqwyMD6Q"/>
<author>
<name>Nene Amano Ch. 天野寧々【kawaii】</name>
<uri>https://www.youtube.com/channel/UCu1INzefw3R7M9-3QEHYmMQ</uri>
</author>
<published>2023-01-05T01:37:18+00:00</published>
<updated>2023-01-05T09:51:47+00:00</updated>
<media:group>
<media:title>I’m Going to Destroy Amelia’s Entire Family [Bloodborne]</media:title>
<media:content url="https://www.youtube.com/v/XHIrqwyMD6Q?version=3" type="application/x-shockwave-flash" width="640" height="390"/>
<media:thumbnail url="https://i1.ytimg.com/vi/XHIrqwyMD6Q/hqdefault.jpg" width="480" height="360"/>
<media:description>#VTuber #ENVTuber #nenetan ☁️Donation Link!☁️ https://streamlabs.com/Neneamano Special alert sounds for $1, $10, $39, $50, $60, $100, $104, and $1000 donations! ☁️Become a memeber and support Tenshi!!☁️ https://www.youtube.com/channel/UCu1INzefw3R7M9-3QEHYmMQ/join ☁️Oppai Mousepad, Acrylic Stand, and more! Official Merch!☁️ https://booth.pm/en/search/Nene%20Amano ☁️Twitter☁️ https://twitter.com/amanene_kawaii Thumbnail is created with novelai. It is an ai generated image. ✰┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈✰ ଘ(❁•̀ 3 •́)━☆*✲⋆[Chat Rules]☆*✲⋆ Let’s make this a fun and enjoyable place for everyone! ε(*´・ω・)з If rules are not followed, your comment may get deleted/muted. 1. No spoilers! They are my worst enemy! 2. Please be nice to other people watching. Don’t spam or troll. 3. Please understand that I try to respond to every comment, but it’s not possible for me to ​engage with every comment. Don’t take it personally if I don’t read your comment out loud! 4. If you see spam or trolling, don’t engage with it. Just block, report, and ignore it. Tenshi will take care of getting rid of them. ଘ(˵╹-╹)━☆ 5. Please keep the conversation relevant to the stream content. Asking me questions is ok, just make sure they are appropriate! 6. Please don’t cross the line with overly inappropriate/sensitive/sexual comments 7. I will only listen to backseating when I need help, and will ask for help if needed. Please keep in mind I may not listen to advice because I want to try for myself. 8. Please don’t talk about other streamers unless I engage the conversation myself. 9. I don’t do shout outs/saying name requests outside of superchats/donations. ✰┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈✰ kawaii members: 【Reina Sun 🍣 レイナ・サン】 https://www.youtube.com/channel/UCKJexadCeNo3lu0U20skmNg Twitter: https://twitter.com/reina_kawaiii 【Charlotte Suzu 🩸 シャーロット・スズ】 https://www.youtube.com/channel/UCoAw3SML_09dF-7yhQU8wFQ Twitter: https://twitter.com/charsuzu_kawaii 【Isla Coleman 👸 アイラ・コールマン】 https://www.youtube.com/channel/UCuaDidGk4HNLOnQE9wzMMeQ Twitter: https://twitter.com/isla_kawaii ✰┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈✰ Contact ／お問い合わせ 【プロダクション「kawaii」HP】 https://production-kawaii.com/en/home-2/ 【「kawaii」公式Twitter】 https://twitter.com/kawaiiOfficial5​✧┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈✧</media:description>
<media:community>
<media:starRating count="54" average="5.00" min="1" max="5"/>
<media:statistics views="0"/>
</media:community>
</media:group>
</entry>

[hizkifw]

Hello,

The version you're running has an additional filter in the RSS scraper that discards entries older than 24 hours by default. This is because hoshinova is mainly a live stream recording tool and I wanted to reduce the chances of hitting rate limits.

https://github.com/HoloArchivists/hoshinova/blob/main/config.example.toml#L21

You can try adding the configuration and setting the value to a high number (e.g. 100 years) if you want to capture all of the entries.

[Carravan]

Ah, removed it from the config after I found that it's counterproductive for my use case. My issue with it is that it would ignore waiting rooms older than 24h, even if the stream starts in the future. Obviously, I would want those recorded and not skipped. Guess that can't be avoided with the information available in the RSS feed. I figured that removing it from the config would remove that filter. Set to 999h and the vods were added. Thanks!