owncloud / music

:notes: Music app for ownCloud
GNU Affero General Public License v3.0
567 stars 198 forks source link

Podcasts not updating automatically #1044

Closed JoshuaPettus closed 1 year ago

JoshuaPettus commented 1 year ago

Hi, I'm noticing a few of my podcasts channels aren't updating automatically (most do nterestingly). I have to manually hit Reload before it pulls the next episode. Is there a command that I can add to crontab to make it check every now and then?

Using NC25.0.3, Music 1.8.1, php 8.1, nginx, pgsql

paulijar commented 1 year ago

By default, the Music app should check for updates for each channel once per 24 hours. The time is counted always from the latest update, meaning that that each channel may be in different phase of the 24 hour cycle. You may change this schedule by defining the key music.podcast_auto_update_interval in settings.php, giving the interval as hours.

You may also use the command occ music:podacast-update to update the podcasts right now. You probably could call this from cron but I don't see why this would work any better than the updates made automatically by the Music app.

JoshuaPettus commented 1 year ago

It is strange, for two of my podcasts it just does not update, even after 72 hours. I knew that the podcast should be there as it was on their respective sites. But it would only appear once I manually update. I never touched that variable.

https://www.scoringnotes.com/feed/podcast and https://feeds.megaphone.fm/

I'll try that command for jokes

paulijar commented 1 year ago

Things to be looking at:

  1. Is there anything in nextcloud.log at the time when the channel update was supposed to happen, i.e. 24 hours after your latest manual update.
  2. Does the "updates checked" get updated in the podcast channel details. image
JoshuaPettus commented 1 year ago

Thank you! I'll keep an eye on that next time they push out another podcast and see If i can get that information for you.

JoshuaPettus commented 1 year ago

Actually just looking at at that, it doesn't look like that it is getting updated everyday. The 10th was when I last did that manual check. I know there isn't a new podcast yet, but shouldn't it be checking all the same?

Screenshot 2023-02-13 113625

paulijar commented 1 year ago

Indeed, it should be checked and the "updates checked" date should reflect this, regardless of if there actually was any updates on the feed. Maybe some unhandled exception has happened while updating another channel before this one, and the update process was cut short before reaching this channel? There should be something in nextcloud.log if this has happened.

If the root cause is the one mentioned above, then it should happen also when you run the occ podcast-update command.

JoshuaPettus commented 1 year ago

Ooh that's good to know and makes sense occ podccast-update should have run at least once since I had stuck it in cron for a day. I'll use that to trigger the event and check in the log when I have a moment.

JoshuaPettus commented 1 year ago

Weird, looks like the command ran without a problem. I had it incorrect on the cron which was why it failed there....forgot the --all or --group sudo -u www-data php ./occ music:podcast-update --all Annoyingly there were no errors. I may just leave that command back in cron and call it a day. Chalk it up to the IT gods hate me.

martin-nohava commented 1 year ago

Hi, I have exactly the same problem, podcasts aren't updating automatically (I have only 2 and both fail to auto-update). I didn't find any error in the error log. The podcast details were last updated more than 48h ago. (manual update works)

Environment: Nextcloud 25.0.3, Music v1.8.1

paulijar commented 1 year ago

@martin-nohava Do you know for fact that the background task in your Nextcloud instance actually runs? Because if none of the channels get updated, then it could also be a symptom of the background task not running regularly.

JoshuaPettus commented 1 year ago

For the record, mine is set to run every 5 minutes in cron and seems to do so. It use to be 15 min but nextcloud changed their recommendation to 5.

martin-nohava commented 1 year ago

Hi, @paulijar, thank you for your reply. I let Music run for a few days and the podcasts never updated. Nextcloud cron.php is run successfully every 10 minutes.

Adding the command occ music:podcast-update --all directly to cron works reliably without errors.

qudiqudi commented 1 year ago

Just to clear things up, occ music:podcast-update --all shouldn't be needed in a seperate cron if the Music app is working as intended? because it seem to not auto updateing my podcasts as well. 24h cycle is always long over and no new updates appear in the Music app although my podcatcher on my phone does have them.

qudiqudi commented 1 year ago

Background tasks work as intended, though podcasts are not updated. I suspect that my nightly AIO-Borg backup restarts the nextcloud container, which than resets the 24h cycle, so the app actually never comes to updating the podcasts like scheduled. Setting the key music.podcast_auto_update_interval may circumvents this but editing the config php may break the AIO install, so it'd be better to implement a GUI element to change the update interval.

JoshuaPettus commented 1 year ago

That's a thought. Maybe, I'll have to go though my scripts to find if I'm restarting the services somehow. But I really don't think so. I run on bare metal, and backup nightly off of an LVM snapshot so it really isn't necessary for me to restart any services.

paulijar commented 1 year ago

Just to clear things up, occ music:podcast-update --all shouldn't be needed in a seperate cron if the Music app is working as intended?

That's right.

I suspect that my nightly AIO-Borg backup restarts the nextcloud container, which than resets the 24h cycle

This shouldn't cause any issues because the logic behind the 24 h cycle works like this:

  1. Whenever the channel is checked for updates, the timestamp of this latest check is saved.
  2. Whenever the background job runs, it checks if the difference between the latest check time and the current time is more than 24 h. In other words, there's no separate timer which need to be running constantly but everything is based on the real time clock of the system.

The real source of the problem seems to be that the background tasks of the Music app do not get triggered at all on the recent version(s) of Nextcloud, even when the background jobs on the cloud-level have been set up correctly. This is the case at least on NC26 but it may have started several major versions ago since this isn't something I would have been testing systematically. The reason probably is changes made by Nextcloud on their API.

JoshuaPettus commented 1 year ago

Yah, I'd say at least 25. 24 I'm pretty sure was working. Not too surprising as 24 - 25 seemed a big jump all around. I even had to update the nextcloud webserver file as they had made major breaking changes between the two from what their earlier recommendations once were. Possibly they made those changes earlier, but it all came to a head at v25. Took me a long time to figure out that what it was too..... Not that I'm bitter.

Out of curiosity, what other background tasks could be effected?

qudiqudi commented 1 year ago

Would the music app appear under occ background-job:list if it were to be operating normally? Because for me it doesn't, I'm on Nextcloud 25.0.5 (AIO install).

paulijar commented 1 year ago

Out of curiosity, what other background tasks could be effected?

In addition to the podcast updates, the Music app has a cleanup task which is supposed to be run on the background. It removes any orphaned database entries like albums which don't have any tracks or (Subsonic) bookmarks pointing to tracks which don't exist. The same thing can be done manually by running occ music:cleanup. Furthermore, the background cleanup task removes any obsolete Ampache sessions from the DB; this part is not covered by the said occ job.

Would the music app appear under occ background-job:list if it were to be operating normally?

Most likely. But this occ command is not available on ownCloud or on older versions of Nextcloud so it can't be tested on a system where the Music background jobs actually work.

qudiqudi commented 1 year ago

According to @JoshuaPettus finding someone with NC 24 would be sufficient.

JoshuaPettus commented 1 year ago

According to @JoshuaPettus finding someone with NC 24 would be sufficient.

I think he is saying NC 24 doesn't have occ background-job:list, that may be a new feature. Anyway, we know it's not working on NC25 and 26. Don't need a command to tell us that

paulijar commented 1 year ago

This is now fixed in Music v1.8.3, hopefully. See the commit message for more detailed explanation of the root cause.

In the new version, you should see the two background tasks of the Music app also on occ background-job:list. However, there is a small gotcha: By default, the command lists only 10 first jobs. To see them all, give the command e.g. the argument --limit=100.