search

owntone / owntone-server

Linux/FreeBSD DAAP (iTunes) and MPD audio server with support for AirPlay 1 and 2 speakers (multiroom), Apple Remote (and compatibles), Chromecast, Spotify and internet radio.
https://owntone.github.io/owntone-server
GNU General Public License v2.0
2.07k stars 236 forks source link

Owntone goes unresponsive after several stop/starts of spocon pipe #1343

Closed r4v5 closed 2 years ago

r4v5 commented 2 years ago

I've been running into an issue that was infrequent before when running forked-daapd on an x64 server running ubuntu 20.04, but seems to be happening more frequently with owntone on ubuntu 20.10 and a few more AirPlay devices.

I mostly use Owntone as a Spotify Connect receiver that splits the audio from spocon out to several Airplay devices in my home. Owntone will frequently (~a few times a week) not start playing when I connect to Spotify Connect. Sometimes I'm able to access the web UI and manually start playback by choosing the pipe in the file browser, and sometimes the web UI goes completely unresponsive as well.

There are 7 AirPlay devices on the network, of which usually 3-4 are enabled. Two are servers running shairport-sync, one is an AirPort Express, three are HomePod Minis, and one is a Mac Mini that just started showing up thanks to the latest Mac OS update. The issue was there when it was just shairport-sync and the AE. Unfortunately, the HomePods' arrival and upgrade to 20.10 were close enough in time that I can't quite pin down one or the other as the proximate cause for its more frequent recurrence.

I also have a HomeAssistant integration configured, and that also stops responding when this happens.

My most recent build was of d6dc923, but this was also present since at least 28.1. Here's two thread backtraces of d6dc923's state as it hanged, first when it didn't detect the pipe opening and then when it went completely unresponsive.

ignoring_pipe.txt stalled_entirely_1636562834.txt

There's a lot of things in the debug logs that look like the internal spotify integration (not spocon) printing auth tokens, so I'm hesitant to attach it here, but I can definitely truncate the existing log and spin up a debug and try to get it into a failed state again if the BTs above aren't helpful.

I haven't had to debug anything in C for a minute, so additional advice about how to get the things that would help you would be great.

ejurgensen commented 2 years ago

At first glance your second backtrace looks like a clear deadlock, which would make OwnTone unresponsive. I'm thinking I should be able to reproduce that and fix it as well.

The issue with the undetected pipe is less clear, the backtrace isn't showing anything unexpected. Since you seem familiar with gdb, what you could do when it fails is print 'pipe_watch_list'. It's a global linked list of the pipes that OwnTone is watching. If you have just one pipe you should see it printed. If the pipe isn't printed then it would be about finding out why the watch wasn't added - not sure how to go about that right now.

r4v5 commented 2 years ago

Awesome, thanks! I'll check the pipe list when/if I can get the spotify one to reproduce.

Also, thanks a ton for the software; it was a neat little self-hosted home automation project for me this summer.

ejurgensen commented 2 years ago

I've added a fix for the deadlock issue now. It would be great if you could take it for a spin to test if it seems more stable.

r4v5 commented 2 years ago

Awesome, thank you! I've just updated; I'll reopen with more diagnostics if it happens again but in the meantime, I'll close this.