search

spl0k / supysonic

Supysonic is a Python implementation of the Subsonic server API.
https://supysonic.readthedocs.io
GNU Affero General Public License v3.0
259 stars 57 forks source link

daemon conflict between watcher an scanner #218

Closed vincentDcmps closed 3 years ago

vincentDcmps commented 3 years ago

hello I have meet yesterday by accident a bug in the daemon. I use beet to manage my library I have add plugin subsonicupdate to launch a scan at the end of import. when beets move file to my library daemon's watcher detect move and launch a scan on these file put at the end of the beets process when beets get startscan URL daemon raise en error:

juin 10 23:02:59 oscar python[89870]: RuntimeError: dictionary changed size during iteration
juin 10 23:02:59 oscar python[89870]:     for path in self.__queue.keys():
juin 10 23:02:59 oscar python[89870]:   File "/srv/supysonic/supysonic/watcher.py", line 230, in unschedule_paths
juin 10 23:02:59 oscar python[89870]:     self.__queue.unschedule_paths(path)
juin 10 23:02:59 oscar python[89870]:   File "/srv/supysonic/supysonic/watcher.py", line 284, in remove_folder
juin 10 23:02:59 oscar python[89870]:     self.__watcher.remove_folder(folder.path)
juin 10 23:02:59 oscar python[89870]:   File "/srv/supysonic/supysonic/daemon/server.py", line 104, in __unwatch
juin 10 23:02:59 oscar python[89870]:     self.__on_folder_start(folder)
juin 10 23:02:59 oscar python[89870]:   File "/srv/supysonic/supysonic/scanner.py", line 124, in __scan_folder
juin 10 23:02:59 oscar python[89870]:     self.__scan_folder(folder)
juin 10 23:02:59 oscar python[89870]:   File "/srv/supysonic/supysonic/scanner.py", line 110, in run
juin 10 23:02:59 oscar python[89870]:     self.run()
juin 10 23:02:59 oscar python[89870]:   File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
juin 10 23:02:59 oscar python[89870]: Traceback (most recent call last):
juin 10 23:02:59 oscar python[89870]: Exception in thread Thread-71:

i think that watcher add some file to the queue during the for but I don't know why because threading.condition object is used each time that __queue is invoke and this should be avoid this type of issue isn't he?

here supysonic daemon log :

2021-06-10 23:02:55,287 [INFO] Scanning: '/mnt/diskstation/music/twenty one pilots/Trench/09 - Nico and the Niners.ogg'
2021-06-10 23:02:55,301 [INFO] Scanning: '/mnt/diskstation/music/twenty one pilots/Trench/10 - Cut My Lip.ogg'
2021-06-10 23:02:55,315 [INFO] Scanning: '/mnt/diskstation/music/twenty one pilots/Trench/11 - Bandito.ogg'
2021-06-10 23:02:55,331 [INFO] Scanning: '/mnt/diskstation/music/twenty one pilots/Trench/12 - Pet Cheetah.ogg'
2021-06-10 23:02:55,344 [INFO] Scanning: '/mnt/diskstation/music/twenty one pilots/Trench/13 - Legend.ogg'
2021-06-10 23:02:55,359 [INFO] Looking for covers: '/mnt/diskstation/music/twenty one pilots/Trench'
2021-06-10 23:02:55,366 [INFO] Scanning: '/mnt/diskstation/music/twenty one pilots/Trench/14 - Leave the City.ogg'
2021-06-10 23:02:55,421 [DEBUG] Freeing scanner
2021-06-10 23:02:59,133 [DEBUG] File created: '/mnt/diskstation/music/twenty one pilots/Trench/cover.jpg'
2021-06-10 23:02:59,139 [DEBUG] File modified: '/mnt/diskstation/music/twenty one pilots/Trench/cover.jpg'
2021-06-10 23:02:59,207 [DEBUG] Received <supysonic.daemon.client.ScannerStartCommand object at 0x7f86686d44c0>
2021-06-10 23:02:59,218 [DEBUG] Received <supysonic.daemon.client.ScannerProgressCommand object at 0x7f86686d0340>
2021-06-10 23:02:59,227 [INFO] Scanning folder music
2021-06-10 23:02:59,228 [INFO] Unscheduling watcher for /mnt/diskstation/music
2021-06-10 23:20:42,181 [DEBUG] Got signal 15. Stopping...
vincentDcmps commented 3 years ago

maybe because here we modifyng the iterator in the loop

     for path in self.__queue.keys(): 
           if path.startswith(basepath):                                   
             del self.__queue[path]