search

wlerin / showroom

Script for archiving AKB48 Group related Showroom streams
MIT License
59 stars 20 forks source link

unable to set temp directory when save dir and temp dir not in same partition, because os.replace doesn't support #27

Open GingerManiac opened 4 years ago

GingerManiac commented 4 years ago

unable to set temp directory when save dir and temp dir not in same partition, because os.replace doesn't support.

Exception in thread Watcher-3---(--):
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/--/workspace/github/showroom/showroom/core.py", line 487, in run
    self.download.wait()
  File "/home/--/workspace/github/showroom/showroom/downloader.py", line 187, in wait
    self.move_to_dest()
  File "/home/--/workspace/github/showroom/showroom/downloader.py", line 269, in move_to_dest
    destpath = self._move_to_dest(self.outfile, self.tempdir, self.destdir)
  File "/home/--/workspace/github/showroom/showroom/downloader.py", line 287, in _move_to_dest
    os.replace(srcpath, destpath)
OSError: [Errno 18] Invalid cross-device link: '/tmp/active/200621 Showroom - Oshi 180426.mp4' ->
   '/home/--/Downloads/Showroom/2020-06-21/Oshi/200621 Showroom - Oshi 180426.mp4'

i'm using seperated partitions and wish to set temp dir on /tmp on sda1 and save dir on /home on sdb1, but i couldn't do that, cuz of what i wrote above. hope to be changed shutil.move rather than os.replace, then it will be fixed i think, if you don't mind, please.

wlerin commented 4 years ago

Honestly, you should probably just make sure your temp and save dir are on the same device. If they aren't the script will not perform as well.

Yes, changing os.replace to shutil.move would be easy and is probably what I should have been doing anyway, but same device moves are near-instantaneous. Copying from one device to another is not. The recording thread will be held up during this transfer, and if it's a false stop that means you're missing part of the recording while it copies over.

I could open a separate thread for the move, I guess. That's probably fine for the master branch, though the current working branch (hls-python) has too many threads as is.

wlerin commented 3 years ago

I've gone ahead and implemented this with 3f1a509d5d525add37f922fc72b2f319a5d60bd3 , but I'm going to leave the issue to remind myself both to address it on the hls-python branch, and to do something to avoid the copy blocking.