Can't open for write without O_TRUNC on existing file without --vfs-cache-mode >= writes

[rkdavies]

Hello I'm inheriting an already existing OmniStream setup and unfortunately, the person who put it together cannot be of any help. I literally inherited the service.

About a week ago, sabnzbd stopped downloading items so I started to dig into things. Below is the rabbit hole I jumped down that has led me to this issue.

2023-04-25 02:15:51,511::INFO::[assembler:121] Traceback:
Traceback (most recent call last):
  File "/app/sabnzbd/sabnzbd/assembler.py", line 86, in run
    self.assemble(nzo, nzf, file_done)
  File "/app/sabnzbd/sabnzbd/assembler.py", line 178, in assemble
    with open(nzf.filepath, "ab", buffering=0) as fout:
PermissionError: [Errno 1] Operation not permitted

-------------
Inside container
-------------
df -h

-------------
uploadcache:unsynced:Gdrive  1.1P  346G  1.1P   1% /cloud

-------------
Process Owenrship
-------------
abc          132 44.7  0.9 193576 158564 ?       Ssl  02:12   2:05 python3 /app/sabnzbd/SABnzbd.py --config-file /config --server ::
abc          468  4.4  0.0   5304  2220 ?        S    02:13   0:10 /usr/local/bin/unrar x -vp -idp -scf -o+ -ai -p- /cloud/Downloads/sabnzbd/MEDIA.FILE.H264.2/yay.rar /cloud/Downloads/completed/movies/_UNPACK_MEDIA.FILE.H264.2/
abc          587  0.9  0.0   5300  2216 ?        S    02:14   0:01 /usr/local/bin/unrar x -vp -idp -scf -o+ -ai -p- /cloud/Downloads/sabnzbd/MEDIA.FILE.H264.1/MEDIA.FILE.H264.rar /cloud/Downloads/completed/movies/_UNPACK_MEDIA.FILE.H264/

-------------
Permissions in path
-------------
root@sabnzbd:/cloud/Downloads/sabnzbd# ls -alF
total 84
drwxrwxrwx 21 abc users 16384 Apr 25 02:14 ./
drwxrwxrwx  3 abc users  4096 Apr 25 02:14 MEDIA.FILE.H264.1/
root@sabnzbd:/cloud/Downloads/sabnzbd# ls -alF MEDIA.FILE.H264.1/
total 3190408
drwxrwxrwx  3 abc users      4096 Apr 25 02:14 ./
drwxrwxrwx 21 abc users     16384 Apr 25 02:14 ../
drwxrwxrwx  2 abc users      4096 Apr 25 02:16 __ADMIN__/
-rw-rw-rw-  1 abc users      3320 Apr 25 02:14 MEDIA.FILE.H264-sample.par2
-rw-rw-rw-  1 abc users    387388 Apr 25 02:14 MEDIA.FILE.H264-sample.vol00-01.par2
-rw-rw-rw-  1 abc users       466 Apr 25 02:14 MEDIA.FILE.H264.nfo
-rw-rw-rw-  1 abc users     48552 Apr 25 02:14 MEDIA.FILE.H264.par2
-rw-rw-rw-  1 abc users 650000000 Apr 25 02:14 MEDIA.FILE.H264.r00
-rw-rw-rw-  1 abc users 650000000 Apr 25 02:14 MEDIA.FILE.H264.r01
-rw-rw-rw-  1 abc users 650000000 Apr 25 02:15 MEDIA.FILE.H264.r02
-rw-rw-rw-  1 abc users 650000000 Apr 25 02:15 MEDIA.FILE.H264.r03
-rw-rw-rw-  1 abc users  15360000 Apr 25 02:16 MEDIA.FILE.H264.r04
-rw-rw-rw-  1 abc users   1128257 Apr 25 02:14 MEDIA.FILE.H264.r12
-rw-rw-rw-  1 abc users 650000000 Apr 25 02:14 MEDIA.FILE.H264.rar
-rw-rw-rw-  1 abc users       700 Apr 25 02:14 MEDIA.FILE.H264.sfv

--------------------------
Permissions looked good the whole way through, from inside the container I could touch, edit, delete files with no problem. So I checked rclone.log

--------------------------
root@server2:/home/schuster/OmniStream/logs# tail rclone.log
2023/04/25 02:16:20 ERROR : Downloads/sabnzbd/MEDIA.FILE.H264.1/MEDIA.FILE.H264.r02: WriteFileHandle: Can't open for write without O_TRUNC on existing file without --vfs-cache-mode >= writes
2023/04/25 02:16:22 ERROR : Downloads/sabnzbd/MEDIA.FILE.H264.1/MEDIA.FILE.H264.r03: WriteFileHandle: Can't open for write without O_TRUNC on existing file without --vfs-cache-mode >= writes
2023/04/25 02:16:22 ERROR : Downloads/sabnzbd/MEDIA.FILE.H264.1/MEDIA.FILE.H264.r03: WriteFileHandle: Can't open for write without O_TRUNC on existing file without --vfs-cache-mode >= writes

--------------------------
I can find the rclone process that has vfs-cache turned off, but I can't find where it is launched from to modify.
--------------------------
root      100589  4.4  4.2 2240604 691384 ?      Sl   Mar29 1697:14 /usr/bin/rclone mount --allow-other --dir-cache-time 96h --gid 1000 --uid 1000 --log-level WARNING --log-file /logs/rclone.log --poll-interval 10s --rc --rc-no-auth --umask 000 --user-agent omnimount --drive-pacer-min-sleep 10ms --drive-pacer-burst 200 --vfs-cache-mode off --config /config/rclone.conf Gdrive:/ /mnt/Gdrive```

But looking at the df info above from the container, it appears some other caching may be in place? Which I'm not sure about either.

Thank you in advance for any help or direction you can point me to!
Rob