Lock ups when accessing through Samba from Windows

[nschlia]

When a file is accessed from a Windows machine for some reason the tail of the file is accessed directly, e.g. "read 14212 bytes from 693174272". These are exactly the last 14212 bytes of the file. The number differs from file to file, but it's always exactly the number of bytes remaining.

This happens on the very CreateFile() API call which is very odd. That call should only open a file but never attempt to read anything. This read access seems to time out after 10 seconds, and the file open operation succeeds.

That causes a big problem: ffmpeg is not notified of the interrupted read and continues to transcode until the data is available, i.e. in the above case up to 690 MB. Alas, because the open does not block the app tries to access the file, but Fuse will not satisfy any read requests before the pending read request has completed. This causes another timeout, the app will receive an EPERM (Permission denied).

Same strange thing happens when Windows Explorer reads a directory. It always access the file's tail. This is a performance hog.

1. Why does Windows access the file tail in that way?
2. How can Fuse/ffmpegfs be notified of the interrupted read?
3. Why does Fuse not allow access to another part of the file while a request is pending? It simply does not call fuse_read again while an earlier call blocks and waits for data.

[nschlia]

For 2./3.: This was a combination of a bug in ffmpegfs - if a read operation waited for data, it blocked subsequent open calls. This was not Fuse problem.

After removing the block concurrent access worked fine locally, but not on a Samba share. This required a configuration change for that Samba share:

When accessed via Samba the pending read locked the whole share, causing Windows Explorer and even KDE Dolphin to lock up. Any access from the same machine to that share was blocked, Even "ls" was not possible and blocked until the data was returned.

Seems others had the same problem:

http://samba.2283325.n4.nabble.com/Hangs-Accessing-fuse-filesystem-in-Windows-through-Samba-td4681904.html

Adding this to the [global] config in smb.conf fixes that:

 oplocks = no
 aio read size = 1

The "aio read size" parameter may be moved to the share config:

 aio read size = 1

[nschlia]

Still need to find the reason for 1.

[nschlia]

Adding this to the [global] config in smb.conf fixes that:

 oplocks = no
 aio read size = 1

The "aio read size" parameter may be moved to the share config:

 aio read size = 1

Should actually be:

  oplocks = no
  level2 oplocks = no
  aio read size = 1

to avoid "Level II oplocks can only be set if oplocks are also set." warnings.

[nschlia]

For 1:

Seems that the requests actually take place via Samba (using log level = all:4):

~#  tail -f /var/log/samba/* | egrep "Accept" 
  norbert opened file ProRes/video/Musik/Accept & Orchestra/Metal Heart (Live Wacken 2017)/Accept & Orchestra - Various.webm read=Yes write=No (numopen=4)
  smbd_smb2_read: fnum 3520236500, file ProRes/video/Musik/Accept & Orchestra/Metal Heart (Live Wacken 2017)/Accept & Orchestra - Various.webm, length=65536 offset=0 read=65536
  smbd_smb2_read: fnum 3520236500, file ProRes/video/Musik/Accept & Orchestra/Metal Heart (Live Wacken 2017)/Accept & Orchestra - Various.webm, length=1048576 offset=20971520 read=1048576
  smbd_smb2_read: fnum 3520236500, file ProRes/video/Musik/Accept & Orchestra/Metal Heart (Live Wacken 2017)/Accept & Orchestra - Various.webm, length=65536 offset=146276352 read=65536
  norbert closed file ProRes/video/Musik/Accept & Orchestra/Metal Heart (Live Wacken 2017)/Accept & Orchestra - Various.webm (numopen=3) NT_STATUS_OK

Samba actually completes the read at 146276352. Strange....

The sequence is

1. Open file on Windows -> smbd_smb2_read: fnum 3520236500, file ProRes/video/Musik/Accept & Orchestra/Metal Heart (Live Wacken 2017)/Accept & Orchestra - Various.webm, length=65536 offset=0 read=65536
2. Read 1M at 20MB offset -> smbd_smb2_read: fnum 3520236500, file ProRes/video/Musik/Accept & Orchestra/Metal Heart (Live Wacken 2017)/Accept & Orchestra - Various.webm, length=1048576 offset=20971520 read=1048576
3. Close file -> smbd_smb2_read: fnum 3520236500, file ProRes/video/Musik/Accept & Orchestra/Metal Heart (Live Wacken 2017)/Accept & Orchestra - Various.webm, length=65536 offset=146276352 read=65536##

At ffmpegfs:

1. Open file on Windows -> Requests 65536 bytes at 0 and 146276352 offsets, blocks 30 seconds
2. Read 1M at 20MB offset -> Requests 1M at 20M
3. Close file -> Blocks until the 64K at 146276352 arrive

Predicted/final size: 146320094/171225961 bytes, diff: 24905867 (117.5%)

By comparison, a physical file on disk:

norbert opened file Accept & Orchestra - Various.webm read=Yes write=No (numopen=4)
  smbd_smb2_read: fnum 75796220, file Accept & Orchestra - Various.webm, length=65536 offset=0 read=65536
  smbd_smb2_read: fnum 75796220, file Accept & Orchestra - Various.webm, length=65536 offset=171180032 read=45929
  smbd_smb2_read: fnum 75796220, file Accept & Orchestra - Various.webm, length=1048576 offset=20971520 read=1048576
  norbert closed file Accept & Orchestra - Various.webm (numopen=3) NT_STATUS_OK

Same behaviour, obviously nothing for the Fuse system, Windows reads 65536 bytes a offset 0 and 171180032 just when the file is opened. Why???

[nschlia]

Testing revealed that the Windows fopen (the underlying CreateFile) call causes a 64K read at the first and last 64K boundary. Offset 0 and 20971520 (which is exactly 320 x 65536) in the above case. No matter what the file size is, the read is at the last 64K boundary. Tested with several dozen files. So once the file is opened, before any call to read, these two blocks are access.

Why is Windows doing that?!?

[nschlia]

Added an experimental parameter "--win_smb_fix=1" that may work around the lock problem.

I seems Windows, right when fopen is called, accesses the file at the first and last 64K boundary, namely at offset=0 and somewhere at the end less than 64K before the end. That causes the fopen call to block until the file is transcoded so far, or at least about 30 seconds (seems Windows times that out).

Very strange, especially because fread was not called, fopen only, Does not happen if accessed from Linux (via Samba), but I could confirm this with Windows 7 and 10. It's not a Samba problem, I can see the reads in the Samba debug logs, comes from the SMB client.

Anyway, if you use the --win_smb_fix=1 parameter ffmpegfs will simply return 0 bytes if the client tries to read 64K at the last 64K boundary. Works for me.

[nschlia]

Seems this was not enough, other Windows installation access at 8K boundaries, so the logic must be improved:

If the client does the following:

1. Open the file and access immediately at offset 0.
2. Right next access the file at a 8K multiple offset. The remaining size must be less than the largest divisor that is a multiple of 8 K.

Then ignore the second request. Any further requests must be honoured.

Example:

File size 623240 bytes. First offset = 0. Second offset = 622592 (this is 76 x 8K) The remaining size is 648 bytes which is less than 8K -> ignore request

or...

File size 623240 bytes. First offset = 0. Second offset = 589824 (this is 9 x 64K) The remaining size is 33416 bytes which is less than 64K -> ignore request

[nschlia]

Optimised code OK.