Possible deadlock in filesystem operations

[alpine9000]

If you really push the filesystem from an Amiga program (reading hundreds of files repeatedly), fs-uae will sometimes freeze the Amiga thread performing the operation.

deterministic=1 seems to prevent the issue from happening, also so far I have not been able to reproduce this issue on real hardware or WinUAE.

I have attatched a test case which is a simple Blitz Basic program that reads a file 300 times. It also includes a test script that repeatedly calls the test program.

To reproduce the issue, enter the BT directory and call the test script "execute test"

It might take hundreds of calls to the test program (so many minutes and thousands of filesystem operations) before the issue will be triggered.

Once the thread is locked, accessing the "About" menu item from the Workbench menu will unfreeze it.

We have reproduced this issue on Mac, Windows and Linux versions of fs-uae.

fs-uae-test.zip

[FrodeSolheim]

Thanks for the testcase, I haven't tried it yet, but a testcase such as this is very valuable to fix the problem :)

Some background information regarding deterministic=1 - in this case, the processing is not done by a thread - but instead synchronously in the emulation thread. That also slows done the processing since we do not want to keep the emulation thread too busy, so it could be that the issue is not triggered simply due to the slowdown in that case.

[ztronzo]

I can confirm this problem, and it DOES NOT happen with netplay (which enables deterministic mode by default)

When it happens, it halts the disk activity and I am not sure what is affecting it. I also used an HDD LEDs software from Aminet to see disk activity from software if I can detect when it happens in the Amiga end, but when it happens the LEDs remain on as still showing HDD activity. Happens always during a disk process (that may have completed earlier without a problem), such as reading or listing lots of files. I have noticed this will happen using a directory folder as a hard-drive. Could it be a storage controller timing or buffer issue?

This will happen at totally random, does not matter if its after a clean emulation start or after a hard-reset. This happens for me when using directory hard drives and using a fast Amiga, increasing the CPU speed using uae_cpu_throttle and makes it more likely to happen at higher speeds, the higher the setting the more likely it may freeze (but still at random).

Happens more often in general when I load games with lots of files that also demand speed, like Blood, Rise of the Triad, Doom, Quake etc. Doesn't matter if its 020 or 040, AGA or RTG, FPU or NO-FPU or even have JIT enabled or disabled. But it matters when using a throttled 020 or an 040 because the higher you push the emulation the more often the problem appears. Doesn't happen with WinUAE as I tested with version 3.6.1

Does not freeze when using HDF images (need confirm by 2nd source) or using RAM disk (confirmed). My workaround for this is to have a large RAM: disk and copy files there before I launch them, that way if a freeze happens I will spot it at start, and once copied to RAM: it wont when using the files from there.

Since even the copy to RAM: will freeze because of the disk read it may get stuck too, currently using parameter BUFFER=2000 during the copy command which seems to help reduce the freeze, but does not eliminate the issue.

I do believe it has to do with either the drive buffers (which cannot be set on directory hard-drives), or with the Storage/IDE controller. Also the buffer size cannot be applied to directory folders.