ecm-pushbx
commented
1 year ago EDR-DOS does not use the fields named fcb_dirclst and fcb_startclst for its search data. In fact, the attempt at restoring the DTA that is prevented from working by the third bug is innately wrong because the DosFindNext call may call down into the redirector and we can't know exactly which of the reserved fields in the search data are used by the redirector.
EDR-DOS simply copies most of the reserved fields of the DTA into the search FCB. This presumably enables concurrent FCB searches even on redirected drives though I did not test this yet.
This is in the function fcb_save_search_state, the entirety of populating the search FCB from the DOS v2 style Find data DTA:
lea di,MSF_NAME[bx] ; ES:DI -> FCB name
mov si,offset fcb_search_buf
lodsb ; get 1st byte = drive info
mov cx,20/WORD ; copy 20 bytes to FCB
rep movsw ; (the rest of the search template)
stosb ; drive info byte follow them(The remainder of this function populates the user DTA with the result data.)
The reverse is in the function fcb_restore_search_state which is called before the equivalent of the DosFindNext call:
mov di,offset fcb_search_buf+1
; ES:DI -> internal state
lea si,1[bx] ; DS:SI -> FCB+1
mov cx,10
rep movsw ; copy info from FCB
lodsb ; get "drive" info
mov es:fcb_search_buf,al ; it's the 1st byte in the srch state
I set up two tests of the FCB FindFirst/FindNext functions on a FAT32 FS hard disk image. All tests are run with qemu, version
QEMU emulator version 5.2.0 (Debian 1:5.2+dfsg-11+deb11u2). The first two test runs were on a current revision of Enhanced DR-DOS, while the latter two test runs were on the FreeDOS kernel.All files can be found in https://pushbx.org/ecm/test/20230821/
These are the commands used to create the disk images: https://pushbx.org/ecm/test/20230821/commands.txt Reproduced here:
These scriptlets use NASM, my macro collection, my FAT FS format script written in NASM, dosemu2 (only for running FreeDOS/EDR-DOS
syson the diskette image), qemu, lDebug, the EDR-DOS system build, and the FreeDOS kernel and FreeCOM build. Also included are HimemX and a number of smaller utilities, not needed for this test but used for prior tests. The test programstestnorm.comandtestexte.comwere created in the debugger so there's no source files corresponding to them.The qemu serial ports are connected to a
socatinstance created using this command:Which in turn is connected to a
picocomrunning like so:Each test was performed by running the DOS command
ldebug \testXXXX.comthen a debugger commandinstall serial. On the serial terminal, a "KEEP" confirmation is entered and finally three commands to the debugger (visible in the serial port logs) that go likere.append @if (word [cs:cip - 2] == 21CD) then d psp:80 l 28thentp FFFFthenq. The RE buffer append command sets it up so that after tracing an interrupt 21h call (CD 21opcode), the debugger will dump the DTA area at address PSP:80h. This allows it to automatically display what result a file search call yielded.All tests are done by calling FCB FindFirst (function 11h) and FindNext (function 12h) with various states (
\TESTdirectory or root directory, prior search call) as well as a search mask of eleven question marks. This is intended to match any directory entry. Thetestnorm.comtest uses normal FCBs as the search/result structure, whiletestexte.comuses extended FCBs with the attribute 17h (directory, system, hidden, read-only).I will not reproduce the full log here, but it is available on the server at https://pushbx.org/ecm/test/20230821/logs.txt
There are several differences:
This is the first result from EDR-DOS running
testnorm.com, searching using a normal FCB in the\TESTsubdirectory:As you can see, the first returned result is one of the (1-byte) test files.
This is the corresponding FreeDOS test run:
This one returns the dot directory entry of the subdirectory. (Look for the 2Eh text byte at address 0081h in the dump.) Evidently, FreeDOS includes subdirectories in its normal FCB search while EDR-DOS does not. (I checked this assumption by doing some more test runs, not depicted in the log. Repeatedly having EDR-DOS search the root directory without an extended FCB does return
ZEROES.BIN,TESTNORM.COM, andTESTEXTE.COMbut not theTESTdirectory.)The testexte.com test reveals that EDR-DOS is able to reconstruct the search DTA from the FCB passed to it, whereas FreeDOS is unable to do that. This is from the EDR-DOS run:
The second, interleaved search starts in the root directory and finds
ZEROES.BIN. Then, the prior search (in the subdirectory) is resumed and findsB, the second non-dots directory entry. Thus, EDR-DOS succeeds in our test.Next, consider the FreeDOS result of the same test:
Trying to resume the subdirectory's search, we instead get the second directory entry of the root directory. The kernel uses its static buffer and is unable to resume a concurrent search, where EDR-DOS (and presumably other 8086 DOS compatibles) is able to do just that.
Also note that this is on a FAT32 file system, so perhaps EDR-DOS is uniquely capable. The interrupt list notes that FCB open only works on FAT32 volume labels rather than files. (Probably they were thinking of FCB create rather than open, actually.) In the Find First FCB description there is no corresponding note, but it is possible that it does not work on a FAT32 FS on MS-DOS 7.10+ either. (I quickly tested it, and it appears that FCB Open succeeds on the FAT32 FS image on EDR-DOS. Did not test actually reading or writing the file.)
Other than the attribute bug (which I assume indicates a bug in the FreeDOS kernel with the EDR-DOS kernel's choice being correct), there appear to be two bugs in the FreeDOS kernel:
There are two
UWORDfields in the FCB structure's reserved parts that are calledfcb_strtclstandfcb_dirclst: https://github.com/FDOS/kernel/blob/29ccb6e45414a8a18346a97fdb0f35ebd6fb4025/hdr/fcb.h#L95Searching for "clst" in the file fcbfn.s reveals that these two fields are only used by the FCB Find functions, not any usual FCB file operations it appears. (The normal FCB file accesses depend entirely on the stored SFT handle number to point to the same, still valid FCB SFT entry.) Both fields are written once and read once: https://github.com/FDOS/kernel/blob/29ccb6e45414a8a18346a97fdb0f35ebd6fb4025/kernel/fcbfns.c
Here, in the early part of FcbFindFirstNext:
And here, a bit further down:
There is one problem immediately apparent here, though the third bug actually completely masks this (second) bug: The
dm_dirclusterfield of the DTA (dmatch) structure is a 32-bit dword for a FAT32-enabled kernel build. But it is truncated to, respectively zero-extended from, a 16-bit word in these accesses. I will study EDR-DOS's approach at a later point but cursory tests seem to indicate it stores the complete 32-bit directory cluster number in the search FCB's structure.The third bug is in this line: https://github.com/FDOS/kernel/blob/29ccb6e45414a8a18346a97fdb0f35ebd6fb4025/kernel/fcbfns.c#L658
It reads
if (First). This condition is reversed from what it should be. The (static) dmatch buffer used by the kernel should be updated from the search FCB if this call is a Find Next. The way this condition is written, it will "update" the dmatch fields from the (uninitialised) search FCB only before the DosFindFirst call, which I believe has no effect whatsoever. Before a DosFindNext call, the dmatch structure is never re-initialised from the search FCB.This did not cause readily apparent bugs because the kernel's static dmatch structure used by the FCB find functions will still hold the prior call's data when an FCB Find Next call happens. However, as in my test with the interleaved concurrent searches, this is not what was intended and actually does constitute a bug whenever more than one search is in progress in an application.
I will prepare patches to these three bugs at a later time. The third bug probably also requires to re-order the
wAttruses a bit.