Free Pascal for DOS (go32v2 extender) compilation error

[nickysn]

I'm trying to use DOSBox-X for the development of the DOS (go32v2 extender) version of Free Pascal. I'm running DOSBox-X 0.83.11 compiled from source (using configure --enable-sdl2 --disable-avcodec) in Fedora 33.

Steps to reproduce:

1. execute the command "VER SET 7 0" to enable long file names

2. install Free Pascal 3.2.0 for DOS (go32v2), using the installer package dos320full.zip from:

   https://sourceforge.net/projects/freepascal/files/DOS_Go32v2/3.2.0/

   Install all packages, using the installer, including the compiler sources. Make sure long file names are enabled, otherwise some packages won't be able to be installed. Add the needed bin directories to the PATH, as recommended by the installer.

3. change the current directory to C:\pp\source (assuming the compiler is installed in C:\pp)
4. execute "make all"

this stops with an error, similar to: ... make.exe[4]: Makefile:76: pipe: Bad file descriptor (EBADF) make.exe[4]: Makefile:139: pipe: Bad file descriptor (EBADF) make.exe[4]: Makefile:2817: pipe: No such file or directory (ENOENT) c:/pp/bin/go32v2/make.exe rtlclean rtl make.exe[5]: Entering directory c:/pp/source/compiler' make.exe[5]: Makefile:76: pipe: Bad file descriptor (EBADF) make.exe[5]: Makefile:139: pipe: Bad file descriptor (EBADF) make.exe[5]: Makefile:2817: pipe: No such file or directory (ENOENT) c:/pp/bin/go32v2/make.exe -C clean make.exe: Entering an unknown directory make.exe: *** clean: No such file or directory (ENOENT). Stop. make.exe: Leaving an unknown directory make.exe[5]: *** rtlclean] Error 2 make.exe[5]: Leaving directoryc:/pp/source/compiler' make.exe[4]: [next] Error 2 make.exe[4]: Leaving directory `c:/pp/source/compiler' make.exe[3]: [ppc1.exe] Error 2 make.exe[3]: Leaving directory c:/pp/source/compiler' make.exe[2]: *** [cycle] Error 2 make.exe[2]: Leaving directoryc:/pp/source/compiler' make.exe[1]: [compiler_cycle] Error 2 make.exe[1]: Leaving directory `c:/pp/source' make.exe: [build-stamp.i386-go32v2] Error 2

C:\pp\source>

The above steps result in successful compilation in a Windows 98 DOS prompt or inside 32-bit Windows XP's command prompt.

[Wengier]

I can confirm this issue. I noticed that the makefile in the c:/pp/source/ directory will call the makefile in the c:/pp/source/compiler directory. However, calling the makefile in the c:/pp/source/compiler directory directly appears to work.

[grapeli]

@Wengier Yes, compilation is not successful, but my error message is completely different (maybe because I'm using the latest version of dosbox-x from git).

pp.pas(266,1) Warning: Object aoptx86.o not found, Linking may fail !
pp.pas(266,1) Warning: Object aoptx86.o not found, Linking may fail !
pp.pas(266,1) Error: Can't open object file: aoptx86.o
pp.pas(266,1) Fatal: There were 3 errors compiling module, stopping
Fatal: Compilation aborted

Full log here. go32v2-dosbox-x.txt

The compilation of this Go32v2 package in Win98SE under Qemu succeeds, although at the end there is some bug with "clean_bootstrap". Full log here. go32v2-win98se.txt

Under Freedos 1.2 with doslfn the compilation attempt failed.

[nickysn]

I have tested it under 32-bit Windows XP and it compiles without any errors. I can attach a build log, if it helps.

In principle, Free Pascal for DOS (go32v2) snapshot building is supported in Windows 95/98/ME and Windows 2000/XP, but maybe there's a regression with Windows 98 in Free Pascal 3.2.0. I admit I haven't tried building it in Windows 98.

32-bit Windows Vista and later have a hard limit on max memory used by DOS applications, so that's why they don't work. There's a possibility to lift the limit in Windows Vista (I forgot how), but not in Windows 7 and later.

FreeDOS with doslfn is not supported and has never worked, in any of the times I have tried it. I don't know why, but I presume it's due to doslfn bugs?

[grapeli]

@nickysn It's good that you provided some information. In the documentation (in the file pp/source/docs/buildfaq/buildfaq.tex) I found only a mention of this type: "Go32v2 is dos based, and used via the dos compatibility system of Windows ..."

Rather, the long filename incompatibility fails under Freedos (doslfn). Probably also under DOSBox-X.

[grapeli]

In my opinion, the reason for the failure is the included version of rm.exe. I swapped it for another. Taken from here. ftp://ftp.delorie.com/pub/djgpp/current/v2gnu/fil41br3.zip I am adding an additional one because this ftp is very slow, it is leaking slowly. rm.zip

After unpacking and installing dos320full.zip and replacing rm.exe from pp/bin/go32v2 directory, I start compiling.

SDL_AUDIODRIVER=dummy SDL_VIDEODRIVER=dummy ./dosbox-x -set nosound=true -set mididevice=none -set sbtype=none -set pcspeaker=false -set memsize=256 -set startbanner=false -set fastbioslogo=true -set output=ttf -set cputype=pentium -set core=dynamic_x86 -set cycles="max 105%" -set lfn=true -set ver="7.0" -set logfile=./dosbox-x.log -c "mount c /tmp/fpc" -c "c:" -c "config -securemode" -c "set path=c:\pp\bin\go32v2" -c "cd pp\source" -c "make all 2>LOG2" -c exit 2>/dev/null You have to be patient. I am looking at the log. tail -F fpc/pp/source/LOG2 This time the compilation comes to an end.

The size of memsize=256 is certainly exaggerated.

Edit: Hmm. Otherwise. Compilation goes to the same stage as under Win98. The compiler - ppc386.exe is built.

[grapeli]

I was able to redirect stdout and stderr to one file. Two logs from fpc compilation run under dosbox-x and win98se. fpc-dosbox-x.txt fpc-win98se.txt

In both cases it reaches the same stage. ppc386.exe is identical in size. The only differences are the warning messages under dosbox-x (not critical).

rm: cannot remove directory `i386/units/i386-go32v2': Permission denied (EACCES)
rm: cannot remove directory `i386/units': Permission denied (EACCES)
make.exe[7]: *** Warning: File `c:/pp/source/rtl/units/go32v2' has modification time in the future (2021-03-17 21:32:34 > 2021-03-17 21:32:24)
warning:  Clock skew detected.  Your build may be incomplete.

And the duration of the compilation is 6 minutes and 40 minutes.

[Wengier]

@grapeli and @nickysn Thanks for the tests, which are certainly very valuable. I am not entirely sure what the "dos compatibility system of Windows" (mentioned above) means, but it is possible that it uses a separate layer specifically designed for running in the Windows environment (which I assume it means NTVDM as used by NT-based systems). DOSBox-X is of course a DOS emulator, with support for Windows 9x which is DOS-based, so we do want to emulate the behavior as in a real DOS environment (including with DOSLFN loaded) and also in a Windows 98 DOS prompt environment. According to the above reports the compilation also fails under DOSLFN and by replacing rm.exe the compilation may go to the same stage as in the Windows 98 DOS environment. On the other hand, emulating the NTVDM environment as used by Windows XP is not a priority for DOSBox-X at this time simply because Windows XP is not DOS based.

[nickysn]

It means either the Windows 98 or the Windows XP dos environment. I don't know why do you even consider DOSLFN for a reference implementation, since its goal is to be compatible with one of the Microsoft LFN implementations (i.e. Windows 95/98/ME or the NTVDM based ones - 2000/XP/Vista/7/8/8.1/10), which are the real reference implementations. The fact that you needs to replace rm.exe with a different version to reach parity with Windows 98 DOS prompt environment means there's an incompatibility between DOSBox-X and Windows 98 that is exposed with the older version of rm.exe. So, if Windows 98 LFN is the compatibility goal of DOSBox-X, it means there's still an incompatibility. Sure, we can upgrade rm.exe in the next FPC version. In fact, we should probably do so. Now, it's probably too late for FPC 3.2.2rc1, since the go32v2 installer has already been built (although it's not public, yet), but it can still be done for rc2 and for the 3.2.2 final release, so I see no reason not do it. However, the fact that you need to update rm.exe still means there's some incompatibility between DOSBox-X and the Windows 98 LFN environment and that could be worth investigating, since fixing that might fix bugs in other tools as well. I can understand why the Windows XP DOS environment is not the goal for DOSBox-X compatibility and I respect that. In fact, Free Pascal only supports Windows XP NTVDM because it has to. NTVDM has significantly more bugs and has historically needed many workarounds in Free Pascal for go32v2. Overall, Windows XP's DOS emulation is not good at all, so it's hardly a worthy goal.

[Wengier]

@nickysn A full compatibility with Windows 98's LFN support is definitely DOSBox-X's goal. Windows 98 is based on MS-DOS 7.x which DOSBox-X does try to fully emulate (such as FAT32 support, etc). So I agree that some further investigations are desired. However, please note that DOSBox-X is a DOS emulator, not a MS-DOS or Microsoft implementation emulator, which means that its DOS emulation is not limited to MS-DOS or Microsoft's implementations. DOSLFN may be considered a reference implementation because it has already become the de-facto standard LFN driver for pure DOS systems in the DOS world. On the other hand, Windows XP is not DOS-based, and its DOS emulation is pretty bad and can not be used as a real reference implementation.

[nickysn]

I'm quite happy that DOSBox-X aim to achieve compatibility with Windows 98's LFN support.

As for DOSLFN - I guess it can be considered a reference implementation as well now, but for some reason it has never worked with Free Pascal for DOS snapshot building and I consider this a bug (or several bugs?) in DOSLFN. We're willing to add workarounds for Windows 95/98/ME and (unfortunately, due to the large number of NTVDM bugs and ugly workarounds) Windows 2000/XP/Vista, because a lot of people use them and because Microsoft wouldn't fix the bugs in their DOS emulation. But adding workarounds for other DOS platforms and emulations makes the compiler and RTL more difficult to maintain and test, and there's quite a substantial number of them already for the various memory managers, DPMI hosts, operating systems, such as OS/2, etc.

If DOSLFN had a bug tracker, I would report this as a bug there also. It would be quite cool if I could build snapshots with long file names in real DOS also, but unfortunately I can't and it has never worked :(.

[Wengier]

@nickysn DOSLFN does not have a bug tracker at this time, but you can report issues to its author(s) via email as I did many times in the past (especially in the 2000s). DOSLFN has become stable because of feedbacks of its users over the years (IMO, the LFN emulation in DOSLFN is may be better than the one in Windows 2000/XP/Vista; Free Pascal works with the latter only because of various ugly workarounds).