drdos, drbio ported to NASM

[ecm-pushbx]

This changeset has the final ident86 report comparing today's drdos.pag with that of 2024-08-02: https://hg.pushbx.org/ecm/edrdos/rev/3d43a9d29b68

Today's build only uses NASM, WarpLink, x2b2, round, compbdos, and convlist.pl to build. Of these some run in DOS: NASM (DPMI) and WarpLink, x2b2, round, compbdos (Real/Virtual 86 Mode), though NASM and the two ltools easily can run on a Linux host too.

Here's a changeset message depicting the scriptlet that I used the most: https://hg.pushbx.org/ecm/edrdos/rev/8a99356109b8

f=misc
cp -a drdos/"$f".nas drdos/"$f".tmp &&
  ~/proj/fixmem/fixmem.pl drdos/*mac drdos/*mac drdos/"$f".tmp drdos/"$f".nas

Another scriptlet was used to update make.bat: https://hg.pushbx.org/ecm/edrdos/rev/2545408cdcfc

perl -i -pe '
if (/JWASMEXE% .*-c/ and not $had) {
  $had = 1;
  my $defines = "";
  while (/(-D[a-zA-Z0-9_]+)/g) { $defines .= $1." "; };
  /([0-9A-Za-z_]+)\.asm/;
  my $file = $1;
  /([\r\n]+)$/;
  my $eol = $1;
  if (not defined $eol or $eol eq "") { $eol = "\n"; };
  $_ = "nasm ${defines}-s -P nasm.mac -f obj $file.nas -l bin/$file.lst -o bin/$file.obj".$eol;
}' drdos/make.bat

Identicalising was done using yet another scriptlet: https://hg.pushbx.org/ecm/edrdos/rev/66d971f99306

f="$(perl -ne '
if (/JWASMEXE% .*-c/) { /([a-z0-9_]+)\.asm/; print "$1\n"; last; }
' drdos/make.bat)"
export f
d="$(perl -ne '
if (/JWASMEXE% .*-c/) {
  while (/(-D[a-zA-Z0-9_]+)/g) { print "$1 "; }; print "\n"; last; }
' drdos/make.bat)"
export d
~/proj/ident86/ident86.py test/drdos.pag drdos/bin/drdos.pag \
  drdos.tls drdos/bin/drdos.map -s -a 256 -S -b \
  'cdexec drdos nasm $d -s -P nasm.mac -f obj "$f".nas \
  -l bin/"$f".lst -o bin/"$f".obj &&
  ./lnk.sh nobiotracelist' -e 2 -d 16 \
  -p '\.lst$::.nas' -p 'bin\/' -c cookie12.dat -E -r | tee text12.e86

Here's a more complete set of scripts: https://hg.pushbx.org/ecm/edrdos/rev/fbf3268b02d2

Identicalised with automatic source edits using this scriptlet:

~/proj/ident86/ident86.py test/drdos.pag drdos/bin/drdos.pag \
  drdos.tls drdos/bin/drdos.map -s -a 256 -S -b \
  'cdexec drdos nasm -s -P nasm.mac \
  -f obj history.nas -l bin/history.lst -o bin/history.obj &&
  cdexec drdos nasm -s -P nasm.mac -P nasmorg.mac \
  -f obj header.nas -l bin/header.lst -o bin/header.obj &&
  cdexec drdos nasm -s -P nasm.mac \
  -f obj pcmif.nas -l bin/pcmif.lst -o bin/pcmif.obj &&
  ./lnk.sh nobiotracelist' -e 2 -d 16 \
  -p '\.lst$::.nas' -p 'bin\/' -c cookie3.dat -E -r | tee text3.e86

Built and compared using this scriptlet:

~/proj/ident86/ident86.py test/drdos.pag drdos/bin/drdos.pag \
  drdos.tls drdos/bin/drdos.map -s -a 256 -S -b \
  'cdexec drdos nasm -s -P nasm.mac \
  -f obj history.nas -l bin/history.lst -o bin/history.obj &&
  cdexec drdos nasm -s -P nasm.mac -P nasmorg.mac \
  -f obj header.nas -l bin/header.lst -o bin/header.obj &&
  cdexec drdos nasm -s -P nasm.mac \
  -f obj pcmif.nas -l bin/pcmif.lst -o bin/pcmif.obj &&
  ./lnk.sh nobiotracelist' -e 2 -d 16 \
  -p '\.lst$::.nas' -p 'bin\/' | tee text4.e86

Noise filtered from the .e86 file using this scriptlet:

cat text4.e86 |
  grep -vE '[0-9A-Fa-f]{6}  first:.*second:[0-9A-F]{2}|first:00 != second:eof|[0-9A-Fa-f]{6} up to below|no difference' |
  less -S

[ecm-pushbx]

ident86 reference: https://hg.pushbx.org/ecm/ident86/ (needs lDebug and a DOS as well)

fixmem reference: https://hg.pushbx.org/ecm/fixmem/ (originally developed in the WarpLink repo, then MSDebug, then carried over into its own repo)

[boeckmann]

I guess it was quite an effort to get all this running. Nice work :) What is your plan regarding the EDR source? Do you plan to only work with the NASM port in the future? If this the case, @mateuszviste and I would have to discuss if we stuck to the JWasm source or import your conversion. But we then would have to rebase some changes we did upon your sources. At least if we want to have them in.

@ecm-pushbx we had a discussion at the FreeDOS video meeting regarding the DRDOS license. Some argue that DRDOS is no CP/M derivate, so the license from July 2022 is not applicable to the EDR sources. Therefore, in consultation with Jim Hall, I wrote Brian Sparks, who pretends to be the current license holder. I also contacted Scott Chapman, who clarified the CP/M license thing in 2022. He told me that Mr. Sparks did not respond to his emails for a long time. So it may be, that we will never get an answer... Btw. is there any chance to contact Udo Kuhnt? I think he would also have to be contacted if we wanted to change the license to for example ISC.

[ecm-pushbx]

I guess it was quite an effort to get all this running. Nice work :)

It was actually very mechanical, convert sources, check conversion, adjust fixmem if need be, convert again, try to build it, adjust fixmem again if it doesn't build, convert again, run ident86 in -e -E -b -r -c mode, and if it did any edits run again as -e -E -b. I had about half a dozen scriptlets using an $f variable as a file basename that did most of the needed steps. (fdos.nas is a little different as it includes utils.nas and funcs.nas so it needs to handle three files at once.)

The only completely manual steps were dropping the placeholder FCB SFT's data, silencing some segment attribute warning (didn't do this for all cases), converting two macros, and changing those macros to .nolist later because they interferred with ident86 -E finding its spot to edit. (It is possible to detect the macro level in a NASM .lst file but convlist.pl doesn't preserve this yet when creating the .tls file.)

What is your plan regarding the EDR source? Do you plan to only work with the NASM port in the future?

Yes, I'd rather. As you know, JWasm isn't considered as free as NASM.

If this the case, @mateuszviste and I would have to discuss if we stuck to the JWasm source or import your conversion. But we then would have to rebase some changes we did upon your sources. At least if we want to have them in.

It's probably simple enough to run fixmem.pl on your changed sources to obtain the updated .nas files.

@ecm-pushbx we had a discussion at the FreeDOS video meeting regarding the DRDOS license. Some argue that DRDOS is no CP/M derivate, so the license from July 2022 is not applicable to the EDR sources.

Yes, I did address such concerns in 2023 August shortly after I started collecting patches already: https://pushbx.org/ecm/dokuwiki/blog/pushbx/2023/0821_late_mid_august_work#enhanced_dr-dos

DR-DOS became free to distribute, modify, and use more than one year ago, with the new license agreement by DRDOS, Inc. on 2022-07-07 which refers to "CP/M and its derivatives". While some have argued that DR-DOS is not a derivative of CP/M, I do think it is. During the last week I happened to notice that EDR-DOS carries copyright statements that reach back to the year 1976. But of course, in 1976 there was neither 86-DOS nor any of its compatible systems yet. So I take this as an additional hint that DR-DOS is considered a derivative of CP/M.

I thought you knew this already to be honest.

Therefore, in consultation with Jim Hall, I wrote Brian Sparks, who pretends to be the current license holder. I also contacted Scott Chapman, who clarified the CP/M license thing in 2022. He told me that Mr. Sparks did not respond to his emails for a long time. So it may be, that we will never get an answer...

That's true. It would be better to get the clarification but I'll continue to distribute EDR-DOS according to the CP/M agreement otherwise.

Btw. is there any chance to contact Udo Kuhnt? I think he would also have to be contacted if we wanted to change the license to for example ISC.

I don't know more than you about that. By my logic he based his EDR-DOS work on the earlier "OpenDOS" license, the only legal path, so with the CP/M license agreement his work also changed status.

[boeckmann]

I thought you knew this already to be honest.

Yes I did, but I asked Jim in the video meeting about the DR-DOS 8.01 licensing things that happened, and discussion shifted to the different opinions if wheather the Juli 2022 license applies to DR-DOS. Based on the source knowledge I also consider it to be a CP/M derivate.

But the best thing would be if Mr. Sparks clarifies this explicitly stating that DR-DOS is under a free license.

[boeckmann]

Because if it was explicitly put under an OSS license the FreeDOS project could reconsider using EDR as an alternate kernel.

[ecm-pushbx]

I finished my port of drbio to NASM. Final report is in https://hg.pushbx.org/ecm/edrdos/rev/20f1471fa5a3 (NB this was a build without the drdos depacker.)

[ecm-pushbx]

Doesn't include your changes to support the no access flag yet, I'll pick and port these at a later time.

[boeckmann]

Doesn't include your changes to support the no access flag yet, I'll pick and port these at a later time.

Yes, better wait a little bit with this until it matured :D

[ecm-pushbx]

By the way, there was another xlatb with a segment override in drdos. This one specified a variable as the operand and the assembler figured out it needed an override. As fixmem doesn't detect these it was added using ident86 -E.

[ecm-pushbx]

Converted here: https://hg.pushbx.org/ecm/edrdos/rev/304c525d9056#l1.424

Override added here: https://hg.pushbx.org/ecm/edrdos/rev/fbf3268b02d2?revcount=120#l1.44

[ecm-pushbx]

I adapted your changes up to https://hg.pushbx.org/ecm/edrdos/rev/ab1918938600 for my NASM port. I'm considering to merge it with your single-file flavour so one source tree can be used to build all variants: lDOS single-file (with/without compressed drbio, drdos), SvarDOS single-file (with/without compression), and SvarDOS dual-file. Would you be interested in that?

[boeckmann]

Would you be interested in that?

Can only speak for me, but this sounds like an interesting idea :) What exactly do you have in mind? Would this mean that you a) provide a means of building a "SvarDOS kernel" through your repository? Or b) do you actually want to do a "full" merge of the repositories to "join forces" and eventually continue development on a single repo? I am open minded to everything, but I doubt that a) would yield much benefit beside holding the two repos more or less "in sync"? But on a second thought, that would make it easier for us to switch NASM, if we decide to do so, as we would not have to re-implement features not imported by you :)

[ecm-pushbx]

Would you be interested in that?

Can only speak for me, but this sounds like an interesting idea :) What exactly do you have in mind? Would this mean that you a) provide a means of building a "SvarDOS kernel" through your repository?

Yes, pretty much.

Or b) do you actually want to do a "full" merge of the repositories to "join forces" and eventually continue development on a single repo?

No, at this point I'd rather keep my hg repo and I suppose you want to stick with git(hub).

I am open minded to everything, but I doubt that a) would yield much benefit beside holding the two repos more or less "in sync"? But on a second thought, that would make it easier for us to switch NASM, if we decide to do so, as we would not have to re-implement features not imported by you :)

Yes, exactingly. I'd only keep the build system and tooling for lDOS flavour kernel, but build from the same sources as you, so picking patches either way would be easier.

[boeckmann]

Yes, exactingly. I'd only keep the build system and tooling for lDOS flavour kernel, but build from the same sources as you, so picking patches either way would be easier.

Yes I am fine with this. So what exactly had to be done? You still have the JWasm and the NASM files in your repo, and the last imports were on both files. Do you plan to continue to do so or eventually ditch the JWasm ones? I assume your recent picks were auto-converted by your tools to NASM?

[ecm-pushbx]

Yes, exactingly. I'd only keep the build system and tooling for lDOS flavour kernel, but build from the same sources as you, so picking patches either way would be easier.

Yes I am fine with this. So what exactly had to be done? You still have the JWasm and the NASM files in your repo, and the last imports were on both files. Do you plan to continue to do so or eventually ditch the JWasm ones?

Yes, if you come along and switch your repo to use NASM too I'd probably drop the JWasm files at a later point.

I assume your recent picks were auto-converted by your tools to NASM?

Mostly, yes. For the buffershigh etc patch it was fairly easy, just hg d | sed -re 's/\.asm/.nas/g' | patch -p1 as the changed parts didn't need any conversion. There were some cases in which I noted down the scriptlet, eg for disk.nas:

Created using this scriptlet with fixmem 383afbcefa87:

cp -a drbio/disk.asm drbio/disk.nas &&
  perl -i -pe 's/UDSC\.BPB/UDSC.UDSCBPB/' drbio/*.nas &&
  cp -a drbio/disk.nas drbio/disk.tmp &&
  ~/proj/fixmem/fixmem.pl drbio/*.mac drbio/*.mac drbio/disk.tmp drbio/disk.nas &&
  hg diff -r bc3e84ed774f^:bc3e84ed774f | patch -p1

In this the https://hg.pushbx.org/ecm/edrdos/rev/bc3e84ed774f is a changeset with a few manual changes, such as the macro conversion. So basically I rerun fixmem and then re-apply all manual changes again afterwards, and this results in a changeset in which only the updates to the converted file are recorded.

Basically the same in https://hg.pushbx.org/ecm/edrdos/rev/42a53efa211f

And another in https://hg.pushbx.org/ecm/edrdos/rev/6ec2d8450426 where I also built the kernel using the JWasm object file (created manually, then linked using lnk.sh) and identicalised it to the NASM port again to make sure the conversion succeeded.

Some more in https://hg.pushbx.org/ecm/edrdos/rev/e440bc3bf3c8