Feature request: please add support for Global EMM Import Specification (GEMMIS)

[volkertb]

Very few expanded memory managers have support for GEMMIS. That's unfortunate, because it is required for proper compatibility with Windows 3.x and Windows 9x. It's a standard/specification that allows proper memory management handover to Windows when it's started from DOS.

Adding GEMMIS support to Jemm386 and JemmEx would make them more flexible and versatile. Notably, it would allow FreeDOS to run Windows 3.x (even in 386 Enhanced Mode) out of the box.

DOSBox currently has a GEMMIS implementation, so its source code could be studied as a reference for implementing this in Jemm.

[Wengier]

I agree with adding support for GEMMIS too, which is pretty useful. As mentioned above DOSBox already has a GEMMIS implementation. In fact, you can see the changes in DOSBox's ems.cpp directly related to GEMMIS support here:

https://sourceforge.net/p/dosbox/code-0/2601/

The actual change for GEMMIS support is apparently smaller than one might think, so I think it can be done for JEMM386 too.

[volkertb]

@Baron-von-Riedesel How feasible do you think it would be to implement this in Jemm? In case the DOSBox commit that @Wengier linked to doesn't provide you with enough context, here is an old article from Dr. Dobb's Journal in which someone documented the specification in detail, after he reverse-engineered it himself, including code examples:

Dr. Dobb's Undocumented Corner September 01, 1994.pdf

The direct link to the article is https://www.drdobbs.com/windows/undocumented-corner/184409317, but for some reason, the layout on that site is messed up and makes it difficult to read. Also the print icon on that page is hidden in modern web browsers, unless you have the NoScript browser add-on enabled.

Anyway, from what I understand from the article, implementing this handover mechanism doesn't seem too complicated. But you can judge that far better than I can. What do you think?

[HyabusaBadge]

Hello. It's a very interesting thread. Is it possible to make FreeDOS EMM386.EXE compatible with GEMMIS? I know FreeDOS EMM386.EXE is old and unmaintained. There is FreeDOS (98), which is a port of FreeDOS to the NEC PC-9800 series. JEMM has not been ported, but FreeDOSEMM386.EXE has been ported. I also think freedos' EMM386.EXE is a great resource for understanding GEMMIS.

[volkertb]

@HyabusaBadge It begs the question though, would it really be worth the effort implementing GEMMIS in the older EMM386 implementation? Wouldn't the effort be much better spent on porting JEMM to the NEC PC-9800 platform, as well as adding GEMMIS support to that? Perhaps the differences between FreeDOS for IBM PC clones and FreeDOS 98 are so small, that porting JEMM to the latter might be fairly trivial?

I know this is an open source project, and we shouldn't have any expectations, but I'm curious to hear from @Baron-von-Riedesel about his opinion on how challenging it would be to implement GEMMIS support, whether to JEMM or the older FreeDOS EMM386. And perhaps other people reading this have the expertise to help him with it?

I'm only just getting started with serious assembly language programming, dabbling with it during my spare time. Perhaps eventually, once I've gained enough experience with it, I'll be able to help with this, but until then, I believe this is beyond my skills.

[Baron-von-Riedesel]

I admit that I'm not very interested in this GEMMIS. I fail to see why it should be useful.

[bugnano]

Its usefulness would be that JEMM386/JEMMEX could be a 100% drop-in replacement for MS-DOS EMM386, in all its documented and undocumented cases (such as the ability to run Windows in 386-enhanced mode), which aligns with the goal of FreeDOS, that is to be a 100% drop-in replacement for MS-DOS

[Wengier]

Well said @bugnano! I wanted to use JEMM386 as a free replacement for EMM386 in MS-DOS, but for this reason alone it apparently cannot replace EMM386 which works with Windows 3.1 much better because of this support.

[volkertb]

I admit that I'm not very interested in this GEMMIS. I fail to see why it should be useful.

Well, first of all, I absolutely respect that JEMM is something you work on voluntarily and as a hobby. and it would be unreasonable and pointless to demand of you to implement this. JEMM already is a cool and useful project for many people as it is, and we're thankful that you've shared it with the world.

It all comes down to the lack of GEMMIS being a remaining unchecked box towards JEMM being a full replacement for EMM386 and QEMM386. Many people do want to play with Windows 3.x and Windows 9x. It would be great if Windows could work with FreeDOS and JEMM out of the box. Lack of GEMMIS support (I believe) is the one thing preventing that from working right now. (Anyone feel free to correct me if that's not to case!)

Yeah, Windows 3.x/9x might still work with FreeDOS if you rebooted without JEMM being loaded, but that's a bit of a hassle, espcially if you have to reboot again afterwards. GEMMIS support makes all of this much more seamless.

But if you don't feel inspired to work on this functionality, would you be open to accepting pull requests for this from others?

[Baron-von-Riedesel]

...would you be open to accepting pull requests for this from others?

My experiences - so far - with patches from unknown people are "mixed". So I'm rather sceptical.

[volkertb]

Would it really be that disruptive to add, though? Even if you don't care about the feature much, it's basically a control handover protocol, so it wouldn't require any drastic changes to the overall architecture of JEMM, right? I'm sure a lot of FreeDOS users would really appreciate this functionality.

[t9999clint]

Would this just fix Win 3.1? or would it fix other things too? I would love it if I didn't have to use emm386 for a bunch of my TSRs like SoftMPU and stuff. Way to many unnecessary reboots. ADLiPT added a Jemm driver, but a lot of these TSRs either aren't actively worked on anymore or they don't want to bother learning a new API and "just use emm386" seems to be their only response.

Andreas, Regardless of your decision on this thanks for all your work on this an your other projects (I use most of them). I'm no programmer so I don't want to make unreasonable demands.

[volkertb]

@t9999clint GEMMIS is specifially for being able to start Windows 3.x (in its optimal 386 Enhanced Mode) and Windows 9x from (Free)DOS with JEMM loaded and returning back to DOS when you exit Windows. When you have an EMM loaded that puts DOS in the V86 mode supported by 386 CPUs, but doesn't support GEMMIS, you would have to reboot the computer without the EMM first before being able to start Windows (unless you're running Windows 3.x in Standard mode or Real mode, which gives you reduced functionality and performance). And then after exiting Windows, you'd have to reboot again, if you were to require the EMM for other DOS software.

GEMMIS is supported by both the standard EMM386 versions that come with later versions of MS-DOS, as well as by Quarterdeck's QEMM386. Microsoft only shared the specification with a select number of software companies, but it was later reverse-engineered and documented in the Dr. Dobb's Journal article that I shared in this earlier comment further above.

[t9999clint]

Oh, so it's just for Windows. Sorry for the pointless post then. Does OS\2 need this as well?

On a sidenote I opened a feature request ticket with FreeDOS to add unloading of emm386 to make it easier to switch between jemm and emm386 (fdos version). Has anyone put in a request for Gemmis support with the FDOS guys yet? I couldn't find a open ticket for one, but I am bad at looking up tickets with their weird system

[Baron-von-Riedesel]

GEMMIS is specifially for being able to start Windows 3.x (in its optimal 386 Enhanced Mode)

Yes, that's also what I know. And IIRC, Win3.1 in enhanced mode, isn't that limited to FAT16 and short filenames? What applications do actually need this ( and won't run in Win95/98?).

[bugnano]

FYI I found a document that could possibly help us implementing GEMMIS:

http://dgi_il.tripod.com/gemmis.txt

[volkertb]

Oh wow, that article @bugnano shared is very interesting! In addition to providing extra technical details on how to implement GEMMIS, it also states an additional use for it besides being able to start Windows 3.x/9x from DOS:

This problem becomes even more important today that the organizers of The Assembly 95 said in a pre-invitation textfile that at this party the demos will have to run with EMM386.EXE loaded. This sucks badly, however in this article you have the solution.

So apparently the demo scene was forced to adopt GEMMIS in the development of their demos as well, at least if they wanted to compete at Assembly 95. (I guess simply using a DOS extender instead of taking over full 386 memory management was somehow considered a no-no in this scene?)

@Baron-von-Riedesel If you don't care much about implementing this just for the sake of Windows compatibility, perhaps this would be a more compelling argument for you? :slightly_smiling_face:

[murkyness]

Any updates on this?

The FreeDOS Manifesto says the intention is to "eventually, create another implementation of MS-DOS," so I'm surprised to see pushback on finally being compatible with MS-DOS.

[volkertb]

@murkyness Hey, I agree that this feature is one of the things required to match full feature parity with MS-DOS. But the developer of Jemm is under no obligation to implement it. This is still a voluntary project, and that's okay. :slightly_smiling_face:

[murkyness]

@volkertb yeah of course, and I hope I didn't sound rude.

Just wanted to point out that it aligns with the project goals, so I was confused by the "not interested, fail to see why it would be useful" response. Another solution would be to update the manifesto to eliminate the contradiction.

Either way, I still like this project :grin:

[volkertb]

@Baron-von-Riedesel Do you think it would be feasible for someone to extend Jemm with GEMMIS support through a loadable module (JLM)? That way, you wouldn't have to implement it in Jemm proper, and the desired functionality could still be implemented as an open source solution, without having to write another EMM386 implementation from scratch.

I don't know if Jemm's modular architecture could facilitate such functionality, but perhaps it could be made more flexible to accommodate that, if need be?

[lss4]

A bump on this.

I personally have tried enabling GEMMIS back in DOS 6.22 days with a very ancient PC through MEMMAKER. I think it's mostly for allowing users to go back and forth between Windows and DOS using a single configuration without rebooting, and not really useful for other stuffs. It doesn't really fix any issues with Windows, as expanded memory managers were never required, it's the DOS kernel that needs to be compatible and has workarounds for certain quirks (see AARD code).

GEMMIS needs to additionally reserve about 32KB of memory in the D000 region, I think (in MEMMAKER this is called "Reserved for Windows", specified using the WIN=xxxx-xxxx argument). In addition to the 64KB page frame required for enabling EMS, this will leave you with almost no free upper memory for heavy TSRs, especially on modern systems where usable upper memory is often less than usual, and that you may not have enough upper memory to enable both EMS and GEMMIS if you have installed IDE/SATA add-on cards that come with their own BIOSes, as they also take up upper memory. Should note that additional onboard IDE/SATA controllers (like those from Marvell and JMicron) present on certain motherboards also count if enabled in BIOS. (PS: If you use a Super VGA driver, you can't use I=B000-B7FF as it'll give you errors)

I admit that I'm not very interested in this GEMMIS. I fail to see why it should be useful.

If you don't have problems with using different sets of configurations for DOS and Windows (which is relatively trivial to do with MS-DOS and FreeDOS), and rebooting to switch to the other, then yes, GEMMIS is completely optional. I personally set up several configurations, with the Windows one loading only the necessary TSRs and without any expanded memory managers installed.

On the other hand (off-topic), addressing issues with certain TSRs and games that only works with MS EMM386 should be of higher priority, as MS EMM386 is very dated and does not work with too much RAM (plus, QEMM386 only allows accessing up to 256MB of memory). Don't know if things like I/O port trapping can be made working in a MS EMM386 complaint manner in the future (so existing apps would work).

PS: I don't use SoftMPU, but would it also work in Win3.x, if loaded along with MS EMM386 (GEMMIS enabled)?

[bugnano]

I personally have tried enabling GEMMIS back in DOS 6.22 days with a very ancient PC through MEMMAKER. I think it's mostly for allowing users to go back and forth between Windows and DOS using a single configuration without rebooting, and not really useful for other stuffs. It doesn't really fix any issues with Windows, as expanded memory managers were never required, it's the DOS kernel that needs to be compatible and has workarounds for certain quirks (see AARD code).

Regarding the DOS kernel, the newest FreeDOS kernel is able to make Windows 3.1 work in 386 enhanced mode 😃, so GEMMIS is really not that important after all.

[lss4]

Regarding the DOS kernel, the newest FreeDOS kernel is able to make Windows 3.1 work in 386 enhanced mode smiley, so GEMMIS is really not that important after all.

I haven't got Win3.x working yet. The CI builds were turned out not built with -DWIN31SUPPORT and from the code it still has some incomplete parts. Also, with that flag enabled the kernel currently doesn't compile with GCC, only the Watcom one can be built (I've filed an issue here).

Maybe it's due to other issues, like the amount of RAM I have and the video card. Though for the former, Win3.x's HIMEM.SYS only handles 64MB and works. Still, it doesn't seem to make any difference yet.

[darkstar252]

A bump on this.

I personally have tried enabling GEMMIS back in DOS 6.22 days with a very ancient PC through MEMMAKER. I think it's mostly for allowing users to go back and forth between Windows and DOS using a single configuration without rebooting, and not really useful for other stuffs. It doesn't really fix any issues with Windows, as expanded memory managers were never required, it's the DOS kernel that needs to be compatible and has workarounds for certain quirks (see AARD code

At this point AARD is implemented on FreeDOS.

[roytam1]

DOS 4 is open sourced with MIT License today. which contains an early version of EMM386: https://github.com/microsoft/MS-DOS/tree/main/v4.0/src/MEMM/MEMM

[volkertb]

Thanks for sharing, @roytam1. However, the reason why GEMMIS hasn't been implemented in Jemm yet is not because of a lack of existing open source implementations of it, since DOSBox supports GEMMIS in its integrated EMM as well. But that's written in C, whereas the Jemm source code is written in assembly. Also, even if it was written in the correct language, it wouldn't guarantee that the porting of such code would be trivial.

By the way, I wonder if that version of EMM386 (MEMM?) was perhaps too old to support GEMMIS. Didn't Microsoft develop that standard after DOS 4 was released?