Enhancement request: EMS memory support

[pawosm-arm]

I've managed to buy lo-tech's 2MB EMS memory 8-bit ISA card for my Amstrad PC 2086. It exposes EMS protocol version 4.0 and works lovely under FreeDOS (although I had to replace default command line interpreter with 4DOS which can utilize EMS memory for its internal data freeing up a chunk of so precious main memory). Things that didn't work before now started to work thanks to that!

I guess gaining access to some extra 2MB RAM could boost ELKS dramatically. It is accessed through 64kB memory window which can be configured to start at selected address in the upper memory address space (usually E000h). The DOS driver and its source code (8086 assembly) is available here: https://www.lo-tech.co.uk/downloads/2MB-EMS-Board/LTEMM-r01.zip

[ghaerr]

I investigated EMS pretty carefully several months ago when I was adding the configurable external L2 buffers, and came to the conclusion that, because of the amount of code and special handling required to access extended memory, that such a thing didn't really help ELKS that much. This is because, unlike upper memory (UMA - between 640k and 1MB), extended memory (EMS) cannot be used to run ELKS applications in, but instead only contain more L2 external system buffers. The extended system buffering would help speed by not requiring disk I/O in some cases, of course, but no more applications could be run, and ELKS is already set to only 16 processes max for other reasons.

In order to access EMS, the A20 line has to be toggled, and in addition the CPU must be put into protected mode, which is only supported by 80286+ processors. The whole thing is pretty complex, and would require more kernel space (we're down to 8k left currently) that it's worth, IMO. The other issue we have with ELKS is that the kernel can't even access the L2 external buffers directly, all access has to them has to be copied in and out to yet another local set of L1 buffers, since the kernel doesn't use any far pointers. A close examination of a full read/write sequence to disk or the network shows way more copying than is needed for efficient operation. This would want a rewrite also, before implementing EMS.

[pawosm-arm]

@ghaerr, your description of the challenges fits to the XMS (exTended memory) which indeed requires 286+, not EMS (exPanded memory) which came to the market much earlier and works with any 8088/8086 (and later) PC. The LTEMM driver for DOS is for all of the PC computers, including old XT's and it definitely does not require protected mode. Even Windows 3.0 (which was the last Windows version to support EMS) could make use of EMS memory only in real mode (started by win /r).

[ghaerr]

@pawosm-arm, thanks for your correction on my XMS/EMS misunderstanding. I took at quick look at your EMS driver source and it is still pretty complicated. EMS's utility is still debatable based on it's use only through a 64k window, although mapping even that 64k memory into the system (as upper memory) could be useful with the memory manager modifications mentioned in #709. Can the UMB card and EMS card be used simultaneously, or is there no space in the UMA?

[pawosm-arm]

Can the UMB card and EMS card be used simultaneously, or is there no space in the UMA?

They can be used both simultaneously and I'm using them both, but it's at cost of available UMB of course. In case of FreeDOS it's no big deal, with all the drivers, buffers, stacks and lastdrive loaded into this reduced UMB, I'm still seeing 11kB of free UMB, leaving most of the 640kB of main memory for the regular DOS programs.

EMS's utility is still debatable based on it's use only through a 64k window, although mapping even that 64k memory into the system (as upper memory) could be useful

Funnily, Commander Keen 4 game, which could not start before I've installed this EMS card, detects EMS and reports that its size is... 64kB.

[pawosm-arm]

@ghaerr seems like I applied modern thinking to an ancient solution (which EMS is). Those DOS programs that use EMS are written with full awareness of what kind of memory they're using, it's not an easily accessible memory we got used to in nowadays userspace programming. If we wish to replicate this approach in ELKS, we'd have to expose it through dedicated syscalls for allocating/deallocating page-sized chunks of memory (and for accessing them too). Other approaches to EMS were:

• 4DOS called it a swap, although it wasn't available for DOS programs, only built-in DOS commands could make use of it
• 8-bit CP/M Plus could create ramdisk in EMS-like memory (big memory visible through a small window) available for the later 8-bit platforms

I don't know if swap is planned for ELKS (surprisingly, FUZIX has it implemented also for no-mmu platforms), I guess doing ramdisk on EMS would be easier to achieve, the best solution would be the one which combines both (something like swap on ramdisk).

[Mellvik]

I think you're spot on, pawosm-arm.

ESM, maybe even LIM based RAM disk, would be a welcome addition to ELKS. And, like you point out, can be used for swapping, increasing the process space.

—Mellvik

@ghaerr https://github.com/ghaerr seems like I applied modern thinking to an ancient solution (which EMS is). Those DOS programs that use EMS are written with full awareness of what kind of memory they're using, it's not an easily accessible memory we got used to in nowadays userspace programming. If we wish to replicate this approach in ELKS, we'd have to expose it through dedicated syscalls for allocating/deallocating page-sized chunks of memory. Other approaches to EMS were:

4DOS called it a swap, although it wasn't available for DOS programs, only built-in DOS commands could make use of it 8-bit CP/M Plus could create ramdisk in EMS-like memory (big memory visible through a small window) available in the later 8-bit platforms I don't know if swap is planned for ELKS (surprisingly, FUZIX has it implemented also for no-mmu platforms), I guess doing ramdisk on EMS would be easier to achieve, the best solution would be the one which combines both (something like swap on ramdisk).

—

[pawosm-arm]

OT: Some hint on availability of those lo-tech cards (namely, XT-IDE, UMB memory, 2MB EMS memory). Although lo-tech is British, they don't produce their ISA cards (anymore), all of their cards I've got, I've imported them from the USA, there's a small company in Texas selling on e-bay both DIY kits and ready to use ISA cards built according to the lo-tech design.

[ghaerr]

Closing this as EMS implemented through definable UMB blocks in #1590. Bank-switched EMS is unlikely to ever be implemented in ELKS.