Problems with color attributes, incorrect colored output, using the OS/2 port.

[johnsonjh]

Hi Bill!

I'm having a bit of trouble trying to get PdCursesMod working as part of a 16-bit DOS 80286-specific protected mode build (of the G Editor which I currently maintain.)

I'm successfully using PdCursesMod's DOS and Windows ports, so I know the application and PdCursesMod get along fine.

For this 286-specifc build, I'm using OpenWatcom 2.0 (March 13 2022) on Linux, as a cross-compiler, and a current checkout of PdCursesMod master.

Now, I don't want to bore you with all the details (because the whole setup is pretty insane), but, what's important to know is that I'm using the Phar-Lap 286|DOS-Extender SDK, which has it's own API but also provides an OS/2 compatibility layer, which is what I'm using for the console output (and thus, I'm using the PdCursesMod OS/2 port).

So, even though this is a MS-DOS program, the toolchain, as far as PdCursesMod is concerned, is OS/2 1.x.

I'm using (slightly modified) OWC 16-bit OS/2 1.x headers and libraries for this. (Before I go further, nothing here would be the culprit - I only had to get rid of the clipboard functionality of PdCursesMod, because it's not exposed to me, unfortunately — nor is it handled in the Phar Lap runtime.)

Now, I did read the note on your website (saying that the OS/2 support is pretty much untested lately), but, I'm really hoping that you'll be able to at least point me in the right direction!

It looks like it's just as (simple?) as the attributes not being set correctly … perhaps this is because PdCursesMod is assuming an OS/2 2.x console, which I imagine is more featured? I think the OS/2 1.x console may provide just 8 or 16 colors?

This is the broken output that I get:

For comparison, the correct output looks like this (the exact same code but built using the PdCursesMod's DOS port, targeting MS-DOS 386 protected mode):

The application actually is working 100% fine (besides the text being green / inverted / black - it's actually there). I should add, if I don't call start_color(), then I can't see anything at all. The displayed text is black on black. Using PDC_RGB doesn't make any difference. I also tried defining CHTYPE_32 on a whim, but that results in no output at all.

Here is the compiled binary, though I'm not sure if that will help you much: G286.ZIP. You'll more than likely encounter problems running this on any emulation (perhaps other than on QEMU.) In particular, it won't work in DOSEMU2 or similar. Something like Bochs may work.

Now, if I do get it working, I'll be happy to document what it takes to get PdCursesMod going in this configuration and what changes were needed.

[Bill-Gray]

Hmmm... the following comes with the warning that I have no idea when OS/2 support on PDCursesMod was most recently tested, on either 1.x or 2.x. Things could have been broken for some years without my even knowing about it. (I've never been able to test it myself. I tried getting a "current" OS/2 -- I think Embarcadero? -- running a few years back, without success.) Which means that on occasion, changes have been made to the OS/2 port completely without testing of any sort.

I'm not sure, but I believe @wmcbrine has mentioned testing PDCurses on OS/2. If so, I think you'd have better odds getting that to work. It certainly would be worth a try... in fact, if it did, it would suggest to me that I perhaps should just use William's current code, possibly with some bits and bobs borrowed from my code. If you give it a try, please let me know what you find. Looking at the PDCurses and PDCursesMod code for OS/2, it does not look to me as if reconciling them would be all that big of a deal.

Both PDCurses and PDCursesMod provide for eight or sixteen colors in OS/2. I've no idea if later OS/2 flavors allowed for more colors in the console.

It should work with either 32-bit or 64-bit chtypes.

[johnsonjh]

Thank you - I'll check it later tonight - I need a break after fighting this all day!

As I don't actually have "real" OS/2 either, it's possible that Phar Lap's implementation of the OS/2 console code is imperfect as well.

To rule this out, I've now built G as a real native 16-bit OS/2 1.x console application, too. While I can't actually test it myself, I've put the word out to some people who have OS/2 systems, and should hear back on it later today or tomorrow.

I really hope the problem happens there too, so I can completely rule out the Phar Lap SDK as the culprit!

It's kind of shocking how little modern "love" exists for 286 systems in 2022. There is almost unlimited resources on the internet about every random 8-bit home system, 8086 systems, 386 systems, etc. but almost nothing for 286 protected mode programming. Tenberry/Rational (the DOS/4G(W) people) had a 286-product called DOS/16M, seemingly lost to the ages now, however, from my research, Phar Lap 286 is as good as it gets for this sort of thing. I've been using it with great success, other than this issue.

It's fun to see it in action:

G provides a both a line mode and a visual editor, like vi's ex-mode, and I've been able to confirm the Phar Lap port works great, with 15+MB data resident, using DOS on a 286.

I'll give you a full update as soon as I hear back about the status of the OS/2 build executing on a real OS/2 system, as well as my result with PdCurses.

Edit: Here is a native 16-bit OS/2 1.x build, for the remote possibility that someone who might be reading this thread wants to try it out and verify the output has the same problem on real OS/2, as I'm still waiting to hear back from my OS/2 people: G_OS2.ZIP

[johnsonjh]

(moved to https://github.com/Bill-Gray/PDCursesMod/issues/233)

[Bill-Gray]

I can definitely see issues with the current PDCursesMod OS/2 code. There are places where it's relying on bits being in places where they no longer are. I may be best off just using William's code as a starting point. He's actually cleaned some stuff up in it, to the point where I can follow the logic of it (which was not always the case before).

[johnsonjh]

I can definitely see issues with the current PDCursesMod OS/2 code. There are places where it's relying on bits being in places where they no longer are.

That's good news for me, since PdCursesMod is fixed more easily than the "black box" of the Phar Lap runtime.

I may be best off just using William's code as a starting point. He's actually cleaned some stuff up in it, to the point where I can follow the logic of it (which was not always the case before).

As soon as I try it, I'll let you know if it's working!

[johnsonjh]

@Bill-Gray Sorry for the delay ...

I can report that using the Phar Lap 286 runtime for the console I/O does work fine with PDCurses OS/2 console code. I'd prefer PDCursesMod, so if there is anything I can do to help you test updates to your OS/2 port (with the disclaimer that I have only the Phar-Lap runtime and not "real" OS/2), let me know.

[johnsonjh]

@Bill-Gray You are on my schedule today, I see - I didn't refresh the page before my response so I didn't see your commit.

I'll test this out now.

[johnsonjh]

@Bill-Gray I can confirm that https://github.com/Bill-Gray/PDCursesMod/commit/bd39b61128828abfd96db7c091aaa7508965ca53 is working for me!

I did have to make two minor alterations (as with PDCurses as well) to use this code with the 286 Phar Lap DOS runtime. These are trivial changes, and wouldn't be needed on "real" OS/2, so I don't think anything is warranted on your side, but I'll leave the details here for the record:

1. There is no clipboard implemented in Phar Lap, so I don't build pdcclip.c, which requires lightly massaging getch.c to not use that functionality.
2. This new code calls DosQuerySysInfo in the PDC_ms_count() routine in pdcutil.c. I don't have that either. I see that's only used for blinkon to set the blinking rate - I don't need that so I just stubbed it out.

I'll do a bit more testing, but this looks good.

[GitMensch]

@Bill-Gray I suggest to add two defines - #ifndef NO_CLIPBOARD + #ifndef NO_BLINK, to enable disabling those. The pdcurses files should still compile but the externalized (or internally used) functions return ERR and have an assert in - so the callers don't need to be adjusted.

I've also seen people that didn't want any blinking and possibly others that at least wouldn't are about the clipboard (or would like to explicit disable the option; if that's possible at all), so this may not be OS2 specific, but "useful for this specific exotic scenario, too".

[Bill-Gray]

Turns out I was able to get eComStation working on an older machine. It couldn't mount a USB drive, but I burned the executables onto a CD and read that, and... they work. (Note that this means "I ran testcurs and its various tests did what they were supposed to do.")

I only had to change about a dozen lines of code relative to William's PDCurses code (none of which need to go upstream). Specifically,

• We're no longer using SP->key_code, so I removed those lines.
• We're now using acs_defs.h instead of acs437.h.
• PDC_color_content() and PDC_init_color() now work with integers, not shorts.
• PDC_set_resize_limits() is PDCursesMod-specific (no PDCurses counterpart).
• PDC_port_val didn't exist in PDCurses.

All rather minor. As mentioned, it compiles in OpenWATCOM, and quite likely with Borland 5.02 or earlier, but I only have 5.5. However, between my results and yours, it's good enough to cause me to merge the 'os2' branch in to 'master'.

[Bill-Gray]

@GitMensch For the clipboard, there are a couple of possibilities. DOS lacks a clipboard concept, so the DOS port has an effectively "internal" clipboard : you can store text to it and retrieve it, but not share it to another process. Even so, you could imagine wanting to copy some text from the screen of your program and then paste it elsewhere within the same program (for a text editor, for example).

Since it just allocates and deallocates memory, it's not OS-dependent. We might have a pdcclip.c that lives in the common directory; other pdcclip.cs could start with

#ifdef NO_CLIPBOARD
   #include "../common/pdcclip.c"
#else
    ('real' clipboard code for the OS in question)
#endif

../include/pdcclip.c would basically just be the current DOS pdcclip.c (which in turn might become the single line #include "../common/pdcclip.c".)

"Normally", blinking can be suppressed (converted to a color intensity) via PDC_set_blink( FALSE). For the issue mentioned by @johnsonjh, we should surround the calls to DosQuerySysInfo with #ifndef _PHAR_LAP_ or whatever the appropriate #define is. (And similarly for the other places where Phar Lap has shortcomings.)

[GitMensch]

Sounds good, but I'd suggest to replace NO_CLIPBOARD with INTERNAL_CLIPBOARD_ONLY or something like that.

[Bill-Gray]

Afterthought : in pdcutil.c, we can #include <time.h> and then replace the existing PDC_ms_count() function with

ULONG PDC_ms_count(void)
{
    const ULONG now = (ULONG)clock( );
    const ULONG n_seconds = now / (ULONG)CLOCKS_PER_SEC;
    const ULONG fraction  = now % (ULONG)CLOCKS_PER_SEC;

    return( n_seconds * 1000UL + fraction * 1000UL / (ULONG)CLOCKS_PER_SEC);
}

In theory, we could just return clock() * 1000UL / (ULONG)CLOCKS_PER_SEC). In practice, CLOCKS_PER_SEC is frequently already 1000, and that expression would overflow after 2^32/1000000 seconds = 71.6 minutes.

Even with the above, the millisecond count will still overflow after 49.7 days. (Or less, if CLOCKS_PER_SEC > 1000.) The only use of this function is to look for half-seconds elapsing for blinking, and it's quite possible to avoid that particular bug... but the above ought to at least evade the Phar Lap issue and make blinking possible on that platform.

[johnsonjh]

Thanks for all the help on this!

I should mention, while we are on the Phar Lap topic, that the DOS version of PdCursesMod could be usable with Phar-Lap 286 (and 386), but needs some modification because the screen memory is not directly available in protected mode.

The following is how it's done from protected mode for both the 286 and 386 runtimes would look something like:

#include <dos.h>

#define GET_VIDEO_MODE      0x0F

static BYTE far *vid_mem;   /* use far pointer */

#if defined(DOSX286)  /* Phar Lap 286|DOS */
# include <phapi.h>
#endif

int
video_mode(void)
{

    int mode;

    _asm mov ah, GET_VIDEO_MODE
    _asm int 10h
    _asm xor ah, ah
    _asm mov mode,ax

    return mode;                                                           
}                                                                    

unsigned short
get_vid_mem(void)
{
    int vmode = video_mode();
    unsigned seg;

    if (vmode == 7)
    seg = 0xB000;
    else if ((vmode == 2) || (vmode == 3))
    seg = 0xB800;
    else
    return 0;

#if defined(DOSX286) /* Phar Lap 286|DOS */
{
    unsigned short sel;  

    /*
     * DosMapRealSeg() takes a real mode paragraph address and a count of
     * bytes, and gives back a selector that can be used to address the memory
     * from protected mode.  Like all PHAPI functions, DosMapRealSeg() returns
     * 0 for success, or a non-zero error code.  Any other information (such as
     * the selector we're interested in) is returned via parameters.
     */

    if (DosMapRealSeg(seg, (long) 25*80*2, &sel) == 0)
    return sel;
    else
    return 0;
}
#endif

    return seg;
}

video_init(void)
{
    vid_mem = MK_FP(get_vid_mem(), 0);
}

and to clean up:

/*
 * When the program is finished with the selector to
 * video memory, it may freed by calling DosFreeSeg()
 */ 

DosFreeSeg(FP_SEG(vid_mem));

I've attached a more complete example here. I'll look at testing this out in the DOS version soon, once I finished the "OS/2" work.

Edit: That's Borland syntax, but Watcom should be very similar.

[Bill-Gray]

Closing this, as I see we addressed the original issue a while back and digressed. The digression about clipboard handling is preserved in issue #237.

@johnsonjh , if you wind up with some OS/2 or DOS Phar Lap code that you'd like to see merged in, please let me know. (I have vague memories of dealing with protected mode in my first job, circa 1987... mostly a blur now.)

[johnsonjh]

@Bill-Gray

Hi Bill - yes, this is safe to close. When I have Phar-Lap specific code, I'll ping you - but I don't yet - it's still planned.

I am, however, now using PdCursesMod successfully now to produce working OS/2 binaries from my cross-compiler setup, and I'm very happy with the results.

Thanks again.