Enhancement ideas

[vipoo]

Hi Wayne,

I have some ideas/desires for enhancement in RomWBW - specifically HBIOS - but i thought, I should just double check with you - and get your ok - instead of just throwing PRs at you.

• [ ] (1) Extend the VDA driver interface to include a port query
• [ ] (2) expose through hbios, the ability to call code in another bank - (HBX_BNKCALL)

Enhancement idea 1

In the sound driver, we had written a function to return the driver's specific ports. It be nice if we could get something similar for the video driver - this would allow applications to be agnostic of port mapping.

Issue is that the VDA's 16 functions range, only has one more entry available. We could use the last remaining entry - but then that kind of limits future expandability.

Perhaps we could use that last remaining entry ($4f), as a subfunction pattern of extended operations. This would make it inconsistent with how we did it in the sound driver. So the sound driver should probably be change to follow a similar pattern - so that there is consistency in approach for querying a driver's port and other potential future operations.

Enhancement idea 2:

The code to call another bank is there - and i did some spiking to prove the concept - where i installed relocatable code in a block allocated in the HBIOS' heap - then called it from my application. Wow - I can access another 10k+ of ram!

Cheers Dean

[wwarthen]

Excellent questions! I also appreciate an opportunity to interact on these before you start generating code.

Regarding idea 1, which I will call more generically a function call to retrieve driver configuration information.

I am a big fan of this and had thought about it a few times. The problem that I always run into is how to do it in a fairly generic way for all drivers.

With respect to the function call, I recommend extending the DEVICE function call. I think all drivers already have a DEVICE call that is pretty consistent and they do not use the HL register. If HL is not zero, the driver could fill the callers memory block pointed to by HL with configuration information. Obviously, this will break caller code, so requires a version bump. However, I don't think there is any use of this function call outside of the RomWBW code.

The bigger question is how to return the configuration information in a simple way that is consistent across all drivers. The sound driver is very simple compared to other drivers, so I think it needs to be carefully thought out. I would like more thoughts on the specifics of that.

One more comment. I am getting very concerned about the size of HBIOS. Solutions to problems like this must be simple and small since they will probably involve adding code to all drivers.

Thanks,

Wayne

[wwarthen]

Oops, I forgot to address idea 2

Yes, making calls into alternate banks does work. I am not exactly sure what you are proposing to change.

I was ultimately hoping to make it possible to make nested calls into arbitrary banks. For example, a user app calls a code block in the HBIOS bank, then that code makes a call into a "secondary" HBIOS bank. I had also hoped to come up with a way to allow banks to be allocated and used through some kind of API so that apps could just use additional banks at will.

Thanks,

Wayne

[vipoo]

Idea 1 - i had thought of the DEVICE function. One of the gaps in my knowledge is the amount of ports and h/w config values the non TMS drivers would need to return.

If its more than can be returned in the available registers, then I guess it would need to implement a pattern of writing to a supplied address - kind of like how the font data can be returned in VDAQRY.

But this second option, requires the caller to set a register to a high address or zero. So I would have thought that to retrofit this approach to an existing function would introduce a backward compatibility issue. Any app that call that function, and does not set the address to zero, would cause unexpected behaviour.

Another option, might be the function returns a bank id, and address, that points to a data structure that contains the requested data.

So perhaps, VDADEV can be extended to return 2 values:

• B -> BANK ID - typically just HBIOS's bank
• 'HL` -> Address of data structure containing port mappings and any other relevant configuration required to directly control the hardware.

Then the application can simply use SYSSETCPY and BNKCPY to get this data. Thats 3 HBIOS calls to get a register though.

So perhaps a better alternative, is VDADEV, does the bank copy prep - so then the app just needs to call BNKCPY after VDADEV. I assume the data length will be fixed.

I think it be good to make the sound driver use the same pattern as the video drivers. To avoid a variation of interface.

What do you reckon?

[vipoo]

Idea 2: I see HBIOS has an internal function to make the bank call - but its not exposed through the public interface. The spike i did - did allow me to call recursively through banks - it only required the stack be located in high bank.

[wwarthen]

Idea 1 - i had thought of the DEVICE function. One of the gaps in my knowledge is the amount of ports and h/w config values the non TMS drivers would need to return.

If its more than can be returned in the available registers, then I guess it would need to implement a pattern of writing to a supplied address - kind of like how the font data can be returned in VDAQRY.

I'm not sure about the amount of data to return. That is the part that requires some serious design consideration. :-) I sort of assumed it would be more than just a couple registers worth of data for most types of drivers (sound driver has a relatively simple configuration).

But this second option, requires the caller to set a register to a high address or zero. So I would have thought that to retrofit this approach to an existing function would introduce a backward compatibility issue. Any app that call that function, and does not set the address to zero, would cause unexpected behavior.

If the configuration data can be simplified down to 16 bits, then returning it in the HL register via the existing DEVICE call would be ideal. If not, we can talk about the options.

Another option, might be the function returns a bank id, and address, that points to a data structure that contains the requested data.

So perhaps, VDADEV can be extended to return 2 values:

• B -> BANK ID - typically just HBIOS's bank
• 'HL` -> Address of data structure containing port mappings and any other relevant configuration required to directly control the hardware.

Then the application can simply use SYSSETCPY and BNKCPY to get this data. Thats 3 HBIOS calls to get a register though.

I like this idea. I think the bank could just always be the HBIOS bank, so no need to specify it.

Perhaps the HL register could just point to the entry in the config table for the unit within the driver. The problem with that is that the caller would need to understand the format of the config entry, so maybe not a good idea. Even if a more generic structure were created in HBIOS, the HL register could return a pointer to it. This would not break anything for existing callers. Great idea.

So perhaps a better alternative, is VDADEV, does the bank copy prep - so then the app just needs to call BNKCPY after VDADEV. I assume the data length will be fixed.

Um, I think it wold be more consistent for the caller to do the bank prep.

I think it be good to make the sound driver use the same pattern as the video drivers. To avoid a variation of interface.

Agreed.

I like where this is going. There have been many requests for programmatic access to the config data in RomWBW.

[wwarthen]

Idea 2: I see HBIOS has an internal function to make the bank call - but its not exposed through the public interface. The spike i did - did allow me to call recursively through banks - it only required the stack be located in high bank.

Agreed this concept seems useful. I still feel like it would be better for applications to be given access to an entire dedicated RAM bank rather than the scraps of the HBIOS bank. The interbank call should work just fine with an arbitrary bank and the application would have a full 32K of unfettered space to use however it wants.

[feilipu]

Agreed this concept seems useful. I still feel like it would be better for applications to be given access to an entire dedicated RAM bank rather than the scraps of the HBIOS bank.

Excuse me for adding 2c.

Another way to get the same answer, with fewer hardware dependencies.

A while ago I was dead keen to implement a RTOS (hypervisor), which would manage multiple CP/M instances running in orthogonal banks. I implemented a bank calling system that was relative to the calling bank, so the originating bank and destination bank were not absolute, but depended on from where the call was made. But, time passed and I found that I had no real application for that functionality. Still haven't to this day.

I think that was because CP/M application writers solved this very problem by writing overlay capabilities, and dealing with the 64kB RAM limitation. And today with fast CF/SSD drives there is less reason to need to do bank calls.

The recently discussed PL-I compiler is a good example of this overlay process.

What I'm still missing is a nice way to write overlay code, using modern tools. Yes, the old tools can do this, but I'd like z88dk to package overlays, and manage the linking properly for CP/M. But, I'm always down another rabbit hole, and haven't actually done anything about this.

So, to draw this ramble to a close, I'm now of the view that the bios should keep out of the way as much as possible, and let the CP/M application do what it does. Unless we are planning to get some new CP/M apps written, we're pretty much stuck with the API from CP/M 2.2 BDOS, as a baseline.

[vipoo]

I'm not sure about the amount of data to return. That is the part that requires some serious design consideration. :-) I sort of assumed it would be more than just a couple registers worth of data for most types of drivers (sound driver has a relatively simple configuration).

I kind of assume each driver would be a completely separate set of data. So the data set is driver specific. Some might only need 1 or 2 ports - so could potentially returned in a register set. While others would have a few bytes of data and would need to be referenced in memory.

I kwas thinking that each driver would specify its data structure - i cant imagine how a generic structure could - nor I wonder, would it be needed. If an application is going to be coded for a specific piece of hardware - its already couple to a specific driver query.

Um, I think it would be more consistent for the caller to do the bank prep.

OK -- I just thought it would make it a little easier for the dev, only require 2 HBIOS calls to get the port maps. - maybe its just me, wanting to be as lazy as possible :sunglasses:

Cheers Dean

[wwarthen]

I'm not sure about the amount of data to return. That is the part that requires some serious design consideration. :-) I sort of assumed it would be more than just a couple registers worth of data for most types of drivers (sound driver has a relatively simple configuration).

I kind of assume each driver would be a completely separate set of data. So the data set is driver specific. Some might only need 1 or 2 ports - so could potentially returned in a register set. While others would have a few bytes of data and would need to be referenced in memory.

I kwas thinking that each driver would specify its data structure - i cant imagine how a generic structure could - nor I wonder, would it be needed. If an application is going to be coded for a specific piece of hardware - its already couple to a specific driver query.

Agreed.

Um, I think it would be more consistent for the caller to do the bank prep.

OK -- I just thought it would make it a little easier for the dev, only require 2 HBIOS calls to get the port maps. - maybe its just me, wanting to be as lazy as possible 😎

Thanks for understanding. Sometimes I have preferences that I have trouble letting go of!

[wwarthen]

Agreed this concept seems useful. I still feel like it would be better for applications to be given access to an entire dedicated RAM bank rather than the scraps of the HBIOS bank.

Excuse me for adding 2c.

Another way to get the same answer, with fewer hardware dependencies.

A while ago I was dead keen to implement a RTOS (hypervisor), which would manage multiple CP/M instances running in orthogonal banks. I implemented a bank calling system that was relative to the calling bank, so the originating bank and destination bank were not absolute, but depended on from where the call was made. But, time passed and I found that I had no real application for that functionality. Still haven't to this day.

I think that was because CP/M application writers solved this very problem by writing overlay capabilities, and dealing with the 64kB RAM limitation. And today with fast CF/SSD drives there is less reason to need to do bank calls.

The recently discussed PL-I compiler is a good example of this overlay process.

What I'm still missing is a nice way to write overlay code, using modern tools. Yes, the old tools can do this, but I'd like z88dk to package overlays, and manage the linking properly for CP/M. But, I'm always down another rabbit hole, and haven't actually done anything about this.

So, to draw this ramble to a close, I'm now of the view that the bios should keep out of the way as much as possible, and let the CP/M application do what it does. Unless we are planning to get some new CP/M apps written, we're pretty much stuck with the API from CP/M 2.2 BDOS, as a baseline.

Thanks for these thoughts Phillip. I think you are helping me see more clearly why I am a little reluctant to jump into a solution for a problem that is not yet fully defined. As you point out, there are multiple potential solutions for an application to expand it's memory footprint. I find such options are best explored in the context of a specific, well-defined need. Once we have that defined, it will be easier to evaluate solutions. I had not thought of overlays, but that is certainly a potential solution.

Thanks,

Wayne

[feilipu]

As you point out, there are multiple potential solutions for an application to expand it's memory footprint. I find such options are best explored in the context of a specific, well-defined need. I had not thought of overlays, but that is certainly a potential solution.

There is an open issue at z88dk on producing PRL (a kind of overlay) code. But it has been unattended for so long now, it has gathered dust. I raised it around support of compiling MP/M from within z88dk, but that is such a niche it doesn't warrant attention. It is on @pauloscustodio long list of enhancements.

IMHO, the bank switching application is to support "huge" applications. "Huge" data can be stored off on disk on 8MByte drives, so that is no longer a compelling argument. I think it would be great to have an easy way to build huge (>50kByte) applications. The Softools compiler promises banking but, aside from that, I don't know of any other automatic way to generate overlays, which would let you build 100kByte or greater applications.

The LINK-80 can generate overlays (which are a kind of PRL that is not relocatable), and .OVL files are used by MS Basic Compiler (not the interpreter), and the PL-I compiler mentioned above. There's even a new version of the Link-80 available, which might be useful if you wanted to write a program to access more RAM through using OVL.

Cruise over to z88dk if you feel like tackling this issue from the app. dev. point of view 😄 . It is certainly something missing from z80 currently.

Cheers, Phillip

[vipoo]

So many options - so many challenges - so much code to learn --- perhaps in time. Its funny to think i only started in the retro computing space a couple of months ago - just before the global lockdowns.

Cheers Dean

[wwarthen]

I'd like to bring some closure to this thread.

I think the ability to make calls into other banks via BNKCALL is working as is. We don't have consensus on how (or if) to allocate space in other banks. I think that aspect of the discussion needs to wait for a real-world use case. We can deal with it at that point.

The aspect of this discussion that I do want to pursue is the ability to query a device driver for configuration information. This is an actual problem for me right now because I want to use such a mechanism to improve XModem.

I spent some time looking at the options for returning config information. We were discussing an approach where a caller could get a pointer to the config data block for any driver/unit. It turns out that access to a unit's config data block is available already. The new functions I just added do just that. They return the function pointer and data blob address for any driver/unit. The data blob address is, in fact, the config data block. So, any caller that needs a detailed view into a driver's config info, can use that.

Despite having this capability, I feel like it would be very beneficial to also have a simpler and more consistent way to get the basic configuration data for a unit. Looking at the actual configuration of the drivers, it really boils down to two values in virtually all cases: 1) mode, and 2) base I/O address. These two values represent a simple and consistent approach to returning the config data for a given unit. Since they can be returned with a pair of 8 bit registers, it is also very easy to implement and to use. I propose to return these two values in the H and L registers in all DEVICE calls. H=mode, and L=I/O base.

I know that extending the DEVICE function call could potentially cause an issue if a newer application was run on an older implementation of HBIOS. However, I think that this will be a moving target no matter what we do. To mitigate this, I think callers should check the HBIOS version before relying on this type of data.

Please let me know of any concerns to this approach. It is trivial to implement and I have a need for it so I plan to start implementing it tomorrow. :-)

Thanks,

Wayne

[vipoo]

Closing this thread, and moving forward as you suggest sounnds cool.

I wonder if it might be helpful - to use the github projects section - to create little cards of features desired, and accepted, and those in play. A simple kanban wall.

That way you can track - who is working on what , conversation of the implementation can have a single thread - i have not use github's implementation - but it does seem quite simple and light

[wwarthen]

I wonder if it might be helpful - to use the github projects section - to create little cards of features desired, and accepted, and those in play. A simple kanban wall.

That way you can track - who is working on what , conversation of the implementation can have a single thread - i have not use github's implementation - but it does seem quite simple and light

That is an interesting idea. I have used Kanban boards in the past and they worked well. I like the idea of using that as a way to know what work is in play.

I setup a basic Kanban style board on GitHub for RomWBW. I put the current issues in it. Let's see how it goes.

Thanks,

Wayne

[wwarthen]

I have implemented the extension to the DEVICE API functions. At least for my purposes, it is working OK. I'm going to go ahead and close this issue. Feel free to let me know if I missed anything.

Thanks,

Wayne