ROM loading order

[SteveFosdick]

After changing the set of ROMs to be loaded for a particular model the order in which they are loaded, and thus which slots they appear in, is not stable.

For each model emulated, B-Em loads the roms from a sub-directory of the roms directory in the distribution. It starts at slot 15 and works downwards processing the rom files in the order they are returned by the Allegro al_findfirst/al_findnext functions. On Linux, at least, these functions return the filenames not in any obvious order, like alphanumeric, but in the order the OS happens to have the filenames stored in the filesystem. On some hash-based filesystems this is in hash order and may as well be random, while on others it is the order in which the files were copied into that directory independent of the name.

Controlling which rom goes into which slot is sometimes important as it sometimes necessary, for example if two ROMs implement the same command its may be necessary to put the one that should take priority in a higher numbered slot.

Two possible solutions come to mind:

1. Read the set of files in the directrory and then sort this list before processing it.

2. Make two passes through the files in the directory. On the first pass process only those rom files whose filename starts with a number indicating a specific slot to be loaded. On the second pass load the remaining roms into the remaining slots.

[ThomasAdam]

You're right -- on UNIX, there is no ordering at all, even ls -f is a poor approximation of any likely ordering if there were to be one.

There's a couple of details here which feeds in to a TODO item on the list -- and that is getting rid of the whole exedir notion, and ROMS looked for in either /usr/[local]/share/b-em/roms/... on UNIX, and support a ~/.bem/roms directory.

What order would a real BBC Micro allocate ROMs? Is it the order they're inserted on the machine, or some pther mechanism? Option 1 sounds the more obvious choice here.

[chrisn]

Of those two options, I'd prefer (2) as it gives users more control over which ROMs go where. Another approach is to specify the ROMs from the b-em.cfg config file (similar to how BeebEm works), which makes changing ROM configurations easier as you don't have to rename files. Something like this:

[roms]
rom_15=BASIC.rom
rom_14=DFS-0.9.rom
# etc.

Just a thought.

[SteveFosdick]

@ThomasAdam on a Model B, at least, there are four sockets in the machine which are nominally ROM IDs 15,14,13 and 12 though I think incomplete address decoding means there are ghost copies, i,e. the ROM at 15 also appears at 11, 7 and 3, the one at 14 appears also at 10, 6, and 2 etc. for the other two. When the OS scans the ROMs it finds the ghost copies and ignores them.

When the OS offers ROMs the chance to intercept something or offer an extended feature in does so top down, so ROM 15 is given the option first and if it doesn't take it, it is offered to 14 and so on.

For many ROMs the order is not at all critical but there are some cases where the ROMs need to go in a certain order. I am sure at least one ADFS/DFS rom pair needs to have the ADFS in the higher priority slot (so it can check if ADFS is current and if not, let the call fall through to DFS, I think) and there maybe other too. If you were installing them in a real machine you'd be advised to put them in certain slots.

Of course, ROM expansion board were common. Usually these do fully decode the ROM select latch and give you much more scope for putting more ROMs in and maybe sideways RAM too.

[SteveFosdick]

@ThomasAdam adressing the other question of a suitable directory to place files such as ROM images I had also thought that would be a step forward but hadn't really got around to doing anything about it. In it's current form, assuming that the various data directories will be sub-directories of the directory in which the executable is stored, has limitations. It very much suites unpacking a distribution and running it in-place but it doesn't make it easy for a Linux packager to package for distribution and it doesn't make it easy to set up for multi-user use on either Linux or Windows.

On the other hand, if we instead adopt fixed paths in the filesystem, it makes it harder to run in-place.

Maybe the answer is to have a search order - a search path - for finding these files. The exact directories being searched would be different on Unix-like systems and Windows but we may be able to share the code to actually do the searching. That way user-specific files can take priority of those installed with the software.

[SteveFosdick]

@chrisn Yes, interestingly that was listed as a feature BeebEm had over B-Em - the ability to easily change the ROM configuration. On B-Em which banks are ROM and which are RAM is fixed by which model is selected and the ROMs are loaded from a directory. BeebEm has a GUI for configuring ROMs and even lets you switch between ROM/RAM in banks at run time.

There are now several ideas here. I am debating how we should proceed, i.e. whether to make small improvements one after the other or to try and work out how this would ideally work and then aim straight for that.

[ThomasAdam]

@SteveFosdick -- so this problem is no different to any other program. What you do is set up something like DATADIR which is configurable via ./configure but default to something like /usr/local/share/b-em/roms -- and that search path is used if, say, ~/.b-em/roms/ isn't there. Indeed, you can even overly them, so you only need to place custom/subset roms in ~/.b-em/roms/ while still having those loaded from DATADIR.

None of this implies fixed-paths, and I'm not sure where that's come from.

As for Windows -- pass, I am not in a position to really know.

As for configuring this -- possibly. But right now, b-em.cfg is only used if menu customisations happen -- that is to say, someone chooses an option from the menu. Currently, this file isn't considered a configuration file, and I'm keen for it not to be for now.

So regardless, my wrangling autotools to use DATADIR as outlined, is a good suggestion. This means you no longer have to ship half of b-em with the binary, because the binary itself has a weird concept that its ROMs are somehow relative to where it's being run, which isn't desirable.

I'll pick this up on the weekend.

[SteveFosdick]

@ThomasAdam Lots of programs have traditionally had run-time paths configured via autotools and used only that but the search behaviour you mention is indeed common to desktop environments looking for files. The latest trend seem to be for having .config .cache and .local under users' home directories rather than an application-named .dir. There may even be a library function to do the search.

I am pretty sure Windows has a similar search system though I don't remember the names of the directories off the top of my head. I think BeebEm implements the Windows version of this so may be a good reference.

[ThomasAdam]

@SteveFosdick -- sure. You're referring to the XDG specification, which is separate to other configuration paths.

I'm still keen to ascertain the ROM loading ordering -- and more importantly if there's ROMs which have to be in a particular slot, as that will affect the ROM load ordering.

I am keen to not overload this with any sort of configuration file, which is a separate problem. If we need to support ROM loading from the menu, then we can do.

[fordp2002]

One idea is to add the Rom slot and Rom/Ram setting to the Rom Filename using a formal naming system. E.g. DFS_12_ROM.bin

[ThomasAdam]

I'm working on this given other changes.

Watch this space.

[SteveFosdick]

Fixed by commits: 0c9b467710ec2cc2a22b47fdbbbe0bea4d4f6bf2 348a88b485c6eb40c922a2f2a406dd2441951aa6 92d28567681be8c140009ea6aa36192baf151bb1 6ced147a37e09809343d4e7bdaddff50bbf1b5ad

This adopts the scheme that a ROM whose filename starts with a number, e.g. 12_dfs1.2,rom, will be loaded into that slot if it is free. That was technically easier but also leaves numbers later in the filename available to label which version of the ROM is contained in the file.

[fordp2002]

Thanks Steve, this simple approach will get the job done for now.

[SteveFosdick]

While I was there I did refactor the ROM loading code a bit so there was less copy & paste but it is still the case that different models take a different approach. The master loads a big file that contains the contents of the 1Mbit "super-ROM" from one file and assigns bits of it to the correct ROM slots, the compact loads specific ROMs from their respective files into specific slots with the filenames hard-coded into mem.c and all the other models load the contents of the specified directory, now ordered as per this change. I like to make things consistent but I have to stop somewhere.

BeebEm has some interesting GUI features around ROM loading.

[fordp2002]

I think we need to make a ROM to allow Star Commands to be passed from the emulation to the client. Then you could do things like load ROMs from the BBC Command Prompt or even more fun from within a BBC Basic program.

[SteveFosdick]

I am slightly confused by your terminology but if you mean what I think you mean we can already do this.

The VDFS ROM, in B-Em guthub, and a thank you to J. G. Harston who wrote most of it, co-operates with the VDFS B-Em module to implement an Acorn filing system that accesses host files, i.e. those on the machine on which B-Em is running, directly, i.e. the files don't need to be inside a disc image. It does constraint access to within a nominated "root" directory for security reasons. So with that, it is possible to *SWLOAD a host file into a ROM slot.

The implementation is in two parts. There is a command parser for the master-style SWRAM command which parses the command into the appropriate OSWORD block which can then be interrepted by the current filing system and an implementation of the OSWORD call used if VDFS is the current filing system.

This doesn't inherently know to look in the correct model's "rom" directory but if that roms directory is within the root exposed to VDFS it should certainly work.

[fordp2002]

This is great and news to me thanks. I was more meaning that rather than using the menu you could type a star command from the BBC.

e.g. *MUSIC5000 could be used to install the Music 5000 hardware on the fly.

This could then be added to a Disc Auto-run too.

This feature could be added to the VDFS ROM?

[SteveFosdick]

The hook is there so we can extend it to do other things. I think we need to be careful how much control over the host we give the emulated BBC but controlling elements of the emulation itself should be fine. Non-FS related there is a *QUIT command and we could easily add support for enabling/disabling sound options, changing speed, even loading disc images as someone commented that using the menu was not as convenient as MMFS (which I have yet to experience but it is on order) provided the images are chosen from within the VDFS root as already mentioned.

[fordp2002]

It was me that mentioned MMFS. I may add support something similar to MMFS on B-Em I will have a think about how it could be even better. I agree about the security it would have to be limited to what it can do.