Feature request: Set of commands to simulate the behaviour of the builtin applications like solver and numerical integration

[fmartinp]

I think that it could be interesting to add a set of commands to be able to mimic built in applications like solver and numerical integration. A set of commands to select functions and deal with MVARs to finally run a program based on the selected functions and parameters.

[thomasokken]

There is no function currently for presenting a menu of programs containing MVARs, but for setting and selecting parameters, given such a program, a function does exist: VARMENU. This is documented in the HP-42S manual, pages 125-128.

[fmartinp]

Thanks for the quick feedback. Indeed, I was thinking more in the first use case:

• a command to present a menu of programs containing MVARs and store the name of the selected function in a variable.
• a command to present a menu of programs and store the name of the selected function in a variable

Both functionalities could be presented with only one command where it firstly attempts to search functions with MVARs, if none exists then presents all the functions. Other implementation is two commands, one for each behaviour, if no functions of the searched type are available, then the command skips the next step in the code.

Probably that would be the missing element to be able to create programs as the built in applications.

ps. congratulations for the project. The current function set is simple enough and allows to develop everything that one could imagine.

[thomasokken]

The missing functionality could be provided using a function that returns a list of labels, either all global labels or only global labels with MVAR, in a matrix. The menu could then be implemented in user code using the programmable menu. Returning a catalog in a list is functionality that will be needed, or at least very useful, once I add directories, which is something I am already thinking of doing. Of course the one drawback of this approach is that it wouldn't work with seven-character labels.

[fmartinp]

Your proposed implementation seems very elegant since it provides a simple interface that gives to the user the flexibility to build custom behaviors (menus or any other thing that the user would like to implement). However, I do not have a clue either on how to generalize the solution for a 7-char label without altering other functions.

A convoluted solution could be to store the memory offsets of the labels in a matrix and somehow XEQ, GTO, VARMENU, 'get function name (only first 6 char)' and maybe other functions could be upgraded to deal with those memory offsets.

[thomasokken]

Or I could get rid of the 6-character limit altogether... https://www.hpmuseum.org/forum/thread-16218.html

[fmartinp]

To expand a bit the convoluted solution...

• the functions that you proposed provide memory offsets in a matrix
• another new function which converts the memory offset in the name function in the alpha register
• XEQ, GTO, VARMENU... can use the alpha register for indirect addressing

On the 6-char limit, is an artificial limit? Or is related to fit the 6 char in the same memory space as other types like numbers?

[thomasokken]

The 6-character limit is a holdover from the HP-41C. On the 41C, registers are 7 bytes each; strings are indicated using a special value in the sign nybble (0=positive number, 9=negative number, 1=string), with the string length in the second nybble and the remaining six bytes available for text. In the HP-42S, they could have raised the maximum to 7 characters since its registers are 8 bytes each, but they chose not to. In Free42, I can do what I want (as long as I maintain HP-42S compatibility), and a special ASTO variant that stores the entire ALPHA register rather than only its first 6 characters, would do the job. Of course memory management would become more complicated because long strings wouldn't fit in the space of a floating-point number... but that's all pretty manageable.

[fmartinp]

If there is no real limitation, in my personal opinion, I think that increasing the storage to 7 char would be enough in order to be able to use variables for indirect addressing.

So far, I do not have a use case that needs longer strings. I do not have anything against longer strings but I would be afraid if adding unlimited strings will translate in a larger footprint of the compiled binary of Free42 in such way that the memory of my DM42 is significantly affected (but this is my personal situation since I am mainly using Free42 there ...).

[thomasokken]

After the big stack is done, I'm going to create a fork, to be called Free42+ or Plus42 or something like that, and that fork will get the hard-core stuff like unlimited strings, equations, big display, and whatnot. That may end up being too heavyweight to run on the DM42. But the catalog-to-list function alone, with 6-character limitation, could be added to the basic Free42 without bloating it significantly, that would be a pretty straightforward function to implement.

[fmartinp]

That would be great. Also, I am happy to listen that you will keep a fork suitable for DM42. If the computer version can display the full stack will be great too. Debugging programs in the computer is hard due to the two lines screen.

In my humble opinion, the beauty of Free42/Hp42s is the power of the simplicity of its types and functions. They are well thought to provide to the user the power to create almost anything (your extensions helped a lot). That makes the difference with other calculators that implement larger stack or mix a lot of heterogeneous types in the stack. I do not have daily use cases for those functionalities so far and they make the calculator too eclectic for my taste.

[thomasokken]

OK. I'll add the new function (MVARCAT? something like that) to the to-do list for Free42 3.0. I'll make it so that it simply omits 7-character labels from the list. In Free42+, it will return everything. I assume that in practice, it won't make much of a difference. I tend not to use such long labels anyway since they just get truncated in the menu. I'm closing this issue now, but feel free to reopen it if you want to continue this discussion later.

[fmartinp]

Hello Thomas,

I have seen that you have already implemented MVARCAT. I was looking at the code, if I understand it correctly, the matrix with the labels will be placed in the stack. After your comment on how you will evolve RTNERR to accept an argument, I was wondering if it would have sense that MVARCAT works in the same way. So, if an argument is provided the resulting matrix is stored in that variable passed as an argument, if no argument is provided, the matrix with the labels is placed in the stack.

What do you think?

[thomasokken]

The reason for making RTNERR take the error number from an argument rather than the stack is so that it doesn't have to leave the error number in X when used without FUNC. I see no compelling reason to make MVARCAT take an argument, you can simply do MVARCAT STO "FOO"...

[fmartinp]

Yes, I was thinking that the possibility to store it directly in the variable would allow not to alter the stack. In the same way that RTNERR with argument does.

[thomasokken]

I'm not following. In the case of RTNERR, it is literally impossible for a function that doesn't use FUNC to return an error code without leaving that error code in X. In other words, RTNERR needs to get the error number from a parameter in order to be usable without FUNC. MVARCAT is not like that at all. It's just a function that returns a result. No functions return their results to named variables, none whatsoever. Why would MVARCAT need that ability?

[fmartinp]

I proposed that because there is the command DIM which stores directly a named matrix and I was a bit biased with its behavior.

However, you are right and it would not be needed. The FUNC command will help to keep the stack unaffected.

[thomasokken]

Hmmm. In Plus42, it would be more natural to return this list as a list, rather than a matrix... If nothing else, lists can be empty, while matrices cannot. But then MVARCAT would work differently in Free42 and Plus42, and I want Plus42 to be backward-compatible with Free42. Maybe this function should actually take care of presenting the menu itself? More like how VARMENU works? It would mean more work for me to implement, but it would be a lot nicer to use in programs that way. Unless there is a reason to want that list of functions for something other than building a menu?

[fmartinp]

I fully agree that working as VARMENU is much better. A menu that when you press on the menukey leaves the name of the function in the alpha register and continues the execution of the program. (Actually, I was considering to start my own C++ implementation of that).

I have spent some time in using MVARCAT with current implementation and I needed about 100 lines of RPN to achieve the same result as VARMENU. In C++, I believe that is is only a bit of code reuse of VARMENU and PRG_SOLVE menu. I can support you in the code writing of the function if needed.

So, I fully support the idea that MVARCAT should behave as VARMENU like.

[thomasokken]

If you can come up with a simple patch for this, I would be happy to use it. I agree that it shouldn't require a lot of code since it should be able to use the existing functionality from the SOLVER and ∫f(x) top-level menus.

[fmartinp]

perfect, I will send you a pull request with a proposal once I finish the patch.

[fmartinp]

I have just sent you the pull request.

[thomasokken]

OK, thanks! I'll have to tweak it a bit because I just made some changes to the catalog myself, so those collisions need to be dealt with; I think I'll want to name the function something different but I'm not sure what yet; and then after I test it a bit I'll merge it into master and plus.

[thomasokken]

I merged the pull request and performed a few additional edits in a subsequent commit. I haven't tested it thoroughly yet, but a quick test indicates that the function is working.

[thomasokken]

It looks good to me. I'm closing this issue again, and the new function will be in the next release. Thank you for the patch!

[fmartinp]

No problem! May I ask you which your development environment in Linux is? I have had some problem with the debugging when gdb was stepping over the GTK functions (I am not used to work with graphical apps).

By the way, I also prefer the new name of the function (PGMMENU).

[thomasokken]

I use Ubuntu 12.04 for Linux release builds, and varying newer versions of Ubuntu when I want to check that things work correctly with later revisions of GTK 3. My development environment is rather old school, I edit in vim and build using make, and debug using ddd.

Debugging GUI applications can be difficult because the debugger can interfere with things like exposure and focus. You may need to debug remotely when things get really hairy. If I remember correctly, ddd is smart enough to notice when the X server has been grabbed, but I'm not really sure, I haven't had problems with that recently. There was a fair amount of debugging involved in the switch from GTK 2 to 3, but I don't remember running into problems with the debugger then.

[fmartinp]

Thanks for the clarification on how to debug GUI applications.

By the way, I have noticed that hp42ext array in core_main.cc does not include CMD_PGMMENU, is that correct? (I don't understand very well the purpose of that array and if it is only for commands without parameters).

[thomasokken]

I made a bunch of mistakes before, so, starting over and proofreading properly before I hit "Comment"...

The hp42ext array is used for decoding certain instructions with parameters in raw files. Briefly: The HP-42S instruction set is based on that of the HP-41C, with several additions. Of those additional functions, those without parameters are encoded using the XROM scheme, and functions with parameters are encoded as strings, where the first character of the string (i.e. the second byte of the instruction, following the initial 0xf<length>), is in the range 0x80-0xff. All the HP-42S instructions with parameters that the HP-41C doesn't have (like INPUT 00 or PGMSLV "FOO", for example) are encoded that way, as well as all instuctions that the HP-41C does have, when used with ALPHA or IND ALPHA parameters where that isn't possible on the 41C (like STO "FOO" and GTO IND "FOO", but not GTO "FOO").

The hp42ext table is used to decode that second byte.

CMD_PMEXEC isn't encoded in that table since it is never stored in programs, so it isn't necessary to be able to encode it in raw files. CMD_PGMMENU is stored in programs, but it has no parameters, so it is simply encoded as 0xa7 0xe8, i.e. XROM 31,40.

[fmartinp]

Thanks for the clarification Thomas. I will try to find some documentation about the command encoding.

By the way, building the latest code with the arm compiler, the compiler warns about the type of SSLENV and SSLENM which are unsigned and are used in comparisons against signed integers. I do not see any problem, but I let you know in case you would like to add an explicit cast.

https://github.com/thomasokken/free42/blob/98f2dfa1372bc08843be28248e19647b9af2de25/common/core_variables.h#L80

[thomasokken]

This is a good starting point for the HP-41C instruction encoding: https://www.hpmuseum.org/prog/synth41.htm#ins https://www.hpmuseum.org/software/xroms.htm

I don't remember finding references on the HP-42S instruction encoding, apart from it being basically HP-41C-compatible. Which XROMs it uses for parameter-less extensions, and how it encodes extensions with parameters, was easy to figure out by just creating small programs in Emu42, saving them, and then looking at hex dumps of the raw files.

Regarding the warnings about SSLENV and SSLENM -- Hmmm, I'm not getting those warnings during the macOS, iOS, or Android builds. What is your build environment? And do the warnings go away if you change the macro definitions to

define SSLENV ((int) sizeof(char *))

define SSLENM ((int) sizeof(phloat) - 1)

?

[fmartinp]

Thanks for the links. I will have a look at them.

Regarding the build environment, I am using gcc-arm-none-eabi-7-2018-q2-update. Yes, the warning disappears either using the cast (int) or (signed). The problem is that size_t is unsigned.

[thomasokken]

OK, I'm adding the type cast to the two macro definitions.

[fmartinp]

Hi Thomas,

I am trying to understand how the new functions are defined based on the information that you provided me in one of your previous answer. I was looking forward to see your implementation of the new comparison functions to check if my understanding was correct. I think that I understand about how you define the code of the instruction: 0xf2 is because a new function of 2 bytes and the code2 is the identifier of the function. However, I have realised that I not understand the role of scode. Also, it puzzles me why argcount and rttypes are different for the 3 functions below when in principle accept the same type of parameters, the same amount of parameters and provide the same output.

If you would have some time to explain how you define the lines below in core_tables.cc, I would be very grateful.

{ /* X_NE_NN */    docmd_x_ne_nn,     "X\014?",              0x00, 0x05, 0xf2, 0x15,  3, ARG_VAR,   1, ALLT },
{ /* X_LT_NN */    docmd_x_lt_nn,     "X<?",                 0x00, 0x06, 0xf2, 0x16,  3, ARG_VAR,    1, 0x01},
{ /* 0_EQ_NN */    docmd_0_eq_nn,     "0=?",                 0x00, 0x0a, 0xf2, 0x1a,  3, ARG_VAR,    0, NA_T },

Thanks, Fernando

[thomasokken]

It's a good thing you asked, because while reviewing this, I noticed that the scodes for X<=?, X>=?, and all the comparisons with 0 were actually incorrect in cmd_array. I fixed them and will upload a new build shortly.

On to the explanation:

scode is the code that is used as the opcode for HP-42S-style extended functions with string parameters. In other words, all functions with string parameters other than LBL, GTO, and XEQ. For normal string parameters, scode is used as is, and for indirect string parameters, it uses scode + 8.

The function starts with an Fn byte, where n is the number of bytes that follow. This is the scheme that is used for all extended functions with parameters. You can tell the difference between an extended function and a string (which also starts with Fn) by looking at the high bit of the second byte: if that bit is 0, it's a plain old string, and if it's 1, it's an extended function.

Taking LSTO as an example, this has scode 0xc7, so LSTO "FOO" gets encoded as f4 c7 46 4f 4f, while LSTO IND "FOO" gets encoded as f4 cf 46 4f 4f, adding 8 to the scode for IND string addressing.

code1 and code2 are used for everything else. In functions without parameters, those codes are the entire instruction, with code1 being zero for a one-byte function. For functions with parameters, code1 and code2 (or code2 alone if code1 = 0) are followed by a single parameter byte. The encoding of this byte is described in the comment just below cmd_array in core_tables.cc.

In the case of HP-42S extensions with non-string parameters, code1 is always f2, which introduces a three-byte sequence (including the f2 itself), following the special string scheme I mentioned above.

There is an added twist: because the second byte of extended functions must be in the range 80-ff, in order to be able to tell it apart from regular strings, HP-42S extended functions with parameters have a space of only 128 opcode bytes to work with. For the real HP-42S, this was sufficient, but because of all of the functions I've been adding, I ran out of opcode bytes, and defined an extended extended encoding, where byte 2 is a7 and the real opcode is in byte 3. When you see instructions with code1 = f2 and code2 between 01 and 7f, they are encoded using the extended scheme.

Again using 0=? as an example, 0=? 99 is encoded as f3 a7 1a 63. So the f2 in code1 becomes an f3 because the actual instruction, being extended extended, is actually one byte longer, namely, the a7 byte in the second position.

The extended extended scheme with a7 applies to scode and to code1/code2. For example, 0=? has scode 0x22, so high bit 0, and that means that 0=? "FOO" is encoded as f5 a7 22 46 4f 4f, and 0=? IND "FOO" is encoded as f5 a7 2a 46 4f 4f.

[fmartinp]

Thank you very much for such detailed explanation.

I would like to ask you also about my last doubt about the last update in the cmd_array table. Why argcount and rttypes are different between those functions?

   { /* X_NE_NN */    docmd_x_ne_nn,     "X\014?",              0x00, 0x05, 0xf2, 0x15,  3, ARG_VAR,    1, ALLT },
    { /* X_LT_NN */    docmd_x_lt_nn,     "X<?",                 0x00, 0x06, 0xf2, 0x16,  3, ARG_VAR,    1, 0x01 },
    { /* 0_EQ_NN */    docmd_0_eq_nn,     "0=?",                 0x00, 0x22, 0xf2, 0x1a,  3, ARG_VAR,    0, NA_T },

Thanks

[thomasokken]

The comparisons with X use an argument from the stack, namely, X, so they have an argcount of 1. The comparisons with 0 do not take an argument from the stack, so their argcount is 0. The difference between X=? and X≠? on the one hand, and X<?, X>?, X≤?, and X≥Y? on the other, is that X=? and X≠? can handle all kinds of objects, while the other four can only compare real numbers.

[fmartinp]

Thank you very much! I understand it now. I did not realise that the last two columns were referring to the stack arguments.