Splitting `ulisp.ino` so userfunctions can be in separate file?

[dragoncoder047]

Copied from https://github.com/technoblogy/ulisp-esp/issues/54#issuecomment-1186288972

How can I split the ulisp.ino up so I can include my "extra" functions in a separate file? Something like this:

#include "ulisp.cpp"

object *fn_myfun (object *args, object *env) {
  // do something
  return nil;
}
// how to register it in the lookup table??

void setup() {
  // other setup as necessary
  ulisp_init();
}

void loop() {
  // NOW WHAT?
}

I have seen on the forums that the source file isn't quite correct C++ (missing forward references, etc), so what changes would need to be made there (as well as the problems pointed out in the comments on my snippet above)?

[technoblogy]

I have seen on the forums that the source file isn't quite correct C++ (missing forward references, etc).

The Arduino IDE takes care of resolving forward references automatically, so I've chosen to keep the source shorter by not explicitly defining them. If it turns out that this was a bad decision I'm prepared to rethink this.

I'm not aware of any other respects in which the source is not correct C++; are you?

[technoblogy]

How can I split the ulisp.ino up so I can include my "extra" functions in a separate file?

I agree this would be a great addition to uLisp, and it's something I've been thinking about.

Currently the procedure for adding functions to uLisp is documented here:

Adding your own functions

You have to add the definition of the function, which could definitely be in a separate file.

However, you also have to add two entries into two lookup tables of function names and entry addresses, and add an index number for the function in an enum, and I can't currently see how these could be merged in from a separate file. If you can see a solution to this I'd be interested.

By the way, I'd again encourage you to post this question on the uLisp forum. That gets a lot more visits than these GitHub issues, and other users may already have suggestions for solving this. If you're worried about getting spammed as a result of signing up as a user perhaps I can reassure you that I run the forum as a personal service on a DigitalOcean Droplet, and it doesn't link into any commercial forums or services.

[dragoncoder047]

I've been looking in the C++ preprocessor documentation and unfortunately I don't think that being able to add userfunctions with a simple macro would be possible. @bobach's refactor (https://github.com/bobach/ulisp/blob/refactor/src/function.cpp) is logically split up as much as possible, but the big lookup table is not (and probably cannot) be split. Short of ditching the hard-coded lookup table, I don't think this is possible.

[dragoncoder047]

Now that I think of it, the whole reason for the lookup table was to save on RAM, but each of the tbl_entry_t parts is the bulk of the memory. Perhaps the table itself could be in RAM, and contain pointers to tbl_entry_t objects in flash that could be dereferenced in the lookup routines. That way since the table itself is in RAM, it could be modified by functions. I don't know how this is possible, especially since uLisp is throwing pointers all over the place already, so I'm probably not going to pursue this myself.

[technoblogy]

One idea: each additional source file could have its own mini lookup table, and uLisp could search each of them after doing the main lookup table.

[dragoncoder047]

One idea: each additional source file could have its own mini lookup table, and uLisp could search each of them after doing the main lookup table.

Great idea... but people would still have to edit the lookup functions to let uLisp know of all the lookup tables. Those could instead be added to a "meta-lookup table" but then we're back at needing a lookup table in RAM.

Then again, the meta-lookup table would necessarily be shorter (2-3 entries, each pointing to a sub-lookup table in flash) rather than one giant (150+ entries) main lookup table. So that's another idea.

And that doesn't solve the problem of adding function indexes to the enum anyway, so the enum will either have to be ditched or some other solution will be needed.

[dragoncoder047]

The enum could be refactored out by storing the functions and special forms in separate tables, so then there would be no need to check indexes in the enum to determine whether we are in functions, special forms, atoms, etc. There would be two or three loops to check all the different tables, but those would be more than made up for by the removal of the enum.

[dragoncoder047]

I also had another idea here yesterday that each tbl_entry_t could instead have a 16-bit field as the last entry, and the lower 8 bits could be used as they are now (minimum and maximum number of arguments), and additionally 3 of the upper 8 bits could be used as flags to indicate to eval() if it is a special form, a symbol, or tail-recursive. Then people could throw in special forms and symbols too in whatever order they want and uLisp wouldn't care. It would also simplify eval() a bit and maybe even do away with the builtins enum now that eval() can just check the bit flags and doesn't have to look in the indices of the enum.

I don't know about memory consumption, so maybe doubling the memory footprint of the lookup table is not possible especially for low-memory AVR microcontrollers. I have an ESP32 so I'm really not worried about memory.

[technoblogy]

Good suggestion! I think someone suggested that on the uLisp forum a while ago, but I can't find it now.

[dragoncoder047]

Closing this now that uLisp 4.4 is out... it's a lot easier now to add custom metatables