stevenvar / OMicroB

An OCaml generic virtual machine for microcontrollers
Other
138 stars 23 forks source link

Could we port OMicroB to the Numworks CPU (an ARMv7-M Cortex-M7 on 32bits) ? Step ONE in a larger project #36

Open Naereen opened 5 days ago

Naereen commented 5 days ago

Hi there everybody :smiley: !

Following the discussion started on #14 I'll explain what I would like to achieve, here. Sorry if this is lengthy, I prefer to explain clearly and in details.

As the title of the issue suggest, I would like to continue the work done by @Vertmo into porting the OMicroB project to Micro:bit, and port it now to another ARM32 processor, which is very close to the Micro:bit and thus it shouldn't be too hard. From what I understood, the Micro:bit is running on a Arm Cortex-M4 32 bit processor with FPU, with RAM 128KB and at 64MHz. Quite less than what the Numworks calculators offer, but in a very close setup!

About the Numworks

The Numworks calculators are open-source: their OS is Epsilon, written in C++ (and parts in C). Its CPU is an ARMv7-M Cortex-M7 on 32bits, with its core clocked at 216 MHz and 256K of Static RAM. There exists two other open-source fork OS, Upsilon and Omega. I'm aiming only at Epsilon, to aim at the largest possible audience.

By default on the main OS, there is already a MicroPython interpreter as well as a (very good) text editor. All that is written in C++ and in C. It works amazingly well, considering the limited hardware...

My goal(s)

My first goal: I would like to port OMicroB to the Numworks devices. That is, being able to write on my laptop, in OCaml, tiny code that could be compiled to bytecode, and then to the correct assembly language (then binary) for the CPU of the Numworks, using OMicroB. But because of the nature of the calculator environment, I don't want to flash the program to the Numworks. If I do that, I'm most certain that nothing will work. And even if it worked, that would overwrite the OS of the Numworks, that's not what I want. Instead, such assembly language or object (.arm_o file) could then be used to be embedded in an application for the Numworks (see below).

My dream goal: it is to have the same experience as the Python app (that is, a full text editor, interpreter + terminal) for the OCaml language. If not possible, most likely because the RAM of the Numworks is limited, then I could focus instead on the camllight language, or even on tinier OCaml-like languages (like Mini-Caml-Interpreter). My guess is that, if the language interpreter itself is written in pure C (like it is the case for Lua, see below, or for caml-light), this should be possible: all I would need is to have a execute_this_phrase(const char* line_of_code) function, available in the rest of the C code.

What I tried so far

It is possible to write an installable ("flashable") application (a .nwa file) either in C++, but also [in C](https://github.com/numworks/epsilon-sample-app-c and in Rust. Following the tutorial, I successfully wrote a tiny C application, which does nothing interesting yet, to try it out. I want to add at least one object file, compiled for the correct ARM32 bit architecture, to this C application, like what is done for the lua app (see below also).

I tried (for hours, all weekend long) to obtain on my laptop a compiler for OCaml that could produce object code for ARM 32bits, this way I hoped of being able to follow the official tutorial which explains how to include compiled OCaml code in a C app. I wanted to have a tiny C app for the Numworks, where a very simple basic function could be written in OCaml separately (on the laptop), and the called in the app. I tried all the projects I could find online, none of them worked for me (and I tried a lot). This lead me to be interested in OMicroB.

Yesterday, I tried using the omicrob -device microbit2 compiler, to produce a .arm_o object file, from a very tiny OCaml application, targeting an ARM32 processor. I encountered several issues, mainly that there are no print_... functions, thus making the whole process quite hard: if the OCaml code cannot print to the screen of the Numworks, it is quite pointless. Let's address this issue later. Then when I focused on a simpler program, I managed to compile it (and obtain a bunch of compiled file, one being the .arm_o object file). I tried to include it in my baby C app, and got a lot of compilation issues (from what I gathered, because of float-abi=hard vs float-abi=softfp differences, and because the main.c file declares a main() function and the compiled OCaml code also declares a main() function).

An app for Lua: a possible inspiration

And... There is also the lua application, which ports a full Lua 5.4.4 interpreter on the Numworks. It doesn't ship a text editor, and only loads and prints the result of one script, that has to be shipped with the app while installing it. It's not amazing in terms of user experience, but it's already great.

If that is not possible to ship a text editor + interpreter (+ terminal/toplevel?) on the Numworks, then at least I would be happy to be have what the Lua app provides, or even less.

Many thanks for reading all the way down. I'll be actively working on this for the next two weeks at least. Thanks if you can help!

Vertmo commented 4 days ago

Hi, thank you for the detailed plan ! I'll comment on the first goal and your current issues.

The lack of print_ function should be easily fixable; we just didn't implement them because they are not really applicable to a micro:bit, but we have all the setup we need to do so by implementing new hardware primitives.

First, we probably need to tweak the configuration that OMicroB is using when calling arm-gcc to fit that expected for numworks binaries. Maybe you could share with me the repo of your "baby C app", so I can get up to speed on exactly what this configuration is, and test on my side ? Unfortunately I do not own a numworks calculator, but I can probably get one fairly quickly; in the meantime, I can at least see if things compile.

Naereen commented 4 days ago

Hi, thanks for your quick reply. I don't expect you to purchase a Numworks "just" for my weird project. I can try on my side. My main objective today is to be able to obtain .arm_o object code, compiled from a test.ml OCaml test file, for the correct architecture of the Numworks.

On one side, I think I can easily find all the options that I should change in the compilation process (when ./bin/omicrob -device numworks test.ml should call /usr/bin/arm-none-eabi-gcc) ! Taking inspiration from what is called, for instance when compiling the example external app in C.

On the other side, I know I'm not that skilled enough to write drivers and all such complicated code...

Naereen commented 4 days ago

From what I was able to read in the output.c file that is produced by the bc2c compiler, if my input OCaml file had let say a fact : int -> int function, the output.c file doesn't let the fact function be accessible from anywhere? It "simply" implements the virtual machine, which will run the content of the test.ml file. Am I correct?

Vertmo commented 4 days ago

From what I was able to read in the output.c file that is produced by the bc2c compiler, if my input OCaml file had let say a fact : int -> int function, the output.c file doesn't let the fact function be accessible from anywhere? It "simply" implements the virtual machine, which will run the content of the test.ml file. Am I correct?

Pretty much yes, you can only run the program in its totality.

Vertmo commented 4 days ago

I have created a new branch (https://github.com/stevenvar/OMicroB/tree/numworks) and started to do some tweaks. Currently, OMicroB should produce a binary compatible with numworks, but that does not even start the OCaml interpreter. Maybe you can give it a try ? You can configure omicrob with ./configure -target numworks, and then use make in the subfolder targets/numworks/tests/hello. The hex file you get should be compatible with the .nwa format.

Naereen commented 4 days ago

It looks like the changes I started to implement on OMicroB, only more mature and cleaner, thanks! So for the hello.ml file you wrote, it should be compiled as a standalone NWA app for my Numworks if it works well... but it's maybe just gonna print, and return directly, so I don't think I'll see anything (if it works, it's gonna be so quick than the Numworks UI will be displayed right after it's done, erasing the printed message "Hello?").

Naereen commented 4 days ago

I think the microbit_print_string function is undefined, I got the following error when running the targets/numworks/tests/hello/Makefile make command:

/usr/bin/ld : /home/lilian/publis/OMicroB.git/lib/libcamlrun.a(bindings.o) : dans la fonction « caml_numworks_print_string » 
bindings.c:(.text+0x22b) : référence indéfinie vers « microbit_print_string »
collect2: error: ld returned 1 exit status
File "hello.ml", line 1:
Error: Error while building custom runtime system
make: *** [/home/lilian/publis/OMicroB.git/targets/numworks/tests/hello_world/Makefile:11 : hello.elf] Erreur 2
make : on quitte le répertoire « /home/lilian/publis/OMicroB.git/targets/numworks/tests/hello_world »
Naereen commented 4 days ago

I tweaked a few things:

Naereen commented 4 days ago

With these changes, the hello.hex compiles without any errors. But when I try to upload it (flash it) to my calculator, on the official flashing website for external apps (https://my.numworks.com/apps), I get many errors:

apps:1 Uncaught (in promise) ./this.program: Missing api level. Please define a .rodata.eadk_api_level section.
./this.program: Missing app name. Please define a .rodata.eadk_app_name section.
./this.program: Missing app icon. Please define a .rodata.eadk_app_icon section.
./this.program: app.nwa: in function `caml_cos_float':
/home/lilian/publis/OMicroB.git/src/byterun/vm/float.c:186: undefined reference to `cos'
./this.program: app.nwa: in function `caml_sin_float':
/home/lilian/publis/OMicroB.git/src/byterun/vm/float.c:190: undefined reference to `sin'
./this.program: app.nwa: in function `caml_sqrt_float':
/home/lilian/publis/OMicroB.git/src/byterun/vm/float.c:198: undefined reference to `sqrt'
./this.program: app.nwa: in function `caml_atan2_float':
/home/lilian/publis/OMicroB.git/src/byterun/vm/float.c:202: undefined reference to `atan2'
./this.program: app.nwa: in function `caml_fmod_float':
/home/lilian/publis/OMicroB.git/src/byterun/vm/float.c:222: undefined reference to `fmod'
./this.program: app.nwa: in function `caml_set_bit':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:65: undefined reference to `set_bit'
./this.program: app.nwa: in function `caml_clear_bit':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:70: undefined reference to `clear_bit'
./this.program: app.nwa: in function `caml_read_bit':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:75: undefined reference to `read_bit'
./this.program: app.nwa: in function `caml_write_register':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:81: undefined reference to `write_register'
./this.program: app.nwa: in function `caml_read_register':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:86: undefined reference to `read_register'
./this.program: app.nwa: in function `caml_delay':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:92: undefined reference to `delay'
./this.program: app.nwa: in function `caml_millis':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:97: undefined reference to `millis'
./this.program: app.nwa: in function `abort':
/build/newlib-pB30de/newlib-3.3.0/build_nano/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/stdlib/../../../../../../../../newlib/libc/stdlib/abort.c:59: undefined reference to `_exit'
./this.program: app.nwa: in function `_kill_r':
/build/newlib-pB30de/newlib-3.3.0/build_nano/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/reent/../../../../../../../../newlib/libc/reent/signalr.c:53: undefined reference to `_kill'
./this.program: app.nwa: in function `_getpid_r':
/build/newlib-pB30de/newlib-3.3.0/build_nano/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/reent/../../../../../../../../newlib/libc/reent/signalr.c:83: undefined reference to `_getpid'
Vertmo commented 4 days ago

I think the microbit_print_string function is undefined, I got the following error when running the targets/numworks/tests/hello/Makefile make command:

/usr/bin/ld : /home/lilian/publis/OMicroB.git/lib/libcamlrun.a(bindings.o) : dans la fonction « caml_numworks_print_string » 
bindings.c:(.text+0x22b) : référence indéfinie vers « microbit_print_string »
collect2: error: ld returned 1 exit status
File "hello.ml", line 1:
Error: Error while building custom runtime system
make: *** [/home/lilian/publis/OMicroB.git/targets/numworks/tests/hello_world/Makefile:11 : hello.elf] Erreur 2
make : on quitte le répertoire « /home/lilian/publis/OMicroB.git/targets/numworks/tests/hello_world »

Oh right, this is a mistake on my part. I will fix it.

With these changes, the hello.hex compiles without any errors. But when I try to upload it (flash it) to my calculator, on the official flashing website for external apps (https://my.numworks.com/apps), I get many errors:

apps:1 Uncaught (in promise) ./this.program: Missing api level. Please define a .rodata.eadk_api_level section.
./this.program: Missing app name. Please define a .rodata.eadk_app_name section.
./this.program: Missing app icon. Please define a .rodata.eadk_app_icon section.
./this.program: app.nwa: in function `caml_cos_float':
/home/lilian/publis/OMicroB.git/src/byterun/vm/float.c:186: undefined reference to `cos'
./this.program: app.nwa: in function `caml_sin_float':
/home/lilian/publis/OMicroB.git/src/byterun/vm/float.c:190: undefined reference to `sin'
./this.program: app.nwa: in function `caml_sqrt_float':
/home/lilian/publis/OMicroB.git/src/byterun/vm/float.c:198: undefined reference to `sqrt'
./this.program: app.nwa: in function `caml_atan2_float':
/home/lilian/publis/OMicroB.git/src/byterun/vm/float.c:202: undefined reference to `atan2'
./this.program: app.nwa: in function `caml_fmod_float':
/home/lilian/publis/OMicroB.git/src/byterun/vm/float.c:222: undefined reference to `fmod'
./this.program: app.nwa: in function `caml_set_bit':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:65: undefined reference to `set_bit'
./this.program: app.nwa: in function `caml_clear_bit':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:70: undefined reference to `clear_bit'
./this.program: app.nwa: in function `caml_read_bit':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:75: undefined reference to `read_bit'
./this.program: app.nwa: in function `caml_write_register':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:81: undefined reference to `write_register'
./this.program: app.nwa: in function `caml_read_register':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:86: undefined reference to `read_register'
./this.program: app.nwa: in function `caml_delay':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:92: undefined reference to `delay'
./this.program: app.nwa: in function `caml_millis':
/home/lilian/publis/OMicroB.git/src/byterun/vm/../prims/bindings.c:97: undefined reference to `millis'
./this.program: app.nwa: in function `abort':
/build/newlib-pB30de/newlib-3.3.0/build_nano/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/stdlib/../../../../../../../../newlib/libc/stdlib/abort.c:59: undefined reference to `_exit'
./this.program: app.nwa: in function `_kill_r':
/build/newlib-pB30de/newlib-3.3.0/build_nano/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/reent/../../../../../../../../newlib/libc/reent/signalr.c:53: undefined reference to `_kill'
./this.program: app.nwa: in function `_getpid_r':
/build/newlib-pB30de/newlib-3.3.0/build_nano/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/reent/../../../../../../../../newlib/libc/reent/signalr.c:83: undefined reference to `_getpid'

I think all of these errors are due to things I forgot in the linking parameters. The first three should be easy enough to fix by following the default app Makefile. The other ones probably require adding some - lm,... flags. I will look into it. Is it possible for me to test this flashing website without owning a calculator?

Naereen commented 3 days ago
Naereen commented 3 days ago
./this.program: Missing api level. Please define a .rodata.eadk_api_level section.
./this.program: Missing app name. Please define a .rodata.eadk_app_name section.
./this.program: Missing app icon. Please define a .rodata.eadk_app_icon section.

These should be added to the file defining the main function, I guess.

Vertmo commented 3 days ago

I've pushed a new commit which should hopefully fix all of that, let me know if that works :)

Naereen commented 3 days ago

Who nice, the names and icons are correctly found on the flashing website. It doesn't let me load the app, and it gives this error:

./this.program: app.nwa: in function `_sbrk':
/build/newlib-pB30de/newlib-3.3.0/build/arm-none-eabi/thumb/v7e-m+fp/hard/libgloss/libnosys/../../../../../../../libgloss/libnosys/sbrk.c:21: undefined reference to `end'

I guess the issue could be solved by what was proposed in the main.c file of the nwagyu/lua app (https://github.com/nwagyu/lua/blob/master/src/main.c#L10):

// TODO: Check why __exidx_start/__exidx_end is needed
void __exidx_start() { }
void __exidx_end() { }

I'll try that!

Naereen commented 3 days ago

It didn't work.The issue is harder. https://stackoverflow.com/questions/5764414/undefined-reference-to-sbrk seems to reference this same issue, and I'm talking to a very skilled person on the Omega (a fork of Epsilon, the OS of Numworks) Discord server. Apparently we should find a way to have a end defined by the linker, as stated here : libgloss/libnosys/sbrk.c:10

In the options given to the linker, we have -specs=nano.specs, and apparently we should add -specs=nosys.specs as well. However, no matter how I try that (both, or only the nosys) in the default example C app, the same bug of undefined reference to 'end' appears, on the flashing website.

Vertmo commented 3 days ago

I'm not sure I understand, does that mean the example app (https://github.com/numworks/epsilon-sample-app-c) does not compile as is ? If it does, what is the fundamental difference between it and our binary (when using nano.specs, as I've pushed in the last commit) ? Could you give me the error message you get with the latest commit ?

Naereen commented 3 days ago

Exactly, the example app does not compile when the linker has the option -specs=nosys.specs (either if it is alone or with nano.specs). I dont know how to proceed, I'm looking it up.

With the latest commit, the flashing website gives this error:

./this.program: app.nwa: in function `abort':
/build/newlib-pB30de/newlib-3.3.0/build_nano/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/stdlib/../../../../../../../../newlib/libc/stdlib/abort.c:59: undefined reference to `_exit'
./this.program: app.nwa: in function `_kill_r':
/build/newlib-pB30de/newlib-3.3.0/build_nano/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/reent/../../../../../../../../newlib/libc/reent/signalr.c:53: undefined reference to `_kill'
./this.program: app.nwa: in function `_getpid_r':
/build/newlib-pB30de/newlib-3.3.0/build_nano/arm-none-eabi/thumb/v7e-m+fp/hard/newlib/libc/reent/../../../../../../../../newlib/libc/reent/signalr.c:83: undefined reference to `_getpid'

It seems even worse :sob:

Vertmo commented 3 days ago

No, I think we're making progress. I pushed a new commit with these three stub functions. Does it work better ?

Naereen commented 3 days ago

YEAS! It works! I tweaked the hello.ml file in order to display the "Hello?" string for quite some time, and it does get printed on the screen when launching the app!

Vertmo commented 3 days ago

Ah good. Do you mean that with the version I put (which already had a while true loop at the end), the string did not stay on the screen ? How did you change the ml program exactly ?

Naereen commented 3 days ago

I was afraid that the version you had would be impossible to exit, so my hello.ml has a (long) for loop instead.

Vertmo commented 3 days ago

Oh right, that makes sense. I dont know exactly how "premptive" the OS of the calculator is, so it's probably a bad idea to block forever you're right. I think that means Step 1 is done.

A good Step 2 would be to add some well chosen primitives to allow for writing interactive programs (like polling for the buttons, drawing on the screen, etc). I can give you some pointers on how to implement these if you want. We could also port some of the existing programs and benchmarks to see the how much power we're dealing with compared to what we had previously.

Then we can start thinking about the larger goal, writing a REPL (Read-Eval-Print-Loop) that runs on the calculator. The main difficulty will be to fit the OCaml compiler in there of course.

Naereen commented 3 days ago

Congrats for making it work, it's already a good step forward!

Naereen commented 3 days ago

For step 2, maybe we could first focus on a non interactive system.

For example, there exists plenty of "tiny OCaml-like language" implemented in OCaml itself, like for instance a student project Mini Caml Interpreter. I've managed to recompile their project using my OCaml v4.14.0 opam switch (and writing a dune project for it), for my x86_64 architecture.

Naereen commented 3 days ago

I think if we can have a tiny OCaml-like interpreter that executes a given string, it shouldn't be impossible to connect it to the local filesystem of the calculator. There exists a project offering a C library that exposes a few C functions to connect to the local storage of the calculator, where the Python scripts are found for instance. See https://framagit.org/Yaya.Cout/numworks-extapp-storage/ and src/storage.c for reference. Note: I'm talking to its developer since Saturday, he/she is very reactive and helpful.

The main idea here is that we could leverage the power of the internal Python editor! I can imagine that we could easily ask the users to create/edit a "ocaml.py" file, using the (great) Python editor app, then to start the "OCaml interpreter" app we will produce, and the app will be in charge of interpreting the content of this file, without any interactive REPL or file-selection menu.

Naereen commented 3 days ago

Just to show you, because it's nice to have achieved together this first step :tada: ! Capture d’écran_2024-10-23_14-45-03

Naereen commented 3 days ago

I started writing additional OCaml functions, to see a little bit how it works.

Naereen commented 3 days ago

I'm struggling to understand what files to modify, between numworks.ml/numworks.mli and bindings.c and arch-specific.c. I've managed to update and fix the millis() function, calling the correct function of the Numworks EADK library (from eadk.h).

But now I'm trying to add a val display_draw_string : string -> int -> int -> unit in the numworks.mli file (with correct lines in the three others). I can compile omicrob with no issue, but when compiling the hello_world example (which now is a test of all the new functions I defined), i get this linking error:

/usr/bin/ld : /home/lilian/publis/OMicroB.git/lib/libcamlrun.a(bindings.o) : dans la fonction « caml_delay_usec » 
bindings.c:(.text+0x2cb) : référence indéfinie vers « delay_usec »
/usr/bin/ld : /home/lilian/publis/OMicroB.git/lib/libcamlrun.a(bindings.o) : dans la fonction « caml_display_draw_string » 
bindings.c:(.text+0x2f1) : référence indéfinie vers « display_draw_string »
collect2: error: ld returned 1 exit status
File "hello.ml", line 1:
Error: Error while building custom runtime system
make: *** [/home/lilian/publis/OMicroB.git/targets/numworks/tests/hello_world/Makefile:12 : hello.elf] Erreur 2

So the hello.ml > numworks.ml(i) part works, and it gets to the bindings.c file, which should use the definitions written in arch-specific.c, except it does not. Is there a part of the main OMicroB project which defines the list of functions that can be available in bindings.c and/or arch-specific.c ? It's more complicated than I expected.

Vertmo commented 3 days ago

That's because this specific compilation pass is for the PC binary, which arch-specific.c is not used to produce. You need to add simulation functions in targets/numworks/byterun/simul/sf-regs.c. These will be compiled with the regular GCC to be ran on your computer, so you may not use eadk functions.

Naereen commented 3 days ago

Hm could we not implement "fake" UI functions of the Numworks in the simulator? I don't really care about the simulator actually.

Vertmo commented 3 days ago

Yes totally, you can just put stubs that do nothing, or a simple printf.

Naereen commented 3 days ago

I successfully added a display_draw_string (text : string) (x : int) (y : int). It "works" fine, proving that it is possible to add in the bindings.c functions that refer to the EADK lib (the developer C lib for the Numworks).

Naereen commented 3 days ago

I simply have this in the Makefile for hello:

# TARGETS := $(PROG).elf $(PROG).hex
TARGETS := $(PROG).hex

I only compile the real app hello.nwa, not the simulator one hello.elf for PC.

Naereen commented 3 days ago

I wonder what direction I should pursue now...

Let me know what you think! And thanks again for the very useful help!

Naereen commented 3 days ago

Also, maybe could I have write access to this repository? Or I use a fork and work there.

Naereen commented 2 days ago

I work on this fork: https://github.com/Naereen/OMicroB/tree/numworks (https://github.com/stevenvar/OMicroB/compare/numworks...Naereen:OMicroB:numworks for the comparison between your branch and mine, if you're curious)

Naereen commented 2 days ago
Naereen commented 2 days ago

@Vertmo do you think there could be a way to define the name of the compiled app from the Makefile or the source.ml, and not have it to be constant to "OMicroB OCaml" (the name I chose, modifying a bit the one you chose yesterday) ?

Naereen commented 2 days ago

Huho, I'm encountering an issue with the various functions that are in caml/alloc.h : they are not found when compiling the hello.ml program (which uses the bindings.c file where I'm trying to use for example caml_copy_string to create in C an OCaml value of type (internal) value, from a string. I don't understand why all the rest worked, the macro Val_string, the function caml_string_length for instance... If @Vertmo you don't have any idea, I could ask on stackoverflow or on the Discuss.OCaml.org discussion page.

Naereen commented 2 days ago

I guess I'm starting to understand... in the VM implementation, there is actually no way of creating (in C) an OCaml value which is a string with dynamic content (not statically known in advance), right? I read the src/bc2c/printer.ml file which produces some lines filled with calls to Make_string_data(...) for all the strings that should be defined statically. I read the hello.c file, produced by the bc2c binary, and it contains many lines of Make_string_data(c0,c1,c2,c3) (4 chars).

Naereen commented 2 days ago

I uploaded here (https://github.com/Naereen/OMicroB/issues/1#issuecomment-2434423039) a tiny video which shows the very tiny success of porting my old https://github.com/Naereen/Tiny-Prolog-in-OCaml-OneFile baby-Prolog interpreter to the Numworks. So far, it does not read the theory and the questions from the local file system, as I've failed to implement the bindings.c: value caml_read_any_file(value v) function (don't know how to produce a value in C which corresponds to a given string in OCaml).

Vertmo commented 2 days ago

For the write access, I wouldn't be opposed to it, but ultimately it's @stevenvar repo and decision ^^. For now feel free to make periodic pull requests on this repo's numworks branch from yours, and I will integrate them.

@Vertmo do you think there could be a way to define the name of the compiled app from the Makefile or the source.ml, and not have it to be constant to "OMicroB OCaml" (the name I chose, modifying a bit the one you chose yesterday) ?

Since it's defined in a C file, we can probably do it using a macro with the -D option at compilation

Huho, I'm encountering an issue with the various functions that are in caml/alloc.h : they are not found when compiling the hello.ml program (which uses the bindings.c file where I'm trying to use for example caml_copy_string to create in C an OCaml value of type (internal) value, from a string. I don't understand why all the rest worked, the macro Val_string, the function caml_string_length for instance... If @Vertmo you don't have any idea, I could ask on stackoverflow or on the Discuss.OCaml.org discussion page.

Hum, I would need to take a look at it... Do you mean its a linking issue ?

I guess I'm starting to understand... in the VM implementation, there is actually no way of creating (in C) an OCaml value which is a string with dynamic content (not statically known in advance), right? I read the src/bc2c/printer.ml file which produces some lines filled with calls to Make_string_data(...) for all the strings that should be defined statically. I read the hello.c file, produced by the bc2c binary, and it contains many lines of Make_string_data(c0,c1,c2,c3) (4 chars).

The subtlety is that strings can either be stored in the ROM (when they are static in the program), or in RAM (if you want to allocate them dynamically). If you want to do the second one, I guess you could use the create_bytes function to allocate the string, and then either String_field or Ram_string_field to read/write to it.

Naereen commented 2 days ago

Thanks for the reply.

Naereen commented 2 days ago

I've finished my first example of a "real" app written using OCaml and OMicroB for the Numworks: see https://github.com/Naereen/OMicroB/issues/1#issuecomment-2435319375 for a description, it now works very well.

I'm wondering, what could be the upper limit for the -stack-size and -heap-size parameters that omicrob accepts, for the target of the Numworks calculators? I'm starting to adapt the https://github.com/ViRoLam/Mini-Caml-Interpreter project (I'm at least trying), and I guess the code of just the parser/lexer will be huuuge and require a lot of storage/memory.

Vertmo commented 2 days ago

I'm wondering, what could be the upper limit for the -stack-size and -heap-size parameters that omicrob accepts, for the target of the Numworks calculators? I'm starting to adapt the https://github.com/ViRoLam/Mini-Caml-Interpreter project (I'm at least trying), and I guess the code of just the parser/lexer will be huuuge and require a lot of storage/memory.

Good question. Normally the heap side is in number of words. On the Numworks each word is 4 bytes, so I would say the absolute maximum you can get is (stack size + heap size) * 4 = 256k. Of course you have to account for the memory used for actually running the vm, the driver functions, etc. Also keep in mind that if you use the Stop and Copy Garbage Collector (which I believe is on by default), you essentially divide the avaliable heap space by 2.

So yes, we might run in some memory issues at some point, but it's worth trying, and then seeing if we can optimize the interpreter.

Have you considered porting a mini-ml compiler rather than an interpreter? That way we could use the Omicrob VM itself as the interpreter, which would most likely be more efficient.

Naereen commented 2 days ago

I'll first ask another question (sorry!), do you know what it means to obtain such an error, when calling bc2c in the process of compiling with omicrob?

$ ./bin/bc2c -local -stack-size 4096 -heap-size 4096 -gc MAC -arch 32 minicaml.byte -o minicaml.c
Warning: unknown primitive "caml_ensure_stack_capacity"(314).
Error: PUSHOFFSETCLOSURE 166, argument out of bounds [ -128 .. 127 ].

I guess my code of minicaml.ml (taken and adapted from https://github.com/ViRoLam/Mini-Caml-Interpreter) asks for too much of some memory, but I don't know where, why or so on.

Naereen commented 2 days ago

Thanks for the detail, regarding stack-size and heap-size.

Have you considered porting a mini-ml compiler rather than an interpreter? That way we could use the Omicrob VM itself as the interpreter, which would most likely be more efficient.

I did not. I have more experience with interpreters, and this way I thought we could avoid saving some (binary?) content to the local storage of the Numworks.

Vertmo commented 1 day ago

I'll first ask another question (sorry!), do you know what it means to obtain such an error, when calling bc2c in the process of compiling with omicrob?

$ ./bin/bc2c -local -stack-size 4096 -heap-size 4096 -gc MAC -arch 32 minicaml.byte -o minicaml.c
Warning: unknown primitive "caml_ensure_stack_capacity"(314).
Error: PUSHOFFSETCLOSURE 166, argument out of bounds [ -128 .. 127 ].

I guess my code of minicaml.ml (taken and adapted from https://github.com/ViRoLam/Mini-Caml-Interpreter) asks for too much of some memory, but I don't know where, why or so on.

Oof, this one is not easy. I'm not 100% sure, but I think this function is related to the partial application mechanism of OCaml. But I dont understand why the bounds would be lower than what the compiler produces. I think we need @bvaugon on this one :)

Naereen commented 1 day ago

The mentioned "Error: PUSHOFFSETCLOSURE 166, ..." error comes from src/bc2c/codegen.ml:91:

89-  let check_bounds op n low high =
90-    if n < low || n > high then
91:      Tools.fail "%s %d, argument out of bounds [ %d .. %d ]" op n low high in

The call which gives the error is at src/bc2c/codegen.ml:253:

    | STD (PUSHOFFSETCLOSURE n) ->
      check_bounds "PUSHOFFSETCLOSURE" n (-0x80) 0x7F;
      export_opcode Opcode.PUSHOFFSETCLOSURE;
      export_int8 n;

So what I understand is that at some point in the bc2c (bytecode to C generation) process, some instruction is read as a STD (PUSHOFFSETCLOSURE n) with a value n that is 166 (in the error above), outside of the possible [-128..127] interval. Thus the corresponding bytecode cannot be produced.

I guess it's because my OCaml code (that I've compiled to bytecode already) is "too complicated" or too hungry for recursions, or closures. So far, I don't understand more about that issue. I imagine it would be possible to simply use a larger datatype for the STD (PUSHOFFSETCLOSURE n) opcode? Like a int16 or int32 instead of an int8? Was it mandatory to require a PUSHOFFSETCLOSURE to be "as small as possible"?

Naereen commented 1 day ago