This PR introduces OCaml support by extracting the lambda IR from the ocaml compiler, parsing it into a rust enum, then transforming it into an equivalent lurk program. Note that a lot of the behavior will change and this is still experimental and incomplete.
This adds two new meta commands: !(load-ocaml) which functions like !(load) and !(load-ocaml-expr) which functions like !(load-expr). The former will run the program and print its result, while the latter returns the transformed lurk program.
You can do things like !(load-ocaml "file.ml") !(debug) !(prove), !(debug !(load-ocaml-expr "file.ml")) or !(defq program !(load-ocaml-expr "file.ml")). The file must end in a .ml extension, and currently we only support compiling a single file.
Due to the way individual .ml files are compiled, for now you will want a program like let result = 123 rather than 123. Otherwise the 123 is evaluated inside a begin and its result is not used.
The pipeline works as follows:
Create a temporary directory using tempfile to hold build artifacts. ocamlc creates .cmi and .cmo files that are not relevant to us.
Call ocamlc -dlambda -dno-unique-ids -warn-error +a -c <file>. The code must compile with no warnings. -dlambda makes the compiler output the lambda IR to stderr. -dno-unique-ids removes the unique suffixes added to identifiers by the compiler.
Parse the lambda IR into a LambdaSyntax struct. Forms that are not supported get parsed as FallbackPrimitives or FallbackLiterals.
Generate a lurk program of the form (let (<helper functions>) <ocaml expr>) using transform_lambda.
The process for supporting a new primitive is roughly:
Add an entry to LambdaSyntax
Modify ocaml::parser::syntax::parse_primitive_sexp to correctly parse the sexp into the new type (optionally add a parser test)
Handle the case in ocaml::compile::transform_lambda
Add a test case to lurk::tests::eval_ocaml
Currently we support:
Basic data types: ints, chars, strings, records (no field access yet), functions
let and let rec. seq as begin. function calls
Primitives with a direct lurk version: =, >, etc (though ocaml supports comparing chars and strings with these)
Records, arrays, lists, tuples and etc will initially be represented by simple lists and accessed with nth. A record [0: 123 "abc"] becomes (list 0 123 "abc"), so a record is a list of its tag followed by its members.
It's possible the -dno-unique-ids option could lead to issues in certain programs, but for now we're using it by default. We might need to revisit this and use a different solution if we find a problematic program with this flag.
OCaml tests can be added inline or as files to src/lurk/tests/ml/ and a corresponding test_file! call in tests/eval_ocaml.rs.
Known issues/limitations:
OCaml true/false are represented sometimes by 1/0 ints, which can lead to problems if used as conditions with if
Floats are not supported and will add explicit (fail)s in the lurk program
Calling stdlib functions does not work
Integers are represented as u64s and negative integers are not supported yet
This PR introduces OCaml support by extracting the lambda IR from the ocaml compiler, parsing it into a rust enum, then transforming it into an equivalent lurk program. Note that a lot of the behavior will change and this is still experimental and incomplete.
This adds two new meta commands:
!(load-ocaml)which functions like!(load)and!(load-ocaml-expr)which functions like!(load-expr). The former will run the program and print its result, while the latter returns the transformed lurk program.You can do things like
!(load-ocaml "file.ml") !(debug) !(prove),!(debug !(load-ocaml-expr "file.ml"))or!(defq program !(load-ocaml-expr "file.ml")). The file must end in a.mlextension, and currently we only support compiling a single file.Due to the way individual
.mlfiles are compiled, for now you will want a program likelet result = 123rather than123. Otherwise the123is evaluated inside abeginand its result is not used.The pipeline works as follows:
ocamlccreates.cmiand.cmofiles that are not relevant to us.ocamlc -dlambda -dno-unique-ids -warn-error +a -c <file>. The code must compile with no warnings.-dlambdamakes the compiler output the lambda IR to stderr.-dno-unique-idsremoves the unique suffixes added to identifiers by the compiler.LambdaSyntaxstruct. Forms that are not supported get parsed asFallbackPrimitives orFallbackLiterals.(let (<helper functions>) <ocaml expr>)usingtransform_lambda.The process for supporting a new primitive is roughly:
LambdaSyntaxocaml::parser::syntax::parse_primitive_sexpto correctly parse the sexp into the new type (optionally add a parser test)ocaml::compile::transform_lambdalurk::tests::eval_ocamlCurrently we support:
letandlet rec.seqasbegin. function calls=,>, etc (though ocaml supports comparing chars and strings with these)Records, arrays, lists, tuples and etc will initially be represented by simple lists and accessed with
nth. A record[0: 123 "abc"]becomes(list 0 123 "abc"), so a record is a list of its tag followed by its members.It's possible the
-dno-unique-idsoption could lead to issues in certain programs, but for now we're using it by default. We might need to revisit this and use a different solution if we find a problematic program with this flag.OCaml tests can be added inline or as files to
src/lurk/tests/ml/and a correspondingtest_file!call intests/eval_ocaml.rs.Known issues/limitations:
true/falseare represented sometimes by1/0ints, which can lead to problems if used as conditions withif(fail)s in the lurk programu64s and negative integers are not supported yetmatchstatements are not supported