The meta converter language does not support repeated conversions, which is required if you need some complex rewrite logic (e.g. proof assistants). This idea is an attempt to separate the parsing step and use a more general system for tree transformation.
Piston-Meta already uses a JSON converter to debug meta data. Since Dyon is very similar to Javascript, perhaps converting meta data to a Dyon data structure directly would make it easier to work with it? This structure is easy to debug, and because Dyon is dynamically typed it is easy to transform the structure gradually, without using too much memory.
Instead of using a meta converter language, one could generate the JSON equivalent representation of the meta data. The source range could be stored as secrets, thereby keeping the trace back to the source even under complicated transformations.
println(sec(data[0].x)) // prints `some([(0, 1)])`
Unsolved problem: How to repeat meta data
In the example above, I assumed the following problem was solved:
The JSON representation in Piston-Meta is not correct for all input, since it replaces the same keys with new values when a sub rule is repeated within same parent node.
To solve this problem, it might require a change to Piston-Meta. It need to pass on information about data generated from repetition rules. Repetition rules are .l (lines), .r (repeat), .s (separated by).
One challenge is that recursion can also be repetitive when calling a sub rule without generating a new node. Perhaps this would give ideas to solve problems with left recursion?
Add support for secrets to str
Currently, Dyon only supports secrets for bool and f64. This is used in the mathematical loops.
To trace back text read from an input file, it requires support for str secrets in Dyon.
Add sec intrinsic to reveal secret of bool, f64 and str
Get the secret of bool, f64 and str. Returns none() if there is no secret.
fn sec(any) -> opt[[any]] { ... }
This is required because the existing functions are type specific and does not work in all cases for parsing. why checks for true and where checks for non-nan. These are designed to be used with mathematical loops and detect logical bugs at run-time. Need the more general sec to do this on all parsed data.
Rewriting and transformation
The following strategies can be used to transform the Dyon structure into a target structure:
Visitor pattern with closures to walk over the tree
Rewrite nodes using depth-first
Converter language could generate Dyon => Dyon
Add if <pat> ?= <rexpr> { ... } for pattern matching
An idea is to add some sugar for pattern matching to the Dyon language, in order to make tree transformations easier to deal with:
x := [1, 2]
if [a, b] ?= x {
println(a)
println(b)
}
For functions that returns res, one could use <pat> ?= <rexpr>:
One way to solve the repetition problem is to parse the meta rules using the self-syntax in Piston-Meta, and then run an analysis that outputs a hash set of nodes that are repeating.
Alternative to https://github.com/PistonDevelopers/dyon/issues/182
The meta converter language does not support repeated conversions, which is required if you need some complex rewrite logic (e.g. proof assistants). This idea is an attempt to separate the parsing step and use a more general system for tree transformation.
Piston-Meta already uses a JSON converter to debug meta data. Since Dyon is very similar to Javascript, perhaps converting meta data to a Dyon data structure directly would make it easier to work with it? This structure is easy to debug, and because Dyon is dynamically typed it is easy to transform the structure gradually, without using too much memory.
Instead of using a meta converter language, one could generate the JSON equivalent representation of the meta data. The source range could be stored as secrets, thereby keeping the trace back to the source even under complicated transformations.
Example
Similar example to https://github.com/PistonDevelopers/dyon/issues/182
You want to parse a custom format looking like this:
assets/test_data.txt
So, you write a meta syntax file:
assets/test_meta.txt
In Dyon, you call
load__meta_file
:This generates the Dyon data:
To get the source range in characters:
Unsolved problem: How to repeat meta data
In the example above, I assumed the following problem was solved:
The JSON representation in Piston-Meta is not correct for all input, since it replaces the same keys with new values when a sub rule is repeated within same parent node.
To solve this problem, it might require a change to Piston-Meta. It need to pass on information about data generated from repetition rules. Repetition rules are
.l
(lines),.r
(repeat),.s
(separated by).One challenge is that recursion can also be repetitive when calling a sub rule without generating a new node. Perhaps this would give ideas to solve problems with left recursion?
Add support for secrets to
str
Currently, Dyon only supports secrets for
bool
andf64
. This is used in the mathematical loops.To trace back text read from an input file, it requires support for
str
secrets in Dyon.Add
sec
intrinsic to reveal secret ofbool
,f64
andstr
Get the secret of
bool
,f64
andstr
. Returnsnone()
if there is no secret.This is required because the existing functions are type specific and does not work in all cases for parsing.
why
checks fortrue
andwhere
checks for non-nan. These are designed to be used with mathematical loops and detect logical bugs at run-time. Need the more generalsec
to do this on all parsed data.Rewriting and transformation
The following strategies can be used to transform the Dyon structure into a target structure:
Dyon => Dyon
Add
if <pat> ?= <rexpr> { ... }
for pattern matchingAn idea is to add some sugar for pattern matching to the Dyon language, in order to make tree transformations easier to deal with:
For functions that returns
res
, one could use<pat> ?= <rexpr>
: