Closed GoogleCodeExporter closed 9 years ago
Actually, it plain old needs to be written :-)
Right now I spot check the compiler by running various programs from the
Examples/Compiler and VM Tests folder
(http://code.google.com/p/epoch-language/source/browse/?repo=examples#hg%2FCompi
ler%20and%20VM%20Tests). There really needs to be a way to run those in a
single batch and check the compiled bytecode output for sanity.
I'm waffling over whether to have the compiler test be strictly on compiled
files or if the test should be executed as well, although at that point we're
honestly blurring the lines between testing the compiler and testing the VM.
Part of the reason I haven't automated all of this yet is that regression
testing is nigh on worthless with the rate that syntax and semantic changes are
going into the language. Even the bytecode structure is fluid enough still that
just doing a naive comparison of compiler outputs will generate a lot of false
failures.
I'm totally open to input on how to proceed with this. As Release 12 gets
closer and the new compiler is more or less locked down, I really want to see a
way to make sure it doesn't get subtly broken in the future.
It'd also be nice to have a real validation suite for when I start working on
the self-hosting compiler.
Original comment by don.ap...@gmail.com
on 15 Feb 2012 at 5:23
Well, the first thing is that I don't see any sort of operator precedence.
Which means that a simple expression like:
3+5*4
will produce the wrong results, it ends up grouped as: (3+5)*4, instead of the
expected 3+(5*4). This has to do with the dynamic operator syntax, and the fact
that the library registers the operators.
It is my opinion that the base operators for types (*,/,+,-,++,--,<<,>>,!,~)
should all be built in with a defined precedence order. Baring that, the
operator overloading syntax needs to be redesigned to utilize a couple of
precedence levels (perhaps 3 levels... */ at the highest, then +-++--, then <<
>> (or however you want the shift operators to look)).
In EBNF this looks something like...
expr = expr + term | expr - term | term
term = term * factor | term / factor | factor
factor = identifier | literalNumber | '(' expr ')'
for a basic grammar. Also need to toss in prefix and postfix operators, etc.
Original comment by ryoohki@gmail.com
on 15 Feb 2012 at 6:12
AFter that, and once we have a basic grammar in place, the next thing to do
should be to compose a complete set of tests for each major language component.
We can create a Visual Studio project (an empty project) that has all the epoch
files in it, with a compiler event that executes a batch file which copies the
tests to a bin folder (or doesn't) and then runs the various EpochTools
/execute commands...
On that note, I don't like having to run /execute. It should just be
"EpochTools epochfile" dammit.
ALso on that note, we should get away from MessageBox calls, they're horrible
for running tests as they will stop all execution till you address them.
Instead perhaps redirect to std::cerr, then you can use piping to produce a log
file... epochtools test.epoch 2>> errors.txt
Original comment by ryoohki@gmail.com
on 15 Feb 2012 at 6:23
Operator precedence is a known issue. There's a TODO lurking somewhere deep in
the semantic checking code for reimplementing operation reordering to obey
precedence rules. It's kept dynamic to allow for implementing user-defined
operators easily, and to allow new operators to be added/modified by the
library implementation (at any precedence) without having to shuffle the
grammar around.
That said, I might be able to be convinced to embed the precedence ordering
directly in the grammar, but I might not ;-)
I'll see about making the other changes as well; can't promise anything as my
task list is already a mile long but they're good points.
I'll definitely also be thinking about ways to get a good automated testing
setup like you describe.
Original comment by don.ap...@gmail.com
on 15 Feb 2012 at 7:06
Posting the task list would help :P especially if you use the issues list. An
issue per task :P
Then as they get knocked out we can close them. It also means we can use issues
as discussion points around the language to find solutions or alternatives.
Original comment by ryoohki@gmail.com
on 15 Feb 2012 at 7:17
Yeah, now that I'm not flying this thing solo I really need to get more
organized. I'll start posting up issues for everything I know needs work.
Original comment by don.ap...@gmail.com
on 15 Feb 2012 at 7:21
How do we want to report non-fatal errors. I.e. syntax errors?
Original comment by ryoohki@gmail.com
on 15 Feb 2012 at 7:46
Syntax errors or semantic errors?
Syntax errors will require parser work as I need to get the
filename/line/column tracking stuff hooked back into Spirit so I can emit
sensible errors in the first place. It's mainly a matter of writing in rules to
the grammar that catch malformed syntax and enable parsing to continue as much
as possible, which is a slow and tedious blob of work. It's definitely
something that needs to be addressed, though.
Semantic errors are different in that they can happen during the semantic
validation/IR generation/code generation passes and need some kind of context
from the parser as well.
I'm thinking of writing a basic "ErrorReporting" sort of thing that
encapsulates recording error messages, their causes/locations, and so on. This
will probably start by spamming cerr and then later on get plumbed through to
display error details in the IDE and so on.
Original comment by don.ap...@gmail.com
on 15 Feb 2012 at 8:45
Well, syntax errors would be the "on_error" stuff with spirit, which is messy
as hell, but. yeah.
For Semantic errors we need a way to fail out of the semantic validator when we
do encounter an error though.
My initial thought would be to store some sort of std::vector<Error> and test
that its non-empty before continuing on to the compilation/bytecode generation
Original comment by ryoohki@gmail.com
on 15 Feb 2012 at 8:51
Now that we have a proper test suite in the Examples repo I'm going to go ahead
and close this; we certainly could use more test cases in the repo but I'll be
adding them over time as I get features fixed up in the Release 12 compiler.
So far this has already caught a good half-dozen issues and helped fix all of
them without causing collateral breakage, so I'm happy.
Original comment by don.ap...@gmail.com
on 31 Mar 2012 at 4:45
Original issue reported on code.google.com by
ryoohki@gmail.com
on 13 Feb 2012 at 8:20