debug locations

[vitiral]

There are a lot of possible ways to find the location of an instruction in a source file. You could:

1. Have an instruction for updating the position (U2). This has cost: both in code size AND in execution time (bad!)
2. Keep some kind of detailed token map with pointers to the code. This would be pretty bloated
3. Have a byte array which maps 1:1 with the function body. To find the position, add bytes until you get to the same offset. This will be called posData (positional data)

The latter one has several benefits:

• Is guaranteed to only require at exactly twice the memory usage of the function. (1) may have used less memory on average, but could in theory use more -- and costs execution time. (2) would be absolutely bloated in most cases (at least a pointer per token)
• Can more easily be thrown away, whereas (1) would be difficult to discard.
• Is extremely easy to implement

The basic implementation:

• The scanner and token dropper will take care of updating the currentPos in the global compiler.
• Functions will have a pos value, which is the position that the fn token is in
• the compileTy function will take care of actually updating the posData
• It occurs to me that this does not work with text macros

Option 3 doesn't work with text macros. Let's go with Option 2

How do I design a text macro system that works well with this? I would like to be able to view the expansions of text macros when compilation fails

Well, I think it's fairly obvious isn't it? Compile text macros to a file... then parse them! In fact: compile all text macros to an append-only file. Don't use memory for text macros, as that could quickly explode our available memory.

This leaves us with (2) -- but we have a secret weapon: our debug symbols can also be stored in a file, aka they will be byte-serialized to a file. This will happen eventually: for now, they will be malloc'd to aid in simplicity.

So when walking the debug symbols:

• Keep a stack of files and positions, starting at the function definition (typically a source text file) and expanding to macro text files.
• Walk the debug symbols until we get to the location we are looking for, keeping track of the function stack / position locations. The debug symbols can have various commands like "push file", "pop file", "add position" etc. They will be their own tiny vm!