SubX in SubX: computing addresses for labels

[akkartik]

This branch is work in progress on implementing SubX in SubX. The big-picture plan is to divide up the work into the following phases:

$ cat file1.subx file2.subx ... |dquotes |assort |pack |survey |hex > a.elf

Each phase is an ELF binary constructed out of a corresponding .subx file.

• hex.subx converts a sequence of bytes in hex into a binary. (done)
• survey.subx will turn labels into addresses, and also slap on ELF headers. It's called 'survey' because the act of imposing a coordinate system on a program feels somewhat like surveying terrain and determining various distances and heights. (in progress here)
• pack.subx converts bitfields in instructions into whole bytes in the right order. For example, 0/mod 1/rm32 1/r32 is turned into the single byte 08. (done)
• assort.subx just reads in many segments starting with == and containing lines of code or data, and concatenates all segments with the same name. (done)
• dquotes.subx moves string literals in the code segment to the data segment. (done)

This branch is where the survey phase is being built.

To run its tests (and so see how far along things are):

$ ./subx translate 0*.subx apps/subx-common.subx apps/survey.subx -o apps/survey  &&  \
      ./subx run apps/survey test

Contact me if you'd like to contribute. Commit access freely given.

[akkartik]

This PR is done, but sadly I still get an error when compiling examples/ex1 with:

$ cat examples/ex1.subx |subx_bin run apps/dquotes |subx_bin run apps/assort |subx_bin run apps/pack |subx_bin run apps/survey

(It only fails in the final survey pipe stage. So that's something at least.)

We should probably get a sense of whether any of our examples complete without error right now.

[charles-l]

Hrmm... Yeah, I'm only getting segfaults when I run through examples right now, but now that we have tracing, it might be easier to track it down.

$ cat run.sh
for f in $(find examples -name "*.subx"); do
  echo $f
  cat $f |./subx_bin run apps/dquotes |./subx_bin run apps/assort |./subx_bin run apps/pack |./subx_bin run apps/survey
done

$ sh run.sh                                                                    
examples/ex1.subx
   0 error: Tried to access uninitialized memory at address 0x78000043\n
examples/ex10.subx
   0 error: Tried to access uninitialized memory at address 0x78000056\n
examples/ex11.subx
stream overflow
   0 error: Tried to access uninitialized memory at address 0x00000000\n
   0 error: The entire 4-byte word should be initialized and lie in a single segment.\n
   0 error: Tried to access uninitialized memory at address 0x00000000\n
   0 error: The entire 4-byte word should be initialized and lie in a single segment.\n
Segmentation fault
examples/ex12.subx
   0 error: Tried to access uninitialized memory at address 0x7800005c\n
examples/ex2.subx
   0 error: Tried to access uninitialized memory at address 0x78000042\n
examples/ex3.subx
   0 error: Tried to access uninitialized memory at address 0x78000042\n
examples/ex4.subx
   0 error: Tried to access uninitialized memory at address 0x78000042\n
examples/ex5.subx
   0 error: Tried to access uninitialized memory at address 0x7800006c\n
examples/ex6.subx
   0 error: Tried to access uninitialized memory at address 0x78000042\n
examples/ex7.subx
   0 error: Tried to access uninitialized memory at address 0x7800004c\n
examples/ex8.subx
   0 error: Tried to access uninitialized memory at address 0x78000056\n
examples/ex9.subx
   0 error: Tried to access uninitialized memory at address 0x78000056\n

[akkartik]

Are you back on master? 10/12 examples work for me now.

[akkartik]

Just FYI, things are improving rapidly on master. All example programs are now translating using this new self-hosted translator, and the resulting binaries are identical to the output of the C++ translator. That's programs with 5k+ lines so far.

I'm now trying to get the translator phases themselves through. "SubX in SubX" in "SubX in SubX".

I've been posting status updates on this thread: https://mastodon.social/@akkartik/102488929327915911

[charles-l]

@akkartik oh sweet -- yeah I gave it a shot on master and many of the examples work run through the pipeline now.

[akkartik]

All done!

https://mastodon.social/@akkartik/102495274992610155 https://mastodon.social/@akkartik/102499076373416165

[charles-l]

Whoo!!! That is an awesome milestone!!

[akkartik]

Thank you so much!

I'm taking a breath to think about what programs I want to write next on this foundation. Something silly/fun just to decompress. Any ideas?

Beyond the next few days, creating some syntactic sugar for function calls would be pretty sweet. I think it may first require some sugar for reg/mem operands.

%reg => 3/mod/direct reg/rm32
*reg => 0/mod/indirect reg/rm32
*(reg+d) => 2/mod reg/rm32 d/disp32
*(r1+r2+d) => 2/mod 4/rm32/SIB r1/base r2/index 0/scale d/disp32
*(r1+r2<<1+d) => 2/mod 4/rm32/SIB r1/base r2/index 1/scale d/disp32
etc.

If we had this, we could then have calls like foo(%EAX, 32/imm32) expand to:

ff 6/subop/push %EAX
68/push 32/imm32
e8/call foo/disp32
81 0/subop/add %ESP 8/imm32  # num args * 4

So that's one idea.

I'm also tempted to push on https://github.com/ivandavidov/minimal and see how much I can learn from it.