search

DavidKinder / Inform6

The latest version of the Inform 6 compiler, used for generating interactive fiction games.
http://inform-fiction.org/
Other
204 stars 34 forks source link

put_token_back() reverts symbols #235

Closed erkyrath closed 1 year ago

erkyrath commented 1 year ago

If we call get_next_token() and it creates a symbol, but then you call put_token_back(), it uncreates the symbol.

(Really it just unlinks the symbol from the symbol-hash-table, so it can never be recognized in the future.)

This fixes https://github.com/DavidKinder/Inform6/issues/163 , https://github.com/DavidKinder/Inform6/issues/201 , and https://github.com/DavidKinder/Inform6/issues/231 (without the hack I used to fix 231 before). And https://github.com/DavidKinder/Inform6/issues/233 of course.

Under the hood:

symbol_index() now has an extra out-argument *created which returns whether the symbol already existed (false), or had to be created (true). If you don't care about this, pass NULL.

The only caller that cares about this is interpret_identifier() in the lexer. This now stashes the flag in the lexeme ring as newsymbol. (Only for tokens that are symbols, of course.)

In put_token_back(), if we're putting back a symbol with newsymbol set, we call end_symbol_scope() to nuke it. We'll be seeing it again very soon (get_next_token() calls interpret_identifier()), but maybe dont_enter_into_symbol_table will be true next time.

As part of this work, I moved the check for dont_enter_into_symbol_table inside interpret_identifier(). This is the case where we want to return UQ_TT (unquoted string) instead of SYMBOL_TT. This makes interpret_identifier() quite a bit simpler; I was able to get rid of the confusing dirs_only_flag argument entirely. And a goto!

On the other hand, I had to repeat the keyword search loop for the extra-special case dont_enter_into_symbol_table == -2 (recognize directive keywords only).

Several places that called symbol_index(), but required the symbol to exist, now call get_symbol_index() instead. (This is an alternative that just searches for a symbol, without trying to create it.) I also check the return value of that, which means a bit more error-checking code.

I commented all the _SFLAG definitions in header.h.

erkyrath commented 1 year ago

This does not change any compiled code, except that in one case OMIT_UNUSED_ROUTINES actually became smarter! In library_of_horror-36.inf (with punylib), the Amusing symbol was wrongly left in the symbol table, which led to the Amusing() stub not being omitted even though it was never called. Now the stub is omitted, saving two bytes.