search

rhaiscript / playground

A Rhai scripting playground that runs Rhai scripts using WebAssembly in a web browser.
https://rhai.rs/playground
MIT License
20 stars 5 forks source link

String interpolation #12

Closed schungx closed 3 years ago

schungx commented 3 years ago

@alvinhochun the new version will include:

To handle these, TokenizeState has a new field: is_within_text_terminated_by which is Option<char>.

Type Example Return is_within_text_terminated_by
Normal string "hello" StringConstant None
Normal string unterminated "hello LexError None
Normal string continued "hello\ StringConstant Some('"')
Literal string unterminated `hello|StringConstant|Some('`')
Literal string interpolated `hello ${|InterpolatedString|None
Literal string after interpolation `... ${...} hello`|StringConstant|Some('`')

When you see an InterpolatedString, you just push a level onto a stack, and start counting braces. The next token will be the { of the ${. Once you finish the last closing } for the level, pop the stack, set state.is_within_text_terminated_by = Some("`") to continue parsing as string.

alvinhochun commented 3 years ago

Implemented in 02c3bc7 and deployed on https://rhai.rs/playground/dev/. Does it look right?

Not related to the tokenizer but I noticed that the parser panics (panicked at 'internal error: entered unreachable code: expected a string within an interpolated string literal, but gets LexError(UnterminatedString)', D:\dev\toolchain\rust\cargo\git\checkouts\rhai-f99f240b2f7f4b04\10f334d\src\parser.rs:1050:35) if an interpolated string is not terminated, like this:

`${}
schungx commented 3 years ago

Ah, good catch! Let me check it out!

schungx commented 3 years ago

Fixed it. Forgot to check the case of unterminated string. I'll merge it to master in a bit.

Implemented in 02c3bc7 and deployed on https://rhai.rs/playground/dev/. Does it look right?

It certainly looks FINE!

schungx commented 3 years ago

BTW, there is an updated strings.rhai version with multi-level interpolated strings.

schungx commented 3 years ago

The fix has landed inhttps://github.com/rhaiscript/rhai/pull/393

It no longer should panic...

alvinhochun commented 3 years ago

Done, updated!

schungx commented 3 years ago

image

Not sure why, but if a string is not terminated at the end of the line, now it colors all lines below as text. stable doesn't do that. Seems that the is_within_text property is not property cleared, but I should have cleared it together with raising the error.

So not sure where the problem is...

schungx commented 3 years ago

Ah, I think I found where the problem is. It is in get_next_token when given an unterminated string. I'll fix it.

schungx commented 3 years ago

OK, I should have fixed the bug with https://github.com/rhaiscript/rhai/pull/394

Please rebuild again to see if it solves the problem?

alvinhochun commented 3 years ago

Updated again.

schungx commented 3 years ago

Working fine! However, just realized that I have a typo in function_decl2.rhai.

The line should be:

print(`addme(a, 4) should be 46: ${result}`);

I put the backtick in the wrong place...

schungx commented 3 years ago

@alvinhochun I think it is time to move dev into stable?

alvinhochun commented 3 years ago

Done