Infrastructure: Additional Directive Utilities - [merged] #684

Closed benloh closed 1 year ago

In GitLab by @daveseah on Jun 28, 2022, 12:01

Merges dev-sri/pragma-scaffolding -> dev-next-gui

KNOWN BUG: some scripts may experience shifting of block indentation, but we should merge this anyway because the bug is already in the system

Every GEMSCRIPT 1.0 text source has some technical metadata in the form of "directives", which appear in lines with #. In many cases we want to hide technical requirements from students in the GUI, so this merge request is our quick-and-dirty approach to do that in time for the July deadline.

There are three directives that must be used in every GEMSCRIPT 1.0 text source. A specific directive is called a pragma

the blueprint pragma: # BLUEPRINT BlueprintName BaseBlueprintName
the tag pragma: # TAG TagName TagValue
that program pragma: # PROGRAM BundleOut

NEW FEATURES

1. Enforcing required TAG and PROGRAM directives

Currently we have three tags:

isCharControllable
isPozyxControllable
isPTrackControllable

The script scanner inserts missing tags with the default value false. Existing tags are left alone. The script scanner also inserts missing PROGRAM directives at the end.

2. Determining Program Contexts

The first two must appear at the top of a GEMSCRIPT text. The last, PROGRAM BundleOut, establishes the "execution context" of the following lines of code. There are certain keywords that must be executed in specific execution contexts:

addProp and useFeature must be in PROGRAM DEFINE
when must be in PROGRAM CONDITION
onEvent must be in PROGRAM EVENT
every must be in PROGRAM UPDATE or it will not get a chance to fire.
There are also runtime initialization programs that run just once before simulation starts for all blueprint instances (PROGRAM INIT), and the "every frame" code (PROGRAM UPDATE). Knowing which program to insert what keyword would get in the way of a good user experience. The GUI should hide this.

This merge request does not check/correct/flag the issues above. However, there is a new parallel script scanner that specifically scans for # PROGRAM BUNDLEOUT statements and creates a map that has the start and end of this section. This can be used by the GUI, through the new method GetProgramContextForLine(lineNum), which returns a TLineContext data structure { start:number, end:number, program:string } if the line is inside a program section. It's undefined otherwise.

TESTING

You can run Main and DevWizard to ensure that things have not broken; there currently is no use of these new routines in the GUI. However, DevWizard prints out some diagnostic information in the console.

build!
run DevWizard, and open the console
see the debug info. See if the script output in the console (regenerated using the new algorithm) matches what's in the script view
run ScriptEditor after running Main
The debug info will report that it couldn't get script_tokens or program_map, but that is probably because ScriptEditor isn't initialized at the time UR/APP_START runs in the phase machine

In GitLab by @daveseah on Jun 28, 2022, 14:45

marked the checklist item update TRANSPILER to check and auto-insert BLUEPRINT and TAG at the top of every script during TextToScript as completed

In GitLab by @daveseah on Jun 28, 2022, 14:45

marked the checklist item update TRANSPILER to auto-insert all the # PROGRAM BUNDLEOUT statements in the right order as completed

In GitLab by @daveseah on Jun 28, 2022, 14:48

marked this merge request as draft

Adding to the list of PROGRAM CONTEXT requirements:

every must be in PROGRAM UPDATE or it will not get a chance to fire.

In GitLab by @daveseah on Jun 29, 2022, 06:29

added

In GitLab by @daveseah on Jun 29, 2022, 16:29

added 6 commits

d6efa0a2 - sri/pragma-scaffolding: new TRANSPILER.EnforceBlueprintPragmas() called in...
3746291d - sri/pragma-scaffolding: rename pragma token to directive
09b5e999 - sri/pragma-scaffolding: bug fixes to various tokenizer methods
2285ab88 - sri/pragma-scaffolding: new support for getting context of a line.
cbdbbb1e - sri/pragma-scaffolding: fix bugs EnforceBlueprintPragmas(), so it should now...
03ad7923 - sri/pragma-scaffolding: remove explicit exports in favor of * exports

Compare with previous version

In GitLab by @daveseah on Jun 29, 2022, 19:23

added 4 commits

a6d18d92 - sri/pragma-scaffolding: Fix "skipped line after tags" bug in EnforceBlueprintPragma()
f5542a63 - sri/pragma-scaffolding: Fix bug in spreading ...args adding numeric props, changed to just args
5eaf700e - sri/pragma-scaffolding: Fix bug setting range of last seen PROGRAM directive at EOF
359144c0 - sri/pragma-scaffolding: regroup and change order of functions

Compare with previous version

This is probably not a problem, but I just want to make sure.

Go to Dev Wizard
Click on line 008 (somewhere in # PROGRAM DEFINE)
Click on + to add a new line
Add a new addProp line, e.g. addProp foo string 'bar'
Click Save
You'll get the following error:

screenshot_1053

This doesn't seem to be a problem in the ScriptEditor, so I'm guessing it's a Dev/Wizard issue.

In GitLab by @daveseah on Jun 29, 2022, 20:37

it might be an interaction with intercept state and how slot editor might affect it, causing a double invocation somewhere that is fatal… will look into it

In GitLab by @daveseah on Jun 30, 2022, 05:42

marked this merge request as ready

In GitLab by @daveseah on Jun 30, 2022, 06:32

it seems to be related to the script update enforce not seeing the last program definition and adding it to the wrong place. It happens any time you click save even without adding a line. On the save, an additional UPDATE section gets added and the original one gets shifted inside the last block. WEIRD.

In GitLab by @daveseah on Jun 30, 2022, 07:00

steps to reproduce:

1. Loaded GUI Wizard

With gui-tester.gemscript as source, a well-formed source file, on first run this is what the bottom looks like.

broken-insert-1

2. Clicked "Save"

Note that the # PROGRAM UPDATE section has MOVED INSIDE of the previous block, and a NEW PROGRAM UPDATE has been added to the bottom. No error has occurred yet.

broken-insert-2

3. Clicked "Save" Again

We see the error occur, as there are two PROGRAM UPDATES now.

broken-insert-3

In GitLab by @daveseah on Jun 30, 2022, 07:16

The tokens are redrawn on the first save, which suggests that there is a bug in the ScriptEditableTokens routines that is reassembling things. For a redraw to happen, that means script_tokens has been updated.

[x] confirmed that _interceptState is receiving new tokens
[x] confirmed there is a bug with the const lsos = TRANSPILER.ScriptPageToEditableTokens(script_page) and TRANSPILER.EditableTokensToScript(lsos) transformation pair inside of WIZCORE.SaveSlotLineScript()
[x] confirmed bug also exists when using const lsos TRANSPILER.ScriptToEditableTokens(script_tokens) as well.

Based on how the EditableToken routines work, shifts of block position is due to a misplaced or missing block marker. There is a condition when doing this kind of batch processing when the very last condition is never properly detected because 'end of file' is not considered as a change in state, so will look at that first and see if the markers are balanced.

In GitLab by @daveseah on Jun 30, 2022, 07:41

The current editable-lines.gemscript tester script doesn't show an error, so this is a new undetected bug.

If this section of gui-tester.gemscript is modified from:

...

# program event
onEvent Tick [[
  dbgTick
  ifExpr {{ agent.prop['carlBoolean'].value }} [[
    prop carlBoolean setTo false
    prop carlBoolean setTo false
  ]]
]]

...

to add an extra dbgTick:

# program event
onEvent Tick [[
  dbgTick
  ifExpr {{ agent.prop['carlBoolean'].value }} [[
    prop carlBoolean setTo false
    prop carlBoolean setTo false
  ]]
  dbgTick
]]

the problem goes away, so it is a bug somewhere in the (sigh) recursive repackager.

In GitLab by @daveseah on Jun 30, 2022, 14:00

ben says it's ok to merge!

In GitLab by @daveseah on Jun 30, 2022, 14:01

mentioned in commit b73acf9ef2ee0d855655b0f6fa3c87572cdcd2dc

theRAPTLab / gsgo