Feature: Programmatic Editable Token Support for Add/Modify/Delete Lines - [merged]

[benloh]

In GitLab by @daveseah on Jun 15, 2022, 08:52

Merges dev-sri/gui-debug-editable-toks -> dev-next-gui

This hopefully is the final fixed version of the system to allow easy modification, insertion, and deletion of Script Lines that are being edited by the GUI. This is more difficult than it sounds because our internal script encoding can have blocks of code, and these blocks can be insider of blocks. For example, this block of GEMSCRIPT text looks like 5 lines but is encoded as one statement:

every 10 [[
  if {{ energyLevel < 10 }} [[
     prop energyLevel setTo 50
  ]]
]]

encodes as "statement" (simplified for readability)

[ { every } { expr:"energyLevel < 10" } { block: [ [ { prop } { energyLevel } { setTo } { 50 } ] ] } ]

TESTING

BUILD & RUN

• npm run bootstrap; npm run gem
• http://localhost/app/dev-codetester
• open javascript console

NOTE

• the script that's being loaded is in editable-lines.gemscript
• all the debug output that is validating that scripts are working
• we are supporting all GEMSCRIPT block formats now, including the consecutive IF-ELSE style blocks

CHANGE

• in ac-wizcore-util, change the MUTATE constant on line 31 to TRUE
• Mutations are reported in the debug output in the console, and this time the scripts will NOT match
• The mutation code in ac-wizcore-util demonstrates how to use the API, and starts at line 138

DOUBLE-CHECK

• Does DevWizard GUI still run?
• Does MAIN still run the simulation projects without crashing?

TECHNICAL

The API for EditableTokens is generally this:

// create editable token array
const editableTokensArray = TRANSPILER.ScriptToEditableTokens(script_tokens);

const line_num = 10;
const index = CHECK.OffsetLineNum(line_num, 'sub'); // line nums start from 1, but arrays start from 0
const { lineScript } = editableTokensArray[index];
// lineScript contains script tokens you can edit or replace
// to add/remove lines, use Array.splice() on the editableTokensArray

// repackage editable token array back into script token
const new_script_tokens = TRANSPILER.EditableTokensToScript(editableTokensArray);

[benloh]

In GitLab by @daveseah on Jun 15, 2022, 20:26

added 2 commits

• 575d2be9 - dev-sri/gui-debug-editable-toks: add validation support for keywords _pragma, addProp, every
• ecaf1c1f - dev-sri/gui-debug-editable-toks: add instrumentation (WIP) before changing...

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 15, 2022, 20:35

After heavily instrumenting the line-out routines, my guesses are

• statements with blocks (which create multiline output) is incorrectly iterating through a rogue forEach that should actually be an indexer. You can see here that the LEVEL going in/out is adding extra lines, which means they're being iterated somehow.
• the use of pushStatement(), peekStatement(), and popStatement() which were added as part of the editable token stuff may not be needed at all because we're processing line-by-line
• possibly, we can't actually run the LSO routines until AFTER the script_page structure has been generated, because this is the unrolled script_tokens that don't require jumping-in and out of statement blocks.

[benloh]

In GitLab by @daveseah on Jun 16, 2022, 18:10

added 1 commit

• 1971dbd0 - dev-sri/gui-debug-editable-toks: (WIP) partially working solution, but...

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 16, 2022, 20:50

added 1 commit

• d9ea6947 - dev-sri/gui-debug-editable-toks: Comparison of two algorithms (WIP) for...

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 17, 2022, 04:26

added 1 commit

• e837aa12 - dev-sri/gui-debug-editable-toks: repacker works (mostly)

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 17, 2022, 05:02

added 1 commit

• 0afae3c8 - dev-sri/gui-debug-editable-toks: ERROR WIP this version breaks now with

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 17, 2022, 06:11

added 1 commit

• 34b9a12d - dev-sri/gui-debug-editable-toks: it's working again, was old code interference

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 17, 2022, 06:21

added 1 commit

• 97c989c5 - dev-sri/gui-debug-editable-toks: Cleanup

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 17, 2022, 06:38

added 2 commits

• 65c2b0dc - dev-scr/gui-debug-editable-toks: remove brute-force texts routine alternate
• 1f9bd86e - dev-sri/gui-debug-editable-toks: re-add MUTATION example

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 17, 2022, 07:42

added 1 commit

• f2c6ae1e - dev-sri/gui-debug-editable-toks: if-else subsequent block support!

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 17, 2022, 10:58

marked this merge request as ready

[benloh]

In GitLab by @daveseah on Jun 17, 2022, 14:09

added 1 commit

• 13df8687 - dev-sri/gui-debug-editable-toks: fix bugs in BUNDLE program key conversion...

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 17, 2022, 14:42

added 1 commit

• 4b134126 - dev-sri/gui-debug-editable-toks: class-sm-bundle was using lower case...

Compare with previous version

[benloh]

In GitLab by @daveseah on Jun 17, 2022, 14:43

CRASH ISSUES caused by the renaming of program outputs from lowercase to uppercase. Should all be fixed now.

[benloh]

No errors in dev wizard or main.

[benloh]

mentioned in commit ae8f778eb960d14c5e3776ab2c411b97fe55716a