Variable name with special character causes all stacks to be hidden and program will not work

Mr-Dave2 commented 6 years ago

Expected Behavior

Program stacks should be visible and the program should execute as it does on Scratch 2.0

Please describe what should happen

Variable names, even ones with Unicode characters, should have no effect on displaying program stacks or how the program executes.

Actual Behavior

Stacks and the variable are not visible and the program will not work.

Steps to Reproduce

I don't know where this character came from. I have tried other Unicode characters and I have not found another one that causes any problems. Credit to @shu279 for creating variable names that start with . I was playing "Cat's Step Adventure -Part1-" https://scratch.mit.edu/projects/194603038 by shu279 and wanted to try it on Scratch 3.0 but it would not work and none of the stacks could be seen.

I did a cut-n-paste to create a variable with the special-character name in a new Scratch 2.0 project (https://scratch.mit.edu/projects/217845144/) and then created a small stack that used that variable. When loaded into Scratch 3.0, the stack is not visible and the program will not work.

I also tried a cut-n-paste into vim and the character showed up as a control-H. Vim does not support Unicode so this probably does not mean anything.

Operating System and Browser

Windows 7 (64 bit), Chrome 65.0.3325.181

Mr-Dave2 commented 6 years ago

I found another program that will not run or display its stacks and it’s apparently also because of characters used in a name (of a list this time). (Actually, it does something; it will play a sound for about a second but there is no motion).

The program is https://scratch.mit.edu/projects/215321405/ and for Scratch 3.0 (https://llk.github.io/scratch-gui/develop/#215321405). I remixed the program and the only change I made was to delete the list named “>> Highscores <<”. After saving the project I could load it into Scratch 3.0 (https://llk.github.io/scratch-gui/develop/#218249134) and it would run with stacks now visible. (The sound still stops after one second but that is a different issue: “Deleting clones while playing notes causes music extension to stop working #1018”.

Mr-Dave2 commented 6 years ago

More people have reported this as the likely problem with their projects.

All projects that use the PTE projects to draw text (https://scratch.mit.edu/projects/172111618/). This user mentions the problem in this post https://scratch.mit.edu/discuss/topic/299791/?page=254#post-3190127 and another mentions it here: https://scratch.mit.edu/discuss/topic/299791/?page=254#post-3190125

I have three more test cases: This project where a list has a "&" in its name 2.0 & 3.0 This project where a variable starts with "<" 2.0 & 3.0 and this project where a broadcast has a "&" in its name: 2.0 & 3.0

project thumbnail

Windows 7, Google Chrome Version 68.0.3440.75 (Official Build) (64-bit) and Firefox 61.0.1.

Mr-Dave2 commented 5 years ago

Just wanted to add another user's program that appears to be affected by this. -ShadowOfTheFuture- reported in this post https://scratch.mit.edu/discuss/topic/308114/?page=14#post-3346362 that stacks in this program don't show up:

https://scratch.mit.edu/projects/269991846/ https://beta.scratch.mit.edu/#269991846

This time it looks like the problem list name is [G][P]"echo"

Mr-Dave2 commented 5 years ago

A user in the forums has found another instance of this problem. https://scratch.mit.edu/discuss/topic/308114/?page=15#post-3355760

This time the problem character is same one that I opened this issue with. I still don't know the name of this character. In this newest program it is the character that starts the message name (there is only one message used in this program.)

https://scratch.mit.edu/projects/271491705/

https://beta.scratch.mit.edu/#271491705

Windows 7 ; FF 64.0

picklesrus commented 5 years ago

Here's another project that runs into this because of the list with the '&' in the name of one of the list names: https://scratch.mit.edu/projects/167201750/

Attaching a simple sb2 project that reproduces this with a variable having a '&' in it. a&b.zip

Mr-Dave2 commented 5 years ago

I'm adding another example that was mentioned in the forums: https://scratch.mit.edu/discuss/topic/326529/ It looks like this project has a list with the stickied name "<>". I'm wondering it there should be a new sitckied topic after January 2 that explains to people what is going on if they can't see their scripts.

UPDATE:

Unfortunately this is no longer a good example because the user acted on my comment that the problem was with the "<>" list and they removed that list. The stacks are now visible.

KristianMischke commented 5 years ago

I have the same issue with one of my projects. I searched all over the scratch forums to see if anyone else had the issue until I found it here on github. I generally use "special" characters to identify that a custom block uses that variable as a return value.

Here is the project: https://scratch.mit.edu/projects/113795977/

abee2 commented 5 years ago

I fixed "Cat's Step Adventure -Part1-" and "Special character Scratch 3.0 issue". https://scratch.mit.edu/projects/277244459/ https://scratch.mit.edu/projects/277244998/ I just removed Backspace (\b) from JSON of those projects. I think it is the same problem of #4190. https://github.com/LLK/scratch-gui/issues/4190

picklesrus commented 5 years ago

After @kchadha's change to parser, there are still some things that we need to fix to solve this problem fully. They're outlined in this doc: https://docs.google.com/document/d/1W5DlZAM4QFuwdeSZHypPDIhd25l6j6kD90uzYBVRLOU/preview but the key info is copy/pasted here:

Questions:

What other characters (other than the backspace one) does this happen for?

Known manifestations of this bug

Control character in costume name stops toolbox blocks (after switch costume) from rendering
Control character in sound name stops toolbox blocks (after play sound) from rendering
Executing say or thing block with control character breaks the whole renderer
Variable blocks with names with control characters don’t render
Broadcast message blocks with control characters don’t render
Text element in paint editor will break the renderer

Here are a couple approaches for fixing

Escape/encode/remove them on the way in and out of the vm. This is tricky because we end up in a state where the value in the blocks and the vm are different and we have to be very careful to keep them in sync. It is easy to get into a state where everything appears to be working (e.g. the blocks render) but the value reported by the blocks is different from the vm (e.g. the set costume to block) and nothing happens when the block executes.
Remove the characters in every spot the user can type in information. Here are some places we’d have to handle: -- Sprite name -- Sound name -- Costume name -- Text input on blocks -- Variable/List/Procedure names -- Text element in paint editor (different repo, duplication of effort between gui and paint) -- List monitor copy/paste

kchadha commented 5 years ago

Removing the critical label from this issue as the blocks should no longer be disappearing on these projects and we have a workaround for some of the other symptoms of this issue.

cc/ @picklesrus @thisandagain

kchadha commented 5 years ago

Adding to the "Known Manifestations of this Bug" section above:

Control character custom block name (or added block labels) causes erroneous empty red hat block to appear on the workspace in place of custom procedure definition block:

control-char-in-custom-block

kchadha commented 5 years ago

Another helpful note for debugging/testing, the backspace control character can be copy pasted from this webpage:

http://www.unicode-symbol.com/u/0008.html

There is a small text field under the description of the character for easy copy-pasting:

scratchfoundation / scratch-vm