Merge 1.2.0 functionality

[bartbutenaers]

The functionality from the 1.2.0 branch should be reviewed and manually integrated into the 1.1.0 branch. Due to the pull-request from @jsccjj, most of the changes probably won't work correctly anymore. No offense of course ;-)

I think these are the most important ones:

1. The current scale/scrollX/scrollY of the blockly workspace is stored, so the next time you open the workspace we can start with the same scale and location.
2. A trashcan icon is displayed in the blockly workspace
3. A new block for special text characters (LF, CR, CRLF, TAB) --> TODO: check if already in the master branch?
4. Allow nested fields to be refrerenced (e.g. msg.payload.data.temperature) --> TODO: check if already available?

[bartbutenaers]

@cymplecy , @jsccjj, About features 1 and 2. I was thinking about adding a 4th tabsheet "Settings" where I can add options like:

• Checkbox "show trash can in Blockly workspace"
• Checkbox "remember workspace position and scale"
• ...

Because some users want it like this, and others like that. Now we can let them choose to customize some settings. Is that ok to add?

[cymplecy]

Stick it in - stick everything in and we can then look and see what its like

Can I suggest creating a new dev branch for all dev changes from now on?

Simon

On Thu, 27 May 2021 at 22:23, bartbutenaers @.***> wrote:

@cymplecy https://github.com/cymplecy , @jsccjj https://github.com/jsccjj, About features 1 and 2. I was thinking about adding a 4th tabsheet "Settings" where I can add options like:

• Checkbox "show trash can in Blockly workspace"
• Checkbox "remember workspace position and scale"
• ...

Because some users want it like this, and others like that. Now we can let them choose to customize some settings. Is that ok to add?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bartbutenaers/node-red-contrib-blockly/issues/63#issuecomment-849951583, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR7RND4ZFTDNPMJPMFNQR3TP2Z5NANCNFSM45VA3HGA .

[jsccjj]

@cymplecy , @jsccjj, About features 1 and 2. I was thinking about adding a 4th tabsheet "Settings" where I can add options like:

• Checkbox "show trash can in Blockly workspace"
• Checkbox "remember workspace position and scale"
• ...

Because some users want it like this, and others like that. Now we can let them choose to customize some settings. Is that ok to add?

Yes. I think it is ok to add. Just that we have to be clear that if we want each Blockly node to have its own trashcan & settings & toolbox & etc... or we put some into a configuration node and some in the node itself.

[bartbutenaers]

@jsccjj , I like your idea of the config node. Since users might have numerous Blockly nodes in their flow, they most probably want to have the same settings all over the place. Via a config node they indeed can adjust their settings once. And if they want separate settings for a particular Blockly node, they only have to create one extra config node. I vote for this!

@cymplecy , The dev branch is indeed a good idea for the future. But if you don't mind, I'm now going to continue in the current branch, because I still get headaches from Git commands in the evenings...

[bartbutenaers]

@cymplecy : is the config node idea from @jsccjj ok for you? It makes sense to me to do it that way. I will continue implementing it now, but I'm rather strike at the moment from my lets-do-the-daily-job-also-on-sunday event ...

[bartbutenaers]

@cymplecy, @jsccjj, Here is a quick demo of how it looks like. I have created a basic config with currently only 2 parameters to test (show trashbin icon + select a renderer). In the demo I have created two configs, one with a trashbin and one without. Both configs also have a different renderer. The workspace is immediately updated when you select another config:

[bartbutenaers]

@cymplecy : since you are the Blockly judge, can you please let me know which settings you find useful?

[cymplecy]

Re Trash can

I always just drag blocks onto categories to delete them - that works in all block programming languages such as Scratch and Snap! etc so to me - it's a redundant feature - I would never enable it but if we are going to have configs - then may as well be an option

What I REALLY like is to have a different rendering so if configs are needed to achieve this - then lets have them :) (I have hated Blocky default rendering since the day it came out!)

I don't think visually, each node needs to have separate config but I don't know enough about NR nodes as to wheter that is possible, I think if a user played around with visual settings - they'd wouldn't want to have to go thru each node and change them all

[cymplecy]

Re default settings https://developers.google.com/blockly/guides/configure/web/configuration_struct#the_options_dictionary

The current defaults are fine for me so I wouldn't bother implenting any of them apart from trashcan - maybe comments? but since we have a comments block - not really needed

[bartbutenaers]

I don't think visually, each node needs to have separate config

@cymplecy : you can choose whatever you want:

• no config node: the blockly default settings are used (like in the old node version)
• a single config node used in multiple blocly nodes. You create e.g. once a single config node, and you select that in all your blockly nodes. So you have the sale settings in all your blockly nodes.
• you could also create multiple config nodes, e.g. 1 per blockly node. Although I don't see a use case now, it is posdible anyway...

[cymplecy]

"a single config node used in multiple blocly nodes. You create e.g. once a single config node, and you select that in all your blockly nodes. So you have the sale settings in all your blockly nodes."

👍

[bartbutenaers]

@cymplecy, I have pushed the blockly config changes to the 1.1.0 branch, so you can have a look at it. Currently the config node screen looks like this:

BTW the settings are not correctly applied yet in the full-screen mode. Todo ...

[cymplecy]

Can't see any way of accessing a config and main Blocky has broken for me :(

couple of errors showing as well

Going to delete instance and recreate from start

[edit] Exactly the same including errors after complete wipe and starting again

[bartbutenaers]

@cymplecy, Sorry my fault! Typo in package.json. Can you please try again

[cymplecy]

Config option has appeared but main category still broken - still getting those console errors

[edit - just realised that Blocky call the categories the toolbox so I'll use that word in future

[cymplecy]

But at least :)

:)

[cymplecy]

There isn't a basic folder in this install, compared to normal install, which seems to be giving the problems

[bartbutenaers]

When I install it from our 1.1.0 branch it works fine ... Will have a look tomorrow evening, because my time is up for today... Can you provide me some extra information to figure out what is going wrong in your setup?

• The basic folder has been removed yesterday from our repository, since that contained a local clone of the blockly library. Now it is downloaded as an npm dependency. Do you have (under node_modules) a "blockly" directory? I expect this is correct, otherwise you should see also a "The blockly npm library path " + blocklyModulePath + " does not exist" error ...
• Do you have tested already before this node that runned with an npm dependency of blockly, or is this the first test since then? Because I don't see why this would be caused by my changes of today ...
• Which errors do you see when you change the language of the blockly node?
• Others?

I get the impression that the language dropdown change handler code has not been converted correctly to work with npm dependencies. I mean that incorrect (old constructs) are perhaps used. Need to have a look tomorrow...

See you tomorrow!

[cymplecy]

JFI - as I said earlier - I deleted the NR instance and recreated it from scratch - I'll delete it again tomorrow and post the logs for you :)

[cymplecy]

Complete new NR instance No errors on install

C:\Users\Simon\btest>npm install https://github.com/bartbutenaers/node-red-contrib-blockly#release-1.1.0
npm WARN deprecated request-promise-native@1.0.9: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
npm WARN node-red-master@1.0.0 No repository field.

+ node-red-contrib-blockly@1.1.0
added 50 packages from 83 contributors and audited 401 packages in 19.316s

6 packages are looking for funding
  run `npm fund` for details

found 4 moderate severity vulnerabilities
  run `npm audit fix` to fix them, or `npm audit` for details

Creates a blocky folder

All looks OK on 1st run

===================

31 May 23:13:25 - [info] Node-RED version: v1.3.5
31 May 23:13:25 - [info] Node.js  version: v10.16.3
31 May 23:13:25 - [info] Windows_NT 10.0.19041 x64 LE
31 May 23:13:27 - [info] Loading palette nodes
31 May 23:13:32 - [info] Settings file  : C:\Users\Simon\btest\data\settings.js
31 May 23:13:32 - [info] Context store  : 'default' [module=memory]
31 May 23:13:32 - [info] User directory : C:\Users\Simon\btest\data
31 May 23:13:32 - [warn] Projects disabled : editorTheme.projects.enabled=false
31 May 23:13:32 - [info] Flows file     : C:\Users\Simon\btest\data\flows_Win8.json
31 May 23:13:32 - [info] Creating new flow file
31 May 23:13:32 - [warn]

---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.

If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.

You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------

31 May 23:13:32 - [info] Server now running at http://127.0.0.1:1881/
31 May 23:13:33 - [info] Starting flows
31 May 23:13:33 - [info] Started flows

Simply adding inject - blocky - debug without doing anything else - no errors logged

add simple blocks and deploy - still no errors (but no basic toolbox showing either)

Able to change theme and deploy - still no errors in console

[bartbutenaers]

Morning Simon, When I start incognito, then I get the same problem. I think the old files stay cached because I have added (at the time being) a statement in our endpoint, which I need to remove. But at least I can reproduce the problem this evening...

When I removed the Blockly library from our repository, I think I have been a bit too enthousiastic. I think that I need to restore our basic/toolbox.xml. It is too long ago, but I think I have generated that toolbox myself 3 years ago. But not sure. Anyway it is not in the blockly npm directory, so we have to add it to our repository again.

This makes sense, because you need to specify which blocks you want to offer to the users and which not. On the other hand, all changes from the last 3 year won't be available in our node (e.g. new blocks from Google, ...). If you should have time to investigate a bit of background info about this, that would be nice.

I have to leave now for the physiotherapist and then to the daily job ...

[cymplecy]

"If you should have time to investigate a bit of background info about this, that would be nice." way beyond my capabilities I'm afraid :(

[jsccjj]

Hi Guys,

I went into looking for a Blockly "Default Toolbox" and found this which was created in 2020: https://github.com/google/blockly-samples/blob/master/plugins/dev-tools/src/toolboxCategories.js

In the JS file, it is literally a XML file which we can use as long as we removed some JS syntaxes.

The toolbox is only fractionally different from @bartbutenaers's. Only a few new nodes added such as

I haven't diligently looked into the JS file yet but I think it might be a good start.

[jsccjj]

Just confirmed with Google Blockly group that the JS file contains all the official default blocks. So, we might want to spend a little time to compare the differences. Shall we?

[bartbutenaers]

@jsccjj , @cymplecy,

I "think" I have done it like below 3 years ago.
Will explain it step by step, because I could not quickly find a tutorial for noobs:

1. Go to the Blockly developer tools.

2. Go to the tabsheet "Workspace Factory":

   

3. Start with a standard toolbox (i.e. all standard blocks), to avoid having to add all the standard Blockly categories one by one manually:

   

   P.S. This way all the standard blocks will also be available automatically in those categories.

4. As a result all categories will become available in "Your categories":

   

5. Use the buttons (+, -, arrow up, arrow down) to edit the list of category names.

6. When you click one of your categories, the corresponding blocks from that category will be displayed on the left panel:

   

   I find this a bit confusing, but it works like this: the left toolbox in my movie is everything what standard Blockly offers. On the right are your own categories. And the white workspace in the middle are the blocks in your currently selected category ...

7. You can remove blocks from your category, by dragging them into the trashbin. Or you can add blocks to your category, by dragging them from the standard categories (from the left):

   

8. You can see a preview of your toolbox on the right side.

9. Once you are done, export the toolbox to a toolbox.xml file:

   

So if anybody wants to compare this, be my guest!!!

[cymplecy]

That looks awful :(

On Tue, 1 Jun 2021 at 21:59, bartbutenaers @.***> wrote:

@jsccjj https://github.com/jsccjj , @cymplecy https://github.com/cymplecy,

I "think" I have done it like below 3 years ago. Will explain it step by step, because I could not quickly found a tutorial:

1.

Go to the Blockly developer tools https://blockly-demo.appspot.com/static/demos/blockfactory/index.html . 2.

Go to the tabsheet "Workspace Factory":

[image: image] https://user-images.githubusercontent.com/14224149/120385061-3ed1a680-c327-11eb-8355-ec0a129fc75d.png 3.

Start with a standard toolbox (i.e. all standard blocks), to avoid having to add all the standard Blockly categories one by one manually:

[image: image] https://user-images.githubusercontent.com/14224149/120385757-1dbd8580-c328-11eb-827f-72ac49ad9080.png

P.S. This way all the standard blocks will also be available automatically in those categories. 4.

As a result all categories will become available in "Your categories ":

[image: image] https://user-images.githubusercontent.com/14224149/120386042-6a08c580-c328-11eb-8f39-fba788241b5e.png 5.

Use the buttons (+, -, arrow up, arrow down) to edit the list of category names. 6.

When you click one of your categories, the corresponding blocks from that category will be displayed on the left panel:

[image: blockly_tools_blocks_category] https://user-images.githubusercontent.com/14224149/120388264-2f545c80-c32b-11eb-88cd-e97a6a9176a4.gif

I find this a bit confusing, but it works like this: the left toolbox in my movie is everything what standard Blockly offers. On the right are your own categories. And the white workspace in the middle are the blocks in your currently selected category ... 7.

You can remove blocks from your category, by dragging them into the trashbin. Or you can add blocks to your category, by dragging them from the standard categories (from the left):

[image: blockly_tools_blocks_category_edit] https://user-images.githubusercontent.com/14224149/120388658-b43f7600-c32b-11eb-97d6-7f241de64995.gif 8.

You can see a preview of your toolbox on the right side. 9.

Once you are done, export the toolbox to a toolbox.xml file:

[image: image] https://user-images.githubusercontent.com/14224149/120389029-2e6ffa80-c32c-11eb-9faf-44d755a66dd8.png

So if anybody wants to compare this, be my guest!!!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bartbutenaers/node-red-contrib-blockly/issues/63#issuecomment-852441283, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR7RNB3HGP4T65JCETQLWLTQVC4ZANCNFSM45VA3HGA .

[bartbutenaers]

That looks awful :(

I hope you don't mean my tutorial... If you mean the Blockly developer tools, then I must agree it is FAR from self explaining. But at least you can compose at toolbox, by dragging and dropping. So if you would have time and energy to create a new toolbox and compare it with our old one, then we can add in our release notes which new blocks have arrived ... Otherwise I will have a look tomorrow evening...

[cymplecy]

Sorry - I thought that this was the new way your Blocky node was goign to operate :)

I have no clue what your demo does or how to help

But I'll be quick to let you know if you get it wrong ..... :)

[jsccjj]

Great tutorial! I think this will be used in the future that once we decide to allow users to define their own blocks/categories

BTW, I will look into the differences since I have two NR instances running one with the latest and one with the old settings

@jsccjj , @cymplecy,

I "think" I have done it like below 3 years ago.
Will explain it step by step, because I could not quickly find a tutorial for noobs:

1. Go to the Blockly developer tools.

2. Go to the tabsheet "Workspace Factory":

   

3. Start with a standard toolbox (i.e. all standard blocks), to avoid having to add all the standard Blockly categories one by one manually:

   

   P.S. This way all the standard blocks will also be available automatically in those categories.

4. As a result all categories will become available in "Your categories":

   

5. Use the buttons (+, -, arrow up, arrow down) to edit the list of category names.

6. When you click one of your categories, the corresponding blocks from that category will be displayed on the left panel:

   

   I find this a bit confusing, but it works like this: the left toolbox in my movie is everything what standard Blockly offers. On the right are your own categories. And the white workspace in the middle are the blocks in your currently selected category ...

7. You can remove blocks from your category, by dragging them into the trashbin. Or you can add blocks to your category, by dragging them from the standard categories (from the left):

   

8. You can see a preview of your toolbox on the right side.

9. Once you are done, export the toolbox to a toolbox.xml file:

   

So if anybody wants to compare this, be my guest!!!

[bartbutenaers]

Just confirmed with Google Blockly group that the JS file contains all the official default blocks. So, we might want to spend a little time to compare the differences. Shall we?

@jsccjj: Then I assume it will contain the same blocks as in the Blockly Developer Tools? Perhaps we can use the Developer tools from now to make sure we don't forget any new blocks.

Sorry - I thought that this was the new way your Blocky node was goign to operate :)

Oh no,

I have no clue what your demo does or how to help

The toolbox in our workspace is based on a toolbox.xml file. By using the steps I have described above, we can create a new toolbox.xml file for the standard Blockly categories and corresponding blocks. I.e. the blocks that you currently don't see anymore. Our Blockly node contains some other toolbox.xml files, one for each of our own categories (e.g. a toolbox.xml file for our Node-RED blocks). So we need to use the blockly developer tools to compose a new toolbox.xml file, i.e. specify which standard Blockly categories and blocks we want to show in our toolbox of the Blockly node.

@jsccjj : my time is almost up for this evening. If you have time to compare the new toolbox and our old one, then @cymplecy can determine which changes we need or not. Of course you can give him hints for his decisions ;-). And otherwise I will have to do the comparision tomorrow evening...

[bartbutenaers]

Great tutorial! I think this will be used in the future that once we decide to allow users to define their own blocks/categories

That could indeed be an option in the future. Not thought about it yet ...

BTW, I will look into the differences since I have two NR instances running one with the latest and one with the old settings

Thanks a lot for the offer !!!!!!!!!!!!!!!!!

[jsccjj]

Just confirmed with Google Blockly group that the JS file contains all the official default blocks. So, we might want to spend a little time to compare the differences. Shall we?

@jsccjj: Then I assume it will contain the same blocks as in the Blockly Developer Tools? Perhaps we can use the Developer tools from now to make sure we don't forget any new blocks.

The reply I received:

I would assume Beka is correct but we should always double check...haha

[jsccjj]

Looked into the differences. Checked the source codes from Blockly repo. There are only four new blocks:

The generated JS code:

Tooltips:

I think it's fine to include all four blocks. My two pennies here.

[bartbutenaers]

Hi Jeff, I appreciate your help!!! For me it is fine to add those 4 new blocks. But you know this repo is hosted by a good cop and a bad cop, so we also need feedback from the latter one ... @cymplecy is this ok for you? I will meanwhile try to generate a new toolbox.xml file using the Blocky Developer tools ...

[cymplecy]

👍

[bartbutenaers]

@jsccjj, This is so weird ... I have generated a new toolbox.xml file with the Blockly Developer tools, and added it again to our repository. Then indeed the standard Blockly categories are available again in our blockly node. That is good news ...

But I cannot find those 4 new blocks that you are talking about. And indeed when I have a look in the Blockly Developer tools, then indeed they seem not to be available there either:

I would expect that the Blockly developer tools always contain all available blocks... But seems that is not the case, so I will have to use your file ... When I use that file in the blockly node, indeed I get those 4 new blocks:

Not sure where we need to get the "ifelse" block that Beka is talking about. Anyway we also didn't have that block in our old blockly node (see old toolbox.xml), so we cannot break old flows by not adding it...

P.S. I'm not going to push my changes to Github yet, because there seems now - for some reason - a problem with the Node-RED blocks:

Need to solve that first, because now the workspace becomes corrupted when opened ;-(

[jsccjj]

@jsccjj, This is so weird ... I have generated a new toolbox.xml file with the Blockly Developer tools, and added it again to our repository. Then indeed the standard Blockly categories are available again in our blockly node. That is good news ...

But I cannot find those 4 new blocks that you are talking about. And indeed when I have a look in the Blockly Developer tools, then indeed they seem not to be available there either:

The four blocks reside in the latest source code. For example https://github.com/google/blockly/blob/master/blocks/text.js

I guess the develop tool is not yet updated... just a guess.

I would expect that the Blockly developer tools always contain all available blocks... But seems that is not the case, so I will have to use your file ... When I use that file in the blockly node, indeed I get those 4 new blocks:

Not sure where we need to get the "ifelse" block that Beka is talking about. Anyway we also didn't have that block in our old blockly node (see old toolbox.xml), so we cannot break old flows by not adding it...

The "ifelse" is not a must but rather an alternative that does not use a mutator.

P.S. I'm not going to push my changes to Github yet, because there seems now - for some reason - a problem with the Node-RED blocks:

Need to solve that first, because now the workspace becomes corrupted when opened ;-(

Oh nooo.... I will try look into if my workspace would corrupt tonight.

[bartbutenaers]

Oh nooo.... I will try look into if my workspace would corrupt tonight.

@jsccjj FYI I found last night that the (existing) node_object_set block for Node-RED seem to fail, due to changes standard Blockly blocks :

I have the impression that the Blockly team has made some changes that cause my block to fail. In the above code I try to get the "field_name" value which was available in the old toolbox.xml file at line 164:

And that field doesn't exist anymore in the file that you have found:

Do you have any idea? Perhaps you can ask Beka whether she knows about such changes? Because there might be other changes that cause our custom blocks to fail ...

[bartbutenaers]

Evening of debugging, but no result. Only wasting time ...

The code snippet is called several times,(via the createWorkspace function) and it always goes fine. Except when it is called via the language dropdown change handler, then it fails:

In that case the this.inputList contains an empty array. So I need to figure out what is different when the createWorkspace is called in that handler:

Time is up for today...

[jsccjj]

Hi Bart,

I could not replicate the issues on my setup..... Can you share with me your Node Red flow which created the issues?

My trying: The standing alone NR set-node:

Generated JS

Generated XML

From console, We can see "Blockly.Blocks['node_object_set'].onchange" exists.... It seems normal to me. So I want to replicate your situation first.

[bartbutenaers]

I could not replicate the issues on my setup..... Can you share with me your Node Red flow which created the issues?

@jsccjj, I have pushed my latest version to Github. This version contains the toolbox.xml file and some fixes to load the files from the npm dependency. Now the standard Blockly blocks should be available again in our node.

How to reproduce the issue:

1. Add a Node-RED set block to the workspace:

   

2. Press the "Done" button

3. Open the config screen again. Now you will see the error appearing in the console.

4. The inputs will be gone:

   

[cymplecy]

Still no standard categories on a clean install

[bartbutenaers]

@cymplecy, I have installed it on a complete new system from scratch, and it works fine ... Are there any errors in your browser console log, or in the Node-RED log? Does it work in incognito mode? Because in the last version I have also removed some code that prevented new versions of the library to be loaded...

[cymplecy]

Sorry - forgot to do a cache clear and hard reload :(

[cymplecy]

Also seeing that issue

[bartbutenaers]

@cymplecy, Thanks for the feedback! So glad that we finally have Blockly as a simple NPM dependency working, to simplify updates in the future... About the Node-RED blocks issue. I have run out of creativity, so hopefully @jsccjj can find something... Otherwise I will need to start digging again, or ask a question on the Blockly forum...

[cymplecy]

Expanded editor not working for me either

[bartbutenaers]

Expanded editor not working for me either

I did a quick test and it works fine here: fullscreen mode works, I see the standard blocks there.. Of course the issue withe the Node-RED blocks also occurs in fullscreen mode.

You will need to give a bit more context information... What does not work? Do you see problems in browser console or Node-RED log?

[cymplecy]

I'm going to do another clean install and hard reload again

[cymplecy]

Expanded editor worked on clean install

It must have been the other error having an effect

I will check any more errors using a clean install before reporting!