Byte block doesn't behave in the manner I think it should

bartbutenaers / node-red-contrib-blockly

A Node Red node for visual programming a function using Blockly

Apache License 2.0

91 stars 22 forks source link

Byte block doesn't behave in the manner I think it should #62

Closed cymplecy closed 3 years ago

cymplecy commented 3 years ago

Since we are restarting :)

Back in the mists of time, I raised the point that I didn't think the way byte blocks are interpreted was correct.

We had lots of "discussion" and I never manged to convince you - maybe I did but who can remember after so long :) but this is BIG issue for me so I'm re-raising it again :)

In summary, I believe a byte block should return a single byte (or char) e.g byte(10) should give a single buffer with a value of 10 (e.g. a newline char) not the decimal number 10 as it does at the moment

This should not produce 10

it should give

cymplecy commented 3 years ago

I am VERY confused where we are with this - I'm re-opening but lets wait till all the other stuff is finished and then we can concentrate on this again :)

I've re-read all the stuff we talked about back in 2019. I got myself up to speed back then on JS buffer's but I'd need to wind my brain back into advanced JS/Blockly mode to deal with again

I only want go go round on this subject one last time :)

cymplecy commented 3 years ago

@bartbutenaers Here is the bit of our messages that refers to this https://discourse.nodered.org/t/getting-blockly-1-2-0-out-of-the-door/14294/6?u=cymplecy

bartbutenaers commented 3 years ago

@cymplecy,

I have tried to summarize our discussions from the past, which started to look like the Hundred Years' War ;-) Please correct me if I have interpreted anything incorrectly! Then we can start from scratch here, and hopefully @jsccjj can join the discussion for technical insights...

Issue 1 - A byte block should return a single byte (or char)

For example byte(10) should return a single buffer with a value of 10 (e.g. a newline char) not the decimal number 10 as it does at the moment.

That makes sense to me, but:

That might break existing flows, so we need to inform users about this breaking change.
Currently the Byte-block output is type Number. When this output type is changed, we will have to adapt the other blocks to, to accept the two new types.
It is not clear yet to me which output type you expect, because type Char doesn't exist here?

Issue 2 - Allow other types of inputs for buffer_set_index block

At this moment the blocks in this branch allow this kind of input:

fill_buffer allows String, Buffer and Number"
buffer_set_index accepts only Number

You wanted to allow extra input types on the buffer_set_index block (to have both blocks accepting the same input):

Although I understand your intentions, it feels a bit like cheating to me... Because you had to propose to throw away a (major) part of the input data:

Strings: You wanted to only keep the first char of the string, and ignore the remaining part of the string (since you only need 1 char). My proposal was to write the entire string into the buffer, starting from the specified index (and only ignore the part of the string that doesn't fit into the buffer length): by generating a buffer.write(someString); statement if length of the string > 1.
Numbers: You wanted to limit the input from 0-255. My idea was again to write all the specified numbers into the buffer, starting from the specified index...

Some remarks:

You wanted to implement the same input preprocessing (e.g. only keep the first char of a string) also in the fill_buffer node. Not sure if that is what we want.
You also suggested to remove the Byte-block from the toolbar, as soon as this change was implemented. I'm not sure anymore why that was suggested?
You also wanted to allow variables as input.

Although that would offer a lot of flexibility, it is not clear to me how we can check which data is in the variable. Otherwise people can store any kind of data in a byte, even an entire jpeg image ... And we only know at runtime which data is stored in a variable, not at code-generation time...

cymplecy commented 3 years ago

Before we get into all the options over filling buffers and stuff like that I'd just want to get agreement on the fundemental issue

e.g byte 65 should NOT return the number 65, it should return a single buffer object with a value of 65.

Now, we can either correct this or we can drop the byte block altogether.

I think it will be much easier to drop the block as it's not actually needed to produce buffer objects by the rest of the blocks. We can just use number and text blocks.

I think we should try this approach and see where it leads.

If it proves to be wrong, then roll it back and proceed down the path of making it return a single buffer object, length 1

So in direct response to:

Yes - it will be a breaking change - but since NR is going to 2.x.x, now is the perfect time for this
My suggestion is remove it
If we do keep it, it should be buffer object, length 1

bartbutenaers commented 3 years ago

Ok, suppose we remove the Byte-block from the toolbar. What happens then with the set-byte-at-index block? It will need to accept other values and get another shadow block. And the we arrive again at the question if we need to truncate the Number/String? Or am I mistaken?

cymplecy commented 3 years ago

"Ok, suppose we remove the Byte-block from the toolbar." OK :)

"What happens then with the set-byte-at-index block? It will need to accept other values and get another shadow block." Yes - it will need to accept numbers, text and variables

"And the we arrive again at the question if we need to truncate the Number/String" My opinion (personal) is that it should just set the byte at the wanted position with either the number or the 1st char of the string.

The existing blocks can convert a long text string to a buffer. I would add an explicit block to concatenate two buffers.

I think we also need a length of buffer block

bartbutenaers commented 3 years ago

Ok, you win. Had too much discussions already lately :-( Will see what I can do...

jsccjj commented 3 years ago

Hi guys,

I wanted to join the discussion but found that this is a lengthy discussion.... Many historical considerations are there... I think I can only provide my two pennies here ( please excuse me if there is any misunderstanding):

I think we can modify the byte block to return a single buffer object. Then, we modify the set-byte-at-index block to convert the input (now, it is a buffer object, length 1) back to value (number, 0-255). So, the existing users won't be affected while the byte block can return a single buffer object.... I hope this is feasible

bartbutenaers commented 3 years ago

I think we also need a length of buffer block

Doesn't that exist already?
See:

I would add an explicit block to concatenate two buffers.

I have added a new block:

Here is an example flow to concatenate buffer1 (containing "Hello ") and buffer2 (containing " world"):

[{"id":"935786d92e05853b","type":"Blockly","z":"c2a7925b.6e143","func":"var buffer1, buffer2;\n\n\nbuffer1 = (Buffer.from('Hello ', \"utf8\"));\nbuffer2 = (Buffer.from('world', \"utf8\"));\nmsg['payload'] = (Buffer.concat([buffer1, buffer2]));\nnode.send([msg]);\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"k;r1=:Al*Fb:A+@;B^]D\">buffer1</variable>\n    <variable id=\"mtb,+}W$:ly1N9jU*k#v\">buffer2</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"#:g7*k:IR]si7j[1]T*M\" x=\"-312\" y=\"-137\">\n    <field name=\"VAR\" id=\"k;r1=:Al*Fb:A+@;B^]D\">buffer1</field>\n    <value name=\"VALUE\">\n      <block type=\"buffer_from_string\" id=\"KN=V^94,q2MiQMLEyqBL\">\n        <field name=\"ENCODING\">utf8</field>\n        <value name=\"STRING_INPUT\">\n          <shadow type=\"text\" id=\"aqb3oX068$Cxr2[wG_-T\">\n            <field name=\"TEXT\">Hello </field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"variables_set\" id=\"]_@5:.A^k`?=V6Z{FY90\">\n        <field name=\"VAR\" id=\"mtb,+}W$:ly1N9jU*k#v\">buffer2</field>\n        <value name=\"VALUE\">\n          <block type=\"buffer_from_string\" id=\"}4o_p-|Yu?nx6sV%zgD_\">\n            <field name=\"ENCODING\">utf8</field>\n            <value name=\"STRING_INPUT\">\n              <shadow type=\"text\" id=\"0*5-G:4L~d;xU$=5Dcca\">\n                <field name=\"TEXT\">world</field>\n              </shadow>\n            </value>\n          </block>\n        </value>\n        <next>\n          <block type=\"node_object_set\" id=\")$Yqthfe9^@=f2!AL!av\">\n            <value name=\"object_field\">\n              <shadow type=\"node_msg\" id=\"]gl2-i%AxlP[,^32MPj(\"></shadow>\n            </value>\n            <value name=\"field_name\">\n              <shadow type=\"text\" id=\"f3ySgBn}o+v5p|@]0jOB\">\n                <field name=\"TEXT\">payload</field>\n              </shadow>\n            </value>\n            <value name=\"value_field\">\n              <shadow type=\"text\" id=\"/(;1bC54JyJza87k46:3\">\n                <field name=\"TEXT\"></field>\n              </shadow>\n              <block type=\"buffer_concatenate\" id=\"6h`{e?{}Vq$k|x~ZmEs}\">\n                <value name=\"BUFFER_FIRST\">\n                  <block type=\"variables_get\" id=\"*j0g6t{0f10:fzsm7wi.\">\n                    <field name=\"VAR\" id=\"k;r1=:Al*Fb:A+@;B^]D\">buffer1</field>\n                  </block>\n                </value>\n                <value name=\"BUFFER_SECOND\">\n                  <block type=\"variables_get\" id=\"f}d{cL,3P2h~F`-Q{cTk\">\n                    <field name=\"VAR\" id=\"mtb,+}W$:ly1N9jU*k#v\">buffer2</field>\n                  </block>\n                </value>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_send\" id=\"tweuq)?Rp7^)T]bVi.YU\">\n                <field name=\"OUTPUT_NR\">1</field>\n                <value name=\"MESSAGE_INPUT\">\n                  <shadow type=\"node_msg\" id=\"G7rwZXpq8EQ@JI_i=pq#\"></shadow>\n                </value>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":1140,"y":360,"wires":[["7011a862f6c538be"]]},{"id":"9512866d85dd94d0","type":"inject","z":"c2a7925b.6e143","name":"Start concatenation","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":950,"y":360,"wires":[["935786d92e05853b"]]},{"id":"7011a862f6c538be","type":"debug","z":"c2a7925b.6e143","name":"Concatenated buffer","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1350,"y":360,"wires":[]},{"id":"d7a036fa.2c0298","type":"blockly-config","language":"en","showTrashcan":true,"allowComments":true,"showZoomControl":true,"enableBackPack":"node","backpackContents":[],"toolboxPosition":"left","renderer":"geras","categories":[{"name":"Node-RED","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Objects","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Buffer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Date/time","files":["blockly-contrib/npm/@blockly%2Ffield-date/dist/date_compressed.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Timer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly extension","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly standard","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml"]},{"name":"Custom category","files":["blockly-contrib/file/c:/temp/myblocks/myBlocksCodeGen.js","blockly-contrib/file/c:/temp/myblocks/myBlocksDefs.js","blockly-contrib/file/c:/temp/myblocks/myToolbox.xml","blockly-contrib/file/c:/temp/myblocks/my_messages/en.js"]}],"customizeToolbox":true,"name":"Left"}]

Which returns this (which is best readable via the "raw" button):

cymplecy commented 3 years ago

"Doesn't that exist already?" Sorry- got confused :)

Concat works for me as well :)

bartbutenaers commented 3 years ago

I think we are done developing for 2.0.0-beta1 if we have fixed the byte block, to return a single byte buffer. To accomplish that, I have asked a new question on the blockly forum: https://groups.google.com/g/blockly/c/9fEPSGFarNM As soon as I have an answer, I will continue with it...

bartbutenaers commented 3 years ago

Hi guys (@cymplecy, @jsccjj), I have pushed my (hopefully) last commit for the release... Content: the byte block returns a buffer of length 1, and the set-byte-at-index block accepts a number/string/buffer.

It is not really 100% working, but the daily job is a bit of a hobby killer for me at the moment ... Would be nice if you guys could have look whether the generated code is ok for strings and buffers. For numbers the code generation still needs to be implemented (see here), but my head is exploding...

Here is my test flow:

[{"id":"935786d92e05853b","type":"Blockly","z":"c2a7925b.6e143","func":"(Buffer.alloc(0))[0] = 123456;\n(Buffer.alloc(0))[0] = (Buffer.alloc(1, 255))[0];\n(Buffer.alloc(0))[0] = '123456'.charAt(0);\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <block type=\"buffer_set_index\" id=\"+/8nH+xwf5(OS@]2DKKz\" x=\"-463\" y=\"-187\">\n    <value name=\"INDEX\">\n      <shadow type=\"math_number\" id=\"ddwr~I.`,0*vaRov}?/u\">\n        <field name=\"NUM\">1</field>\n      </shadow>\n    </value>\n    <value name=\"BUFFER\">\n      <shadow type=\"buffer_empty\" id=\"/?@Vr{yb4i]X1#ZzNR(D\"></shadow>\n    </value>\n    <value name=\"VALUE\">\n      <block type=\"math_number\" id=\"^q59jH:{pIzwq;@=TH[h\">\n        <field name=\"NUM\">123456</field>\n      </block>\n    </value>\n    <next>\n      <block type=\"buffer_set_index\" id=\"{$~{Vv|F4;v7fS3bN#OZ\">\n        <value name=\"INDEX\">\n          <shadow type=\"math_number\" id=\"}+QIQ-|k(.RauNL/$~BG\">\n            <field name=\"NUM\">1</field>\n          </shadow>\n        </value>\n        <value name=\"BUFFER\">\n          <shadow type=\"buffer_empty\" id=\"n}:p:TdXQ+=Iz+h9w9[X\"></shadow>\n        </value>\n        <value name=\"VALUE\">\n          <block type=\"buffer_byte\" id=\"B2r/`@yJyJ`_yh?{M^Ix\">\n            <field name=\"BYTE_VALUE\">255</field>\n          </block>\n        </value>\n        <next>\n          <block type=\"buffer_set_index\" id=\"7$o^+dCMW=e3Mu/0U5^j\">\n            <value name=\"INDEX\">\n              <shadow type=\"math_number\" id=\"~//2cZ[F[t=`@UN5w0Jj\">\n                <field name=\"NUM\">1</field>\n              </shadow>\n            </value>\n            <value name=\"BUFFER\">\n              <shadow type=\"buffer_empty\" id=\"$Ug$F+(_6yt!Tv7Gp(u;\"></shadow>\n            </value>\n            <value name=\"VALUE\">\n              <block type=\"text\" id=\"hEM2=v%U|E~W7%xh1tZ3\">\n                <field name=\"TEXT\">123456</field>\n              </block>\n            </value>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":1140,"y":360,"wires":[[]]},{"id":"9512866d85dd94d0","type":"inject","z":"c2a7925b.6e143","name":"Set byte in buffer","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":940,"y":360,"wires":[["935786d92e05853b"]]},{"id":"d7a036fa.2c0298","type":"blockly-config","language":"en","showTrashcan":true,"allowComments":true,"showZoomControl":true,"enableBackPack":true,"backpackContents":[],"toolboxPosition":"left","renderer":"geras","categories":[{"name":"Node-RED","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Objects","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Buffer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Date/time","files":["blockly-contrib/npm/@blockly%2Ffield-date/dist/date_compressed.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Timer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly extension","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly standard","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml"]},{"name":"Custom category","files":["blockly-contrib/file/c:/temp/myblocks/myBlocksCodeGen.js","blockly-contrib/file/c:/temp/myblocks/myBlocksDefs.js","blockly-contrib/file/c:/temp/myblocks/myToolbox.xml","blockly-contrib/file/c:/temp/myblocks/my_messages/en.js"]}],"customizeToolbox":false,"name":"Left"}]

Thanks a lot !!!

cymplecy commented 3 years ago

Not seeing any behaviour change on my system

cymplecy commented 3 years ago

I went and checked to see if you'd committed it and I see you've removed release-1.1.0 and merged everthing into master :)

I didn't read the error msg when I updated :)

I'll test using master

cymplecy commented 3 years ago

Working now :)

Will do some more testing

cymplecy commented 3 years ago

How to you want to proceed with this. I've got ideas on how to improve things a bit. Do you want me to:

1 Just post them here one at a time

Do all changes/suggestions and then post them here
Do all changes/suggestions in a fork that you can then pull?

bartbutenaers commented 3 years ago

Hi Simon, Yes I should have told that I merged everything to master, so we have a nice clean repository...

Thanks for analyzing the buffer related blocks!

I assume you are only changing the block definition and code generator files. Just do your changes in both files, and describe here what and why you have done it. Then we can try it and discuss it with you...

cymplecy commented 3 years ago

1st issue to discuss is that the get byte from buffer doesn't return a byte (buffer length 1)

but I'd like to suggest that instead of returning a buffer length 1 - the text should be changed to

get value of index ... of buffer ...

as returning a single byte value isn't going to be used much

or maybe it could be modified with a dropdown to return a value, single char text string or a byte???

bartbutenaers commented 3 years ago

Quick feedback, because I had to work today from 7:45 to 21:30 ...

the text should be changed

Damn that is a pity. I had thought we only had to change code. Now I have to ask the translator guys again to translate again... But indeed it makes sense what you suggest.

or maybe it could be modified with a dropdown to return a value, single char text string or a byte???

Personally I don't really like that, because it will simply return the data that is in the buffer at the specified index...

bartbutenaers commented 3 years ago

@cymplecy, Do you like me to implement this?

cymplecy commented 3 years ago

No - leave it with me - had some family stuff to deal with over weekend but will be working on it today :)

On Sun, 25 Jul 2021 at 23:01, bartbutenaers @.***> wrote:

@cymplecy https://github.com/cymplecy, Do you like me to implement this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bartbutenaers/node-red-contrib-blockly/issues/62#issuecomment-886263531, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR7RNBSKTQ7C5G4AEFX57TTZSCSNANCNFSM45S237FQ .

cymplecy commented 3 years ago

So to do the change just alter in en.js

cymplecy commented 3 years ago

I've been playing with trying to get it to handle the differences between numbers,strings, buffer and variable types and come up with this

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  try {  // to see if we can get the type of the value
    var dataType = block.getInput('VALUE').connection.targetConnection.getCheck()[0];
  } catch { // if not - assume its a variable
    const code = buffer + '[' + index + '] = ' + value + ';\n';
    return code;
  }

  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      // Get the first character of the input string
      value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
      value = value.charCodeAt(0);
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
  }

Basic philosphy is that if it fails to find the datatype, then it's a variable and it returns the code to deal with that

If it's a number - it makes sure its in the range 0 - 255

If its a string - it takes the value of the 1st char of the string

if it's a byte - then it uses the value of the byte

It seems to handle all 4 scenarios in my testing

It will probably fail if the contents of the variable are not numeric but maybe some more JS could make it fail safely

cymplecy commented 3 years ago

My testing flow

[{"id":"935786d92e05853b","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testBuffer;\n\n\ntestBuffer = (Buffer.alloc(10));\ntestBuffer[0] = 254;\nmsg['payload'] = testBuffer;\nreturn msg;\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"22_3lZTO,U+Yg[$S,6Qf\" x=\"-387\" y=\"-587\">\n    <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n    <value name=\"VALUE\">\n      <block type=\"buffer_alloc\" id=\"gOpuNQ.fpDvk|f~VV8n-\">\n        <value name=\"LENGTH\">\n          <shadow type=\"math_number\" id=\"/h@`xv2)0Cuib]mc}vP`\">\n            <field name=\"NUM\">10</field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"buffer_set_index\" id=\"7$o^+dCMW=e3Mu/0U5^j\">\n        <value name=\"INDEX\">\n          <shadow type=\"math_number\" id=\"~//2cZ[F[t=`@UN5w0Jj\">\n            <field name=\"NUM\">1</field>\n          </shadow>\n        </value>\n        <value name=\"BUFFER\">\n          <shadow type=\"buffer_empty\" id=\"$Ug$F+(_6yt!Tv7Gp(u;\"></shadow>\n          <block type=\"variables_get\" id=\"KTR5:V1bFT0k`_H6]0#q\">\n            <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n          </block>\n        </value>\n        <value name=\"VALUE\">\n          <block type=\"math_number\" id=\"a(JS7vsw.3WnJXXUpyuo\">\n            <field name=\"NUM\">254</field>\n          </block>\n        </value>\n        <next>\n          <block type=\"node_object_set\" id=\"C3$AJ{J$X4GMcQ~P.Y}C\" inline=\"true\">\n            <value name=\"object_field\">\n              <shadow type=\"node_msg\" id=\"@k~uw$CUXFpD(YZJT^*x\"></shadow>\n            </value>\n            <value name=\"field_name\">\n              <shadow type=\"text\" id=\"~,*uud-2mQ+~{i}a:[+y\">\n                <field name=\"TEXT\">payload</field>\n              </shadow>\n            </value>\n            <value name=\"value_field\">\n              <shadow type=\"text\">\n                <field name=\"TEXT\"></field>\n              </shadow>\n              <block type=\"variables_get\" id=\"N{mmHXgP84oF_Tj0%i]g\">\n                <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_return_message\" id=\"Ztq4^UzX6wdbv9i:{a*F\">\n                <field name=\"OUTPUT_NR\">1</field>\n                <value name=\"MESSAGE_INPUT\">\n                  <shadow type=\"node_msg\" id=\"vY1|C#bvL$?ixB0t4x3k\"></shadow>\n                </value>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":320,"y":60,"wires":[["ce8a88fc.ada2b8"]]},{"id":"9512866d85dd94d0","type":"inject","z":"0ebed3e2bb0753df","name":"Set byte in buffer","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":120,"y":120,"wires":[["935786d92e05853b","d08f055843d1d370","562f46f07824a1f9","61b8cf83e18214e8"]]},{"id":"ce8a88fc.ada2b8","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":510,"y":60,"wires":[]},{"id":"73d66ccb.24a5b4","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":510,"y":120,"wires":[]},{"id":"7219ba03.df7314","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":510,"y":180,"wires":[]},{"id":"d08f055843d1d370","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testBuffer;\n\n\ntestBuffer = (Buffer.alloc(10));\ntestBuffer[0] = 104;\nmsg['payload'] = testBuffer;\nreturn msg;\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"22_3lZTO,U+Yg[$S,6Qf\" x=\"-387\" y=\"-587\">\n    <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n    <value name=\"VALUE\">\n      <block type=\"buffer_alloc\" id=\"gOpuNQ.fpDvk|f~VV8n-\">\n        <value name=\"LENGTH\">\n          <shadow type=\"math_number\" id=\"/h@`xv2)0Cuib]mc}vP`\">\n            <field name=\"NUM\">10</field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"buffer_set_index\" id=\"+/8nH+xwf5(OS@]2DKKz\">\n        <value name=\"INDEX\">\n          <shadow type=\"math_number\" id=\"ddwr~I.`,0*vaRov}?/u\">\n            <field name=\"NUM\">1</field>\n          </shadow>\n        </value>\n        <value name=\"BUFFER\">\n          <shadow type=\"buffer_empty\" id=\"/?@Vr{yb4i]X1#ZzNR(D\"></shadow>\n          <block type=\"variables_get\" id=\"Nw|k`Ev3nCx6ojPpH}{%\">\n            <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n          </block>\n        </value>\n        <value name=\"VALUE\">\n          <block type=\"text\" id=\"qvN~}Ti8MA!{RR%EFUlP\">\n            <field name=\"TEXT\">hello</field>\n          </block>\n        </value>\n        <next>\n          <block type=\"node_object_set\" id=\"g]f+4#M,V)aj2So-m7M}\" inline=\"true\">\n            <value name=\"object_field\">\n              <shadow type=\"node_msg\" id=\"x!8r;JYSdR{+b!pbO_UM\"></shadow>\n            </value>\n            <value name=\"field_name\">\n              <shadow type=\"text\" id=\"neD;Gi@j#~zpZ}zUZHqJ\">\n                <field name=\"TEXT\">payload</field>\n              </shadow>\n            </value>\n            <value name=\"value_field\">\n              <shadow type=\"text\" id=\"-Bo*3[1Cvj@bf{vOli7F\">\n                <field name=\"TEXT\"></field>\n              </shadow>\n              <block type=\"variables_get\" id=\"SOUV)fY1i=,Os191tH,v\">\n                <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_return_message\" id=\"`02SC[6h{,{u@gs0X4hW\">\n                <field name=\"OUTPUT_NR\">1</field>\n                <value name=\"MESSAGE_INPUT\">\n                  <shadow type=\"node_msg\" id=\"}:0JW{Ow,NS$O$lvS$o`\"></shadow>\n                </value>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":320,"y":120,"wires":[["73d66ccb.24a5b4"]]},{"id":"562f46f07824a1f9","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testBuffer;\n\n\ntestBuffer = (Buffer.alloc(10));\ntestBuffer[0] = (Buffer.alloc(1, 127))[0];\nmsg['payload'] = testBuffer;\nreturn msg;\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"22_3lZTO,U+Yg[$S,6Qf\" x=\"-387\" y=\"-587\">\n    <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n    <value name=\"VALUE\">\n      <block type=\"buffer_alloc\" id=\"gOpuNQ.fpDvk|f~VV8n-\">\n        <value name=\"LENGTH\">\n          <shadow type=\"math_number\" id=\"/h@`xv2)0Cuib]mc}vP`\">\n            <field name=\"NUM\">10</field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"buffer_set_index\" id=\"{$~{Vv|F4;v7fS3bN#OZ\">\n        <value name=\"INDEX\">\n          <shadow type=\"math_number\" id=\"}+QIQ-|k(.RauNL/$~BG\">\n            <field name=\"NUM\">1</field>\n          </shadow>\n        </value>\n        <value name=\"BUFFER\">\n          <shadow type=\"buffer_empty\" id=\"n}:p:TdXQ+=Iz+h9w9[X\"></shadow>\n          <block type=\"variables_get\" id=\"xOq/5k(-hTm1lL}etL;2\">\n            <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n          </block>\n        </value>\n        <value name=\"VALUE\">\n          <block type=\"buffer_byte\" id=\"B2r/`@yJyJ`_yh?{M^Ix\">\n            <field name=\"BYTE_VALUE\">127</field>\n          </block>\n        </value>\n        <next>\n          <block type=\"node_object_set\" id=\"R@H@}jC/Fp_9K1Gz7S9n\" inline=\"true\">\n            <value name=\"object_field\">\n              <shadow type=\"node_msg\" id=\":=bCe5*%VD@?_]g?3H(5\"></shadow>\n            </value>\n            <value name=\"field_name\">\n              <shadow type=\"text\" id=\"]lB$ZtFJXRW33lj}|p/$\">\n                <field name=\"TEXT\">payload</field>\n              </shadow>\n            </value>\n            <value name=\"value_field\">\n              <shadow type=\"text\">\n                <field name=\"TEXT\"></field>\n              </shadow>\n              <block type=\"variables_get\" id=\"yfj{Js,YA-`;MI0ai@:8\">\n                <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_return_message\" id=\"[aXz(jH!KTMNQw.~F=YL\">\n                <field name=\"OUTPUT_NR\">1</field>\n                <value name=\"MESSAGE_INPUT\">\n                  <shadow type=\"node_msg\" id=\"6PC*c1vQ?0cB8lMdx_X-\"></shadow>\n                </value>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":320,"y":180,"wires":[["7219ba03.df7314"]]},{"id":"dab3681868768c11","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":510,"y":240,"wires":[]},{"id":"61b8cf83e18214e8","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testVariable, testBuffer;\n\n\ntestVariable = 129;\ntestBuffer = (Buffer.alloc(10));\ntestBuffer[0] = testVariable;\nmsg['payload'] = testBuffer;\nreturn msg;\n\n254;\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</variable>\n    <variable id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"bDUeNBJ~rcEy_RX0tw+O\" x=\"-362\" y=\"-612\">\n    <field name=\"VAR\" id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</field>\n    <value name=\"VALUE\">\n      <block type=\"math_number\" id=\"Rt8~jyotbH+}q[-tLHHb\">\n        <field name=\"NUM\">129</field>\n      </block>\n    </value>\n    <next>\n      <block type=\"variables_set\" id=\"22_3lZTO,U+Yg[$S,6Qf\">\n        <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n        <value name=\"VALUE\">\n          <block type=\"buffer_alloc\" id=\"gOpuNQ.fpDvk|f~VV8n-\">\n            <value name=\"LENGTH\">\n              <shadow type=\"math_number\" id=\"/h@`xv2)0Cuib]mc}vP`\">\n                <field name=\"NUM\">10</field>\n              </shadow>\n            </value>\n          </block>\n        </value>\n        <next>\n          <block type=\"buffer_set_index\" id=\"7$o^+dCMW=e3Mu/0U5^j\">\n            <value name=\"INDEX\">\n              <shadow type=\"math_number\" id=\"~//2cZ[F[t=`@UN5w0Jj\">\n                <field name=\"NUM\">1</field>\n              </shadow>\n            </value>\n            <value name=\"BUFFER\">\n              <shadow type=\"buffer_empty\" id=\"$Ug$F+(_6yt!Tv7Gp(u;\"></shadow>\n              <block type=\"variables_get\" id=\"KTR5:V1bFT0k`_H6]0#q\">\n                <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n              </block>\n            </value>\n            <value name=\"VALUE\">\n              <block type=\"variables_get\" id=\"I*^,}^9JD$k.czaACOBp\">\n                <field name=\"VAR\" id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</field>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_object_set\" id=\"C3$AJ{J$X4GMcQ~P.Y}C\" inline=\"true\">\n                <value name=\"object_field\">\n                  <shadow type=\"node_msg\" id=\"@k~uw$CUXFpD(YZJT^*x\"></shadow>\n                </value>\n                <value name=\"field_name\">\n                  <shadow type=\"text\" id=\"~,*uud-2mQ+~{i}a:[+y\">\n                    <field name=\"TEXT\">payload</field>\n                  </shadow>\n                </value>\n                <value name=\"value_field\">\n                  <shadow type=\"text\">\n                    <field name=\"TEXT\"></field>\n                  </shadow>\n                  <block type=\"variables_get\" id=\"N{mmHXgP84oF_Tj0%i]g\">\n                    <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n                  </block>\n                </value>\n                <next>\n                  <block type=\"node_return_message\" id=\"Ztq4^UzX6wdbv9i:{a*F\">\n                    <field name=\"OUTPUT_NR\">1</field>\n                    <value name=\"MESSAGE_INPUT\">\n                      <shadow type=\"node_msg\" id=\"vY1|C#bvL$?ixB0t4x3k\"></shadow>\n                    </value>\n                  </block>\n                </next>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n  <block type=\"math_number\" id=\"a(JS7vsw.3WnJXXUpyuo\" x=\"-112\" y=\"-413\">\n    <field name=\"NUM\">254</field>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":320,"y":240,"wires":[["dab3681868768c11"]]},{"id":"d7a036fa.2c0298","type":"blockly-config","language":"en","showTrashcan":true,"allowComments":true,"showZoomControl":true,"enableBackPack":true,"backpackContents":[],"toolboxPosition":"left","renderer":"geras","categories":[{"name":"Node-RED","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Objects","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Buffer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Date/time","files":["blockly-contrib/npm/@blockly%2Ffield-date/dist/date_compressed.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Timer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly extension","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly standard","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml"]},{"name":"Custom category","files":["blockly-contrib/file/c:/temp/myblocks/myBlocksCodeGen.js","blockly-contrib/file/c:/temp/myblocks/myBlocksDefs.js","blockly-contrib/file/c:/temp/myblocks/myToolbox.xml","blockly-contrib/file/c:/temp/myblocks/my_messages/en.js"]}],"customizeToolbox":false,"name":"Left"}]

bartbutenaers commented 3 years ago

At first sight that seems to be logical. Thanks for the developments!! Yes indeed those variables are a pain in the ...

We can restructure the code a bit, to make sure we always go to the end of the generator function:

  // Get the data type checks offered by the block that is connected to our VALUE field
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
  }
  else {
     // We assume the input is a variable
     dataType = "Variable";
  }

  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      // Get the first character of the input string
      value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
      value = value.charCodeAt(0);
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
     case "Variable":
       // Keep the value as it is
  }

  const code = buffer + '[' + index + '] = ' + value + ';\n';
  return code;

Not sure if we can check if it is a variable. I only found this, to check to which type of block our input is connected:

But that won't be of much help to us, I'm afraid.

I have not much time tonight, so probably I will test your flow tomorrow evening.

cymplecy commented 3 years ago

I altered my code to your code and it worked fine :)

But then I had an idea - why not try and determine the contents of a variable at runtime and came up with this concept

  // Get the data type checks offered by the block that is connected to our VALUE field
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
  }
  else {
     // We assume the input is a variable
     //dataType = "Variable";
     let code = '';
     code += 'if (isNaN(' + value + ')) {\n';
       code += '    try {\n';
         code += '        ' + value + ' = ' + value + '.charCodeAt(0);\n';
       code += '    } catch {\n';
         code += '        ' + value + ' = ' + value + '[0]\n';
       code += '    }\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(' + value + ')) {\n';
         code += '        ' + value + ' = ' + value + '[0]\n';
       code += '    } else {\n';
         code += '        ' + value + ' = Math.min(Math.max(' + value + ', 0),255)\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + '] = ' + value + ';\n';
     return code;
  }

  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      // Get the first character of the input string
      value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
      value = value.charCodeAt(0);
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
     //case "Variable":
       // Keep the value as it is
  }

  const code = buffer + '[' + index + '] = ' + value + ';\n';
  return code;
};

Tested using this

I'm sure the actual JS could be improved but it seems to work :)

cymplecy commented 3 years ago

Modified version as it doesn't need the try/catch

  // Get the data type checks offered by the block that is connected to our VALUE field
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
  }
  else {
     // We assume the input is a variable
     //dataType = "Variable";
     let code = '';
     code += 'if (isNaN(' + value + ')) {\n';
         code += '    ' + value + ' = ' + value + '.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(' + value + ')) {\n';
         code += '        ' + value + ' = ' + value + '[0]\n';
       code += '    } else {\n';
         code += '        ' + value + ' = Math.min(Math.max(' + value + ', 0),255)\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + '] = ' + value + ';\n';
     return code;
  }

  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      // Get the first character of the input string
      value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
      value = value.charCodeAt(0);
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
     //case "Variable":
       // Keep the value as it is
  }

  const code = buffer + '[' + index + '] = ' + value + ';\n';
  return code;
};

test flow

[{"id":"dab3681868768c11","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":510,"y":240,"wires":[]},{"id":"61b8cf83e18214e8","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testVariable, testBuffer;\n\n\ntestVariable = (msg['payload']);\ntestBuffer = (Buffer.alloc(10));\nif (isNaN(testVariable)) {\n    testVariable = testVariable.charCodeAt(0);\n} else {\n    if (Buffer.isBuffer(testVariable)) {\n        testVariable = testVariable[0]\n    } else {\n        testVariable = Math.min(Math.max(testVariable, 0),255)\n    }\n}\ntestBuffer[0] = testVariable;\nmsg['payload'] = testBuffer;\nreturn msg;\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</variable>\n    <variable id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"bDUeNBJ~rcEy_RX0tw+O\" x=\"-362\" y=\"-612\">\n    <field name=\"VAR\" id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</field>\n    <value name=\"VALUE\">\n      <block type=\"node_object_get\" id=\"i,Z|iYA$zak88n[WSlEd\">\n        <mutation xmlns=\"http://www.w3.org/1999/xhtml\" action=\"GET\"></mutation>\n        <field name=\"action\">GET</field>\n        <value name=\"object\">\n          <shadow type=\"node_msg\" id=\"=88|h?-N6h;kb;?5`/Uv\"></shadow>\n        </value>\n        <value name=\"field_name\">\n          <shadow type=\"text\" id=\"hu=AkkX*0In*g,::C95h\">\n            <field name=\"TEXT\">payload</field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"variables_set\" id=\"22_3lZTO,U+Yg[$S,6Qf\">\n        <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n        <value name=\"VALUE\">\n          <block type=\"buffer_alloc\" id=\"gOpuNQ.fpDvk|f~VV8n-\">\n            <value name=\"LENGTH\">\n              <shadow type=\"math_number\" id=\"/h@`xv2)0Cuib]mc}vP`\">\n                <field name=\"NUM\">10</field>\n              </shadow>\n            </value>\n          </block>\n        </value>\n        <next>\n          <block type=\"buffer_set_index\" id=\"7$o^+dCMW=e3Mu/0U5^j\">\n            <value name=\"INDEX\">\n              <shadow type=\"math_number\" id=\"~//2cZ[F[t=`@UN5w0Jj\">\n                <field name=\"NUM\">1</field>\n              </shadow>\n            </value>\n            <value name=\"BUFFER\">\n              <shadow type=\"buffer_empty\" id=\"$Ug$F+(_6yt!Tv7Gp(u;\"></shadow>\n              <block type=\"variables_get\" id=\"KTR5:V1bFT0k`_H6]0#q\">\n                <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n              </block>\n            </value>\n            <value name=\"VALUE\">\n              <block type=\"variables_get\" id=\"I*^,}^9JD$k.czaACOBp\">\n                <field name=\"VAR\" id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</field>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_object_set\" id=\"C3$AJ{J$X4GMcQ~P.Y}C\" inline=\"true\">\n                <value name=\"object_field\">\n                  <shadow type=\"node_msg\" id=\"@k~uw$CUXFpD(YZJT^*x\"></shadow>\n                </value>\n                <value name=\"field_name\">\n                  <shadow type=\"text\" id=\"~,*uud-2mQ+~{i}a:[+y\">\n                    <field name=\"TEXT\">payload</field>\n                  </shadow>\n                </value>\n                <value name=\"value_field\">\n                  <shadow type=\"text\">\n                    <field name=\"TEXT\"></field>\n                  </shadow>\n                  <block type=\"variables_get\" id=\"N{mmHXgP84oF_Tj0%i]g\">\n                    <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n                  </block>\n                </value>\n                <next>\n                  <block type=\"node_return_message\" id=\"Ztq4^UzX6wdbv9i:{a*F\">\n                    <field name=\"OUTPUT_NR\">1</field>\n                    <value name=\"MESSAGE_INPUT\">\n                      <shadow type=\"node_msg\" id=\"vY1|C#bvL$?ixB0t4x3k\"></shadow>\n                    </value>\n                  </block>\n                </next>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"Insert value at index","x":320,"y":240,"wires":[["dab3681868768c11"]]},{"id":"f53404bf2a6d3c13","type":"inject","z":"0ebed3e2bb0753df","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Hello","payloadType":"str","x":110,"y":360,"wires":[["61b8cf83e18214e8"]]},{"id":"9c6b5188c9f4420b","type":"inject","z":"0ebed3e2bb0753df","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"123","payloadType":"num","x":110,"y":240,"wires":[["61b8cf83e18214e8"]]},{"id":"38035f664ee39bb3","type":"inject","z":"0ebed3e2bb0753df","name":"Buffer [32]","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[32]","payloadType":"bin","x":120,"y":400,"wires":[["61b8cf83e18214e8"]]},{"id":"56e221cb2c2e5641","type":"inject","z":"0ebed3e2bb0753df","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"-100","payloadType":"num","x":110,"y":280,"wires":[["61b8cf83e18214e8"]]},{"id":"2686afe3689edfc9","type":"inject","z":"0ebed3e2bb0753df","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"512","payloadType":"num","x":110,"y":320,"wires":[["61b8cf83e18214e8"]]},{"id":"d7a036fa.2c0298","type":"blockly-config","language":"en","showTrashcan":true,"allowComments":true,"showZoomControl":true,"enableBackPack":true,"backpackContents":[],"toolboxPosition":"left","renderer":"geras","categories":[{"name":"Node-RED","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Objects","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Buffer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Date/time","files":["blockly-contrib/npm/@blockly%2Ffield-date/dist/date_compressed.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Timer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly extension","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly standard","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml"]},{"name":"Custom category","files":["blockly-contrib/file/c:/temp/myblocks/myBlocksCodeGen.js","blockly-contrib/file/c:/temp/myblocks/myBlocksDefs.js","blockly-contrib/file/c:/temp/myblocks/myToolbox.xml","blockly-contrib/file/c:/temp/myblocks/my_messages/en.js"]}],"customizeToolbox":false,"name":"Left"}]

bartbutenaers commented 3 years ago

@cymplecy, I have tested it and it works nice: when no variable is used, then the generated code is short and simple. When a variable is used, then the type checking is very nice. For me this is good enough! Nice work!! Is there anything else we need here, before we close the beta version gates?

cymplecy commented 3 years ago

I think we are good to go and put it out there for others to test. :)

cymplecy commented 3 years ago

It's just failed for me in a loop test - so wait please :(

cymplecy commented 3 years ago

Okay - I found two issues with this test code (Note paylod is string Hello)

The simple one is that the index cannot be auto-decremented at beginning of code as it might not be a number but may be a variable so the Block/JS 1/0 index correction needs to take place just before the code is returned. This effects the get value at index block as well

The second one complicates things further.
If the value is an expression (e.g not a plain number,string, buffer or variable) then it needs special treatment.

My solution is to create a tempoary variable called tempVariable and add the code to evaluate it at runtime and then use the result of that evaluation.

It all seems to work but the code is quite a mess now :(

Blockly.JavaScript['buffer_set_index'] = function(block) {
  var   index = Blockly.JavaScript.valueToCode(block, 'INDEX', Blockly.JavaScript.ORDER_ATOMIC) || 1;
  //console.log(index);
  const buffer = Blockly.JavaScript.valueToCode(block, 'BUFFER', Blockly.JavaScript.ORDER_ATOMIC);
  var   value = Blockly.JavaScript.valueToCode(block, 'VALUE', Blockly.JavaScript.ORDER_ATOMIC);

  // Blockly is 1-based while Javascript is 0-based, so the index needs to be converted
  // This doesn't work if index is a variable so commented out
  //index--;

  // Get the data type checks offered by the block that is connected to our VALUE field 
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType = null;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
     //console.log(dataType);
     //console.log(value);
  }

  if (dataType === null) {
     // We assume the input is a variable
     let code = '';
     code += 'if (isNaN(' + value + ')) {\n';
         code += '    ' + value + ' = ' + value + '.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(' + value + ')) {\n';
         code += '        ' + value + ' = ' + value + '[0]\n';
       code += '    } else {\n';
         code += '        ' + value + ' = Math.min(Math.max(' + value + ', 0),255)\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = ' + value + ';\n';
     return code;
  }

  if ((dataType == 'String') && (value.charAt(0) == '(') && (value.charAt(value.length - 1) == ')') ) {
    // We assume the input is a code expression
    let code = '';
    code += 'let __tempValue__ = ' + value + ';\n';
    code += 'if (isNaN(__tempValue__)) {\n';
         code += '    __tempValue__ = __tempValue__.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(__tempValue__)) {\n';
         code += '        __tempValue__ = __tempValue__[0]\n';
       code += '    } else {\n';
         code += '        __tempValue__ = Math.min(Math.max(__tempValue__, 0),255)\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = __tempValue__;\n';
     return code;
  }

  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      //check if string is JS code
      if ((value.charAt(0) != '(') && (value.charAt(value.length - 1) != ')')  ) {
      // Get the first character of the input string
        value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
        value = value.charCodeAt(0);
      }
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
     //case "Variable":
       // Keep the value as it is
  }

  const code = buffer + '[' + index + ' - 1] = ' + value + ';\n';
  return code;
};

Blockly.JavaScript['buffer_get_index'] = function(block) {
  var   index = Blockly.JavaScript.valueToCode(block, 'INDEX', Blockly.JavaScript.ORDER_ATOMIC) || 1;
  const buffer = Blockly.JavaScript.valueToCode(block, 'BUFFER', Blockly.JavaScript.ORDER_ATOMIC);

  // Blockly is 1-based while Javascript is 0-based, so the index needs to be converted
  // This doesn't work if index is a variable so commented out
  //index--;

  const code = buffer + '[' + index + ' - 1]';
  return [code, Blockly.JavaScript.ORDER_NONE];
};

cymplecy commented 3 years ago

Slight issue - a modified looping fill buffer script

is converting the space between Hello and World to a 0 instead of 32

bartbutenaers commented 3 years ago

Hi @cymplecy,

Can you please share your flow, so that I can quickly test it.

And can you please explain a bit more in detail why the __tempValue__ had to be introduced. I'm pretty sure you do it correctly, but at first sight I don't see how e.g.

code += 'if (isNaN(' + value + ')) {\n';

is not correct for an expression, while:

code += 'let __tempValue__ = ' + value + ';\n';
code += 'if (isNaN(__tempValue__)) {\n';

would work better? Enlighten me please ;-)

cymplecy commented 3 years ago

[{"id":"aad9b2b9721868cb","type":"inject","z":"0ebed3e2bb0753df","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Hello World","payloadType":"str","x":130,"y":640,"wires":[["1fc93ad399ae75c3"]]},{"id":"6975824ad5fc3cba","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":550,"y":640,"wires":[]},{"id":"1fc93ad399ae75c3","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testVariable, testBuffer, i2;\n\n\ntestVariable = (msg['payload']);\ntestBuffer = (Buffer.alloc(10));\nvar i2_end = Math.min.apply(null, [testVariable.length, testBuffer.length]);\nvar i2_inc = 1;\nif (1 > i2_end) {\n  i2_inc = -i2_inc;\n}\nfor (i2 = 1; i2_inc >= 0 ? i2 <= i2_end : i2 >= i2_end; i2 += i2_inc) {\n  let __tempValue__ = (testVariable.slice((i2 - 1), i2));\n  if (isNaN(__tempValue__)) {\n      __tempValue__ = __tempValue__.charCodeAt(0);\n  } else {\n      if (Buffer.isBuffer(__tempValue__)) {\n          __tempValue__ = __tempValue__[0]\n      } else {\n          __tempValue__ = Math.min(Math.max(__tempValue__, 0),255)\n      }\n  }\n  testBuffer[i2 - 1] = __tempValue__;\n}\nmsg['payload'] = testBuffer;\nnode.send([msg]);\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\")ZBKTUC^^!J999c!;(dH\">testVariable</variable>\n    <variable id=\"cx:#6yd(cD`NZ32~!cqA\">testBuffer</variable>\n    <variable id=\")jU4tpD],qwF{9((T=aQ\">i</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"6?g29wT;A~@ji_TT/A(Y\" x=\"-63\" y=\"-212\">\n    <field name=\"VAR\" id=\")ZBKTUC^^!J999c!;(dH\">testVariable</field>\n    <value name=\"VALUE\">\n      <block type=\"node_object_get\" id=\"RHT8|1}}=EH.@V?L8_D)\">\n        <mutation xmlns=\"http://www.w3.org/1999/xhtml\" action=\"GET\"></mutation>\n        <field name=\"action\">GET</field>\n        <value name=\"object\">\n          <shadow type=\"node_msg\" id=\"u19[kk_q8Yi6^.Vo~OAC\"></shadow>\n        </value>\n        <value name=\"field_name\">\n          <shadow type=\"text\" id=\"v)Dxd*`^w/X:f_i.I[t(\">\n            <field name=\"TEXT\">payload</field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"variables_set\" id=\"b$;g^y.O~{njEy%uq/Xi\">\n        <field name=\"VAR\" id=\"cx:#6yd(cD`NZ32~!cqA\">testBuffer</field>\n        <value name=\"VALUE\">\n          <block type=\"buffer_alloc\" id=\"ZWUvTn7gm`*YtjYx8PgZ\">\n            <value name=\"LENGTH\">\n              <shadow type=\"math_number\" id=\"Ay]l-cD9-QMZ2H+5Ki^R\">\n                <field name=\"NUM\">10</field>\n              </shadow>\n            </value>\n          </block>\n        </value>\n        <next>\n          <block type=\"controls_for\" id=\"5|uarU#@t$Q99lhk/5K/\">\n            <field name=\"VAR\" id=\")jU4tpD],qwF{9((T=aQ\">i</field>\n            <value name=\"FROM\">\n              <shadow type=\"math_number\" id=\"qKtGgMI)FI]vgn!@HB9g\">\n                <field name=\"NUM\">1</field>\n              </shadow>\n            </value>\n            <value name=\"TO\">\n              <shadow type=\"math_number\" id=\"84aZ`1[F~A:((UaTWfhp\">\n                <field name=\"NUM\">5</field>\n              </shadow>\n              <block type=\"math_on_list\" id=\"Fui^^pp#8Yde7)S1G%$6\">\n                <mutation op=\"MIN\"></mutation>\n                <field name=\"OP\">MIN</field>\n                <value name=\"LIST\">\n                  <block type=\"lists_create_with\" id=\"GT.:/FS{g%mj=7oA$=HI\">\n                    <mutation items=\"2\"></mutation>\n                    <value name=\"ADD0\">\n                      <block type=\"text_length\" id=\"v_;U{KO9kR0(?=bBn6.$\">\n                        <value name=\"VALUE\">\n                          <shadow type=\"text\" id=\".kCoiLNeD`bLABJ%bh~3\">\n                            <field name=\"TEXT\">abc</field>\n                          </shadow>\n                          <block type=\"variables_get\" id=\"PB(F98[vtB;jDaz//gcT\">\n                            <field name=\"VAR\" id=\")ZBKTUC^^!J999c!;(dH\">testVariable</field>\n                          </block>\n                        </value>\n                      </block>\n                    </value>\n                    <value name=\"ADD1\">\n                      <block type=\"buffer_length\" id=\"vc*{)^RKT!$Po;3c#:*)\">\n                        <value name=\"BUFFER_INPUT\">\n                          <shadow type=\"buffer_empty\" id=\"Q*s^Qo%d%_XP%G,!WBK`\"></shadow>\n                          <block type=\"variables_get\" id=\"ZW`D3E6bm,;NtLhgl7}Z\">\n                            <field name=\"VAR\" id=\"cx:#6yd(cD`NZ32~!cqA\">testBuffer</field>\n                          </block>\n                        </value>\n                      </block>\n                    </value>\n                  </block>\n                </value>\n              </block>\n            </value>\n            <value name=\"BY\">\n              <shadow type=\"math_number\" id=\"$YXlbBa1H}%/U$wz^a4C\">\n                <field name=\"NUM\">1</field>\n              </shadow>\n            </value>\n            <statement name=\"DO\">\n              <block type=\"buffer_set_index\" id=\"jM.Q4g0lQ8-$9uEaws}q\" inline=\"false\">\n                <value name=\"INDEX\">\n                  <shadow type=\"math_number\" id=\"2vMhj{C/D`F3!R2.N1R2\">\n                    <field name=\"NUM\">1</field>\n                  </shadow>\n                  <block type=\"variables_get\" id=\"Q1XiaotCgxMAC?9PQ{`z\">\n                    <field name=\"VAR\" id=\")jU4tpD],qwF{9((T=aQ\">i</field>\n                  </block>\n                </value>\n                <value name=\"BUFFER\">\n                  <shadow type=\"buffer_empty\" id=\"H:Sr)H.$1il1Cih%rp_^\"></shadow>\n                  <block type=\"variables_get\" id=\"c?u34UG8hraXq%Ow_/.5\">\n                    <field name=\"VAR\" id=\"cx:#6yd(cD`NZ32~!cqA\">testBuffer</field>\n                  </block>\n                </value>\n                <value name=\"VALUE\">\n                  <shadow type=\"buffer_byte\" id=\"4hb*}}uAu:Ct]-(D/HWk\">\n                    <field name=\"BYTE_VALUE\">127</field>\n                  </shadow>\n                  <block type=\"text_getSubstring\" id=\"3Xg^Ih=;u@$W8jkf8p2s\">\n                    <mutation at1=\"true\" at2=\"true\"></mutation>\n                    <field name=\"WHERE1\">FROM_START</field>\n                    <field name=\"WHERE2\">FROM_START</field>\n                    <value name=\"STRING\">\n                      <block type=\"variables_get\" id=\"w8iKs:6;*6~cvr~PX$7M\">\n                        <field name=\"VAR\" id=\")ZBKTUC^^!J999c!;(dH\">testVariable</field>\n                      </block>\n                    </value>\n                    <value name=\"AT1\">\n                      <block type=\"variables_get\" id=\"^tMZ8{3o8UfzMx~)8.jy\">\n                        <field name=\"VAR\" id=\")jU4tpD],qwF{9((T=aQ\">i</field>\n                      </block>\n                    </value>\n                    <value name=\"AT2\">\n                      <block type=\"variables_get\" id=\"Wy_``_G}ej#7X8(8dWAz\">\n                        <field name=\"VAR\" id=\")jU4tpD],qwF{9((T=aQ\">i</field>\n                      </block>\n                    </value>\n                  </block>\n                </value>\n              </block>\n            </statement>\n            <next>\n              <block type=\"node_object_set\" id=\"T`]uSK?]/[w:jE:Vx%oq\" inline=\"true\">\n                <value name=\"object_field\">\n                  <shadow type=\"node_msg\" id=\"M]SQx{ba}1W7I@fGv(e8\"></shadow>\n                </value>\n                <value name=\"field_name\">\n                  <shadow type=\"text\" id=\"aFVf2mVPAgU3-c)0c;29\">\n                    <field name=\"TEXT\">payload</field>\n                  </shadow>\n                </value>\n                <value name=\"value_field\">\n                  <shadow type=\"text\" id=\"7*882J=9M!1=u`$y!RM3\">\n                    <field name=\"TEXT\"></field>\n                  </shadow>\n                  <block type=\"variables_get\" id=\"b+e5cTeOeAtOZL6BHJkH\">\n                    <field name=\"VAR\" id=\"cx:#6yd(cD`NZ32~!cqA\">testBuffer</field>\n                  </block>\n                </value>\n                <next>\n                  <block type=\"node_send\" id=\"wJ;Eh_RQE|YJiqa,%7;M\">\n                    <field name=\"OUTPUT_NR\">1</field>\n                    <value name=\"MESSAGE_INPUT\">\n                      <shadow type=\"node_msg\" id=\"h`33W1BFD,t?urGt=g:T\"></shadow>\n                    </value>\n                  </block>\n                </next>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"46e073e1.66e10c","backpackContents":[],"noerr":0,"name":"Fill buffer from payload string","x":340,"y":640,"wires":[["6975824ad5fc3cba"]]},{"id":"46e073e1.66e10c","type":"blockly-config","language":"en","showTrashcan":true,"allowComments":true,"showZoomControl":true,"enableBackPack":true,"backpackContents":["<block xmlns=\"https://developers.google.com/blockly/xml\" type=\"object_create\" inline=\"true\"><mutation xmlns=\"http://www.w3.org/1999/xhtml\" num_fields=\"1\"><field name=\"property name\"></field></mutation><field name=\"field1\">payload</field></block>","<block xmlns=\"https://developers.google.com/blockly/xml\" type=\"node_return_message\"><field name=\"OUTPUT_NR\">1</field><value name=\"MESSAGE_INPUT\"><shadow type=\"node_msg\"></shadow></value></block>","<block xmlns=\"https://developers.google.com/blockly/xml\" type=\"node_object_set\" inline=\"true\"><value name=\"object_field\"><shadow type=\"node_msg\"></shadow></value><value name=\"field_name\"><shadow type=\"text\"><field name=\"TEXT\">payload</field></shadow></value><value name=\"value_field\"><shadow type=\"text\"><field name=\"TEXT\"></field></shadow></value></block>","<block xmlns=\"https://developers.google.com/blockly/xml\" type=\"node_object_get\"><mutation xmlns=\"http://www.w3.org/1999/xhtml\" action=\"GET\"></mutation><field name=\"action\">GET</field><value name=\"object\"><shadow type=\"node_msg\"></shadow></value><value name=\"field_name\"><shadow type=\"text\"><field name=\"TEXT\">payload</field></shadow></value></block>"],"toolboxPosition":"left","renderer":"geras","categories":[{"name":"Node-RED","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Objects","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Buffer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Date/time","files":["blockly-contrib/npm/@blockly%2Ffield-date/dist/date_compressed.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Timer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly extension","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly standard","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml"]}],"customizeToolbox":false,"name":""}]

cymplecy commented 3 years ago

"And can you please explain a bit more in detail why the tempValue had to be introduced."

In the test flow above, the byte at index is set to

value ends up being a string

(testVariable.slice((i2 - 1), i2))

So if we just pass it thru the old code - it sees it as a string (and not a variable) so the old code just ended up delivering 40 (the ASCII code for at (

So I think it needs to be evaluated at runtime (I may have gone too far down the rabbit hole and it might be much simpler to do)

So I came up with the idea of assigning the value of value (pardon using that expression) to a temp variable at runtime - decided on

 __tempValue__

to avoid namespace clashes with real variable names that users might use .

I'm pretty certain that the whole code can be simplfied but that can be done later on down the line - it just needs to work to get the beta out of the door

cymplecy commented 3 years ago

I've fixed my " " issue getting treated as a number zero instead of ASCII 32

  // Blockly is 1-based while Javascript is 0-based, so the index needs to be converted
  // This doesn't work if index is a variable so commented out
  //index--;

  // Get the data type checks offered by the block that is connected to our VALUE field 
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType = null;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
     //console.log(dataType);
     //console.log(value);
  }

  if (dataType === null) {
     // We assume the input is a variable
     let code = '';
     code += 'if (isNaN(' + value + ')) {\n';
         code += '    ' + value + ' = ' + value + '.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(' + value + ')) {\n';
         code += '        ' + value + ' = ' + value + '[0]\n';
       code += '    } else {\n';
        //JS thinks a single " " is !isNan so we need to treat it as a special case
         code += '        if (' + value + ' == " ") {\n';
           code += '            ' + value + ' = ' + value + '.charCodeAt(0);\n';
         code += '        } else {\n';
           code += '            ' + value + ' = Math.min(Math.max(' + value + ', 0),255)\n';
         code += '        }\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = ' + value + ';\n';
     return code;
  }

  if ((dataType == 'String') && (value.charAt(0) == '(') && (value.charAt(value.length - 1) == ')') ) {
    // We assume the input is a code expression
    let code = '';
    code += 'let __tempValue__ = ' + value + ';\n';
    //code += 'node.warn(__tempValue__ );\n';
    code += 'if (isNaN(__tempValue__)) {\n';
         //code += 'node.warn("thinks its a string");\n';
         code += '    __tempValue__ = __tempValue__.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(__tempValue__)) {\n';
         //code += 'node.warn("thinks its a buffer");\n';
         code += '        __tempValue__ = __tempValue__[0]\n';
       code += '    } else {\n';
         //code += 'node.warn("thinks its a number");\n';
         //JS thinks " " is !isNan so we need to treat it as a special case
         code += '        if (__tempValue__ == " ") {\n';
         code += '            __tempValue__ = __tempValue__.charCodeAt(0);\n';
         code += '        } else {\n';
         code += '            __tempValue__ = Math.min(Math.max(__tempValue__, 0),255)\n';
         code += '        }\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = __tempValue__;\n';
     return code;
  }

  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      //check if string is JS code
      if ((value.charAt(0) != '(') && (value.charAt(value.length - 1) != ')')  ) {
      // Get the first character of the input string
        value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
        value = value.charCodeAt(0);
      }
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
     //case "Variable":
       // Keep the value as it is
  }

  const code = buffer + '[' + index + ' - 1] = ' + value + ';\n';
  return code;
};

bartbutenaers commented 3 years ago

@cymplecy,

There is nasty thing in Blockly when you generate variable names, like e.g. the `__tempValue__ variable. Because as soon as you add more than one instance of such a block, you end up with duplicate variable names...

For example when I duplicate the buffer_set_index block:

Then code will be generated that contains an error:

P.S. Now you see why I wanted to have syntax error highlighting in this version of the blockly node ;-)

I think there are 3 solutions:

Make the variable name unique by adding a random value in the name. But then the code becomes typical unreadable generated code...
Make the variable name unique by adding a counter in the name.
Generate a function and call the same function from every block.

Personally I would generate a function (which contains your code), but I'm not sure anymore how we have to generate such a function. Because if we have N instances of such a block, we only want to generate the function code only once...

I will search how to do that...

cymplecy commented 3 years ago

I like 1 - I don't think this code is very readable at the moment so having a variable like

buffercodeTempValue_1456723

isn't going to make things much worse :) 2 would be nicer than 1 but then we would have to keep track of a counter and off we go down another rabbit hole :)

I vote for 1 just to get the beta out of the door :)

This whole code can be tidied up later :)

bartbutenaers commented 3 years ago

With "later" you mean in 3 years from now ;-)

My time is up for today.
I have asked it on the Blockly forum. Would be nice if we could generate readable code (i.e. a function with your code inside), because I assume some users try to learn from the generated code ... I think that such a function could also be called in if dataType === null. Is that correct?

If I don't get a simple to implement example, we will continue with one of the other options...

cymplecy commented 3 years ago

Doing a bit of googling, if I switched to using var tempValue instead of let wouldn't that solve the problem? [2nd edit - seems to work for me - just gives a warning triangle but works ]

Current code

  // Blockly is 1-based while Javascript is 0-based, so the index needs to be converted
  // This doesn't work if index is a variable so commented out
  //index--;

  // Get the data type checks offered by the block that is connected to our VALUE field 
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType = null;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
     //console.log(dataType);
     //console.log(value);
  }

  if (dataType === null) {
     // We assume the input is a variable
     let code = '';
     code += 'if (isNaN(' + value + ')) {\n';
         code += '    ' + value + ' = ' + value + '.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(' + value + ')) {\n';
         code += '        ' + value + ' = ' + value + '[0]\n';
       code += '    } else {\n';
        //JS thinks a single " " is !isNan so we need to treat it as a special case
         code += '        if (' + value + ' == " ") {\n';
           code += '            ' + value + ' = ' + value + '.charCodeAt(0);\n';
         code += '        } else {\n';
           code += '            ' + value + ' = Math.min(Math.max(' + value + ', 0),255)\n';
         code += '        }\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = ' + value + ';\n';
     return code;
  }

  if ((dataType == 'String') && (value.charAt(0) == '(') && (value.charAt(value.length - 1) == ')') ) {
    // We assume the input is a code expression
    let code = '';
    code += 'var __tempValue__ = ' + value + ';\n';
    //code += 'node.warn(__tempValue__ );\n';
    code += 'if (isNaN(__tempValue__)) {\n';
         //code += 'node.warn("thinks its a string");\n';
         code += '    __tempValue__ = __tempValue__.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(__tempValue__)) {\n';
         //code += 'node.warn("thinks its a buffer");\n';
         code += '        __tempValue__ = __tempValue__[0]\n';
       code += '    } else {\n';
         //code += 'node.warn("thinks its a number");\n';
         //JS thinks " " is !isNan so we need to treat it as a special case
         code += '        if (__tempValue__ == " ") {\n';
         code += '            __tempValue__ = __tempValue__.charCodeAt(0);\n';
         code += '        } else {\n';
         code += '            __tempValue__ = Math.min(Math.max(__tempValue__, 0),255)\n';
         code += '        }\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = __tempValue__;\n';
     return code;
  }

  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      //check if string is JS code
      if ((value.charAt(0) != '(') && (value.charAt(value.length - 1) != ')')  ) {
      // Get the first character of the input string
        value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
        value = value.charCodeAt(0);
      }
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
     //case "Variable":
       // Keep the value as it is
  }

  var code = buffer + '[' + index + ' - 1] = ' + value + ';\n';
  return code;
};

Blockly.JavaScript['buffer_get_index'] = function(block) {
  var   index = Blockly.JavaScript.valueToCode(block, 'INDEX', Blockly.JavaScript.ORDER_ATOMIC) || 1;
  const buffer = Blockly.JavaScript.valueToCode(block, 'BUFFER', Blockly.JavaScript.ORDER_ATOMIC);

  // Blockly is 1-based while Javascript is 0-based, so the index needs to be converted
  // This doesn't work if index is a variable so commented out
  //index--;

  const code = buffer + '[' + index + ' - 1]';
  return [code, Blockly.JavaScript.ORDER_NONE];
};

cymplecy commented 3 years ago

Just consolidated the code and tidied it up

Blockly.JavaScript['buffer_set_index'] = function(block) {
  var   index = Blockly.JavaScript.valueToCode(block, 'INDEX', Blockly.JavaScript.ORDER_ATOMIC) || 1;
  //console.log(index);
  const buffer = Blockly.JavaScript.valueToCode(block, 'BUFFER', Blockly.JavaScript.ORDER_ATOMIC);
  var   value = Blockly.JavaScript.valueToCode(block, 'VALUE', Blockly.JavaScript.ORDER_ATOMIC);

  // Get the data type checks offered by the block that is connected to our VALUE field 
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType = null;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
  }

  if ((dataType === null) || ((dataType == 'String') && (value.charAt(0) == '(') && (value.charAt(value.length - 1) == ')') )) {
    // We assume the input is a variable
    let code = '';
    if ((dataType == 'String') && (value.charAt(0) == '(') && (value.charAt(value.length - 1) == ')') ) {
    // We assume the input is a code expression so we need to evaluate it before continuing at runtime
      code += 'var __tempValue__ = ' + value + ';\n';
      value = '__tempValue__';
    }
    code += 'if (isNaN(' + value + ')) {\n';
        code += '  ' + value + ' = ' + value + '.charCodeAt(0);\n';
    code += '} else {\n';
      code += '  if (Buffer.isBuffer(' + value + ')) {\n';
        code += '    ' + value + ' = ' + value + '[0];\n';
      code += '  } else {\n';
       //JS thinks a single " " is !isNan so we need to treat it as a special case
        code += '    if (' + value + ' == " ") {\n';
          code += '      ' + value + ' = ' + value + '.charCodeAt(0);\n';
        code += '    } else {\n';
          code += '      ' + value + ' = Math.min(Math.max(' + value + ', 0),255);\n';
        code += '    }\n';
      code += '  }\n';
    code += '}\n';
    code += buffer + '[' + index + ' - 1] = ' + value + ';\n';
    return code;
  }

  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      // Get the first character of the input string
        value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
        value = value.charCodeAt(0);
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
  }

  var code = buffer + '[' + index + ' - 1] = ' + value + ';\n';
  return code;
};

Blockly.JavaScript['buffer_get_index'] = function(block) {
  var   index = Blockly.JavaScript.valueToCode(block, 'INDEX', Blockly.JavaScript.ORDER_ATOMIC) || 1;
  const buffer = Blockly.JavaScript.valueToCode(block, 'BUFFER', Blockly.JavaScript.ORDER_ATOMIC);

  const code = buffer + '[' + index + ' - 1]';
  return [code, Blockly.JavaScript.ORDER_NONE];
};

bartbutenaers commented 3 years ago

Ah, I didn't know that about "let". Good catch!!!

The code generator code looks very compact and nice this way!
Good enough for me to announce a beta version.

I will try tonight to:

create a git tag for the beta
create a github release for that tag
write a summary of all changes in the release page
publish a beta version on npm
announce it on discourse (@jsccjj : do you have a username on Discourse, so we can mention you?)

Remark: I still would like to generate a function for the "_buffer_setindex" block: because if we have N of such blocks in a workspace, then we generate N times the same duplicate code snippet (resulting heavy readable code ...). Would be nice if we could generate the code snippet once in a function, and call that function N times.

bartbutenaers commented 3 years ago

@cymplecy, @jsccjj, Have taken a few hours holiday... The release notes are ready: https://github.com/bartbutenaers/node-red-contrib-blockly/releases/tag/v2.0.0-beta.1 Would be nice if you could review them!

cymplecy commented 3 years ago

All good - minor suggestion just to clarify

The "byte" block doesn't return a number anymore, but a buffer of length 1 (containing a number between 0 and 255).

bartbutenaers commented 3 years ago

Suggestion is implemented and beta published on npm. Will put it on Discourse later on. Now I'm off ...

bartbutenaers commented 3 years ago

Remark: I still would like to generate a function for the "buffer_set_index" block: because if we have N of such blocks in a workspace, then we generate N times the same duplicate code snippet (resulting heavy readable code ...). Would be nice if we could generate the code snippet once in a function, and call that function N times.

@cymplecy, I now that you have just announced on Discourse that you will never have to write Javascript again ;-) However I would appreciate if you would do it one more time ... I have updated the _buffer_setbyte block to generate a function, based on Beka's guidelines in the Blockly forum.

When you have now e.g. multiple _buffer_setbyte blocks, with a variable as input:

Then a single function will be generated (containing your code), which is called by all of the buffer_set_byte blocks:

So no code duplication anymore. Hopefully I haven't corrupted your code. For example I have removed your tempVar because I 'think' that it will now also work because the value is now passed to the function as input parameter. But I might be mistaken. Then we have to put it back in place.

I would appreciate it if you could do your tests one more time. If this is finished we will publish the 2.0.0.beta-2, since that was the last thing that I wanted in this release ... Thanks!!

cymplecy commented 3 years ago

Initial tests seem to indicate an off by one error somewhere - just investigating further

bartbutenaers commented 3 years ago

Simon, If you can give me a flow and the expected result, then I will also have look.

cymplecy commented 3 years ago

I don't want to waste your time if its my testing - give me a few more minutes

bartbutenaers commented 3 years ago

No hurry!! Just please don't start swearing here :-D Because I cannot block you from this repo, since I need you here...