bartbutenaers / node-red-contrib-blockly

A Node Red node for visual programming a function using Blockly
Apache License 2.0
88 stars 22 forks source link

Blockly Editor not working #101

Closed guenter-ms closed 1 year ago

guenter-ms commented 1 year ago

I could successful install the node-red-contrib-blockly, but the editor is not working. When pulling blocks into the workspace, no code is generated in the "Generated Javascript" tab and after saving and reloading the node, the editor tab is empty.

What can I do/investigate? Any configuration data missing?

Environment is Homeassistant with nodered add-on, where everything else is running fine, e.g. function nodes.

See further details attached.

Thank you! Bildschirmfoto 2022-10-09 um 20 21 18

Bildschirmfoto 2022-10-09 um 20 21 28

Bildschirmfoto 2022-10-09 um 20 22 37

Bildschirmfoto 2022-10-09 um 20 19 41

![Bildschirmfoto 2022-10-09 um 20 18 48]

Bildschirmfoto 2022-10-09 um 20 18 30

(https://user-images.githubusercontent.com/6253315/194821789-eeff9107-9f6d-4d36-a92f-6a9181524fde.jpg)

bartbutenaers commented 1 year ago

Hi @guenter-ms, I don't use HomeAssistant, so not sure how if that can cause it somehow. The code generation (i.e. Blockly blocks to Javascript conversion) happens in the client side, which means in your browser. You can have a look in your browser console log (via your browser developer tools menu) to see if there are any errors or warnings, and share it here. Bart

guenter-ms commented 1 year ago

Hi Bart, thank you, I attached from the js console in chrome, see below, but I am unable to interpret that... Bildschirmfoto 2022-10-10 um 17 20 09

bartbutenaers commented 1 year ago

@guenter-ms, Sorry for the long delay. Complete lack of free time lately ...

I see two different issues in your console log:

  1. The Syntax Error. Not sure if this is related to the Blockly node or to another node. You can find it out using the following steps:

    1. Open your browser developer tools, where you see the syntax error.
    2. Refresh your browser screen where the Node-RED flow editor is running (while your developer tools window is still open)
    3. Now click on the link at the end of your Syntax error line in the console log:
    4. Take a screenshot of the code where the syntax error is and share it here.
    5. Also have a look at which node this is. You can see that normally in the file with the syntax error, because there should be somewhere a statement like RED.nodes.registerType('some_node_id',.
  2. The workspace-search is not defined. That is loaded from the server in this code snippet. Since this cannot be loaded from your Node-RED server, there should be somewhere in the "Network" tabsheet of your developer tools an error. Please share a screenshot of that also.

I assume there is something in the HomeAssistant integration that this node doesn't like for some reason...

guenter-ms commented 1 year ago

Hi bartbutenaers

and thank you.

I tried to get the best possible informations for you, see attached.

If you like, Homeassistant can be setup on a pi with a click, using https://www.raspberrypi.com/software/

What can I do?

Best Günter

Am 15.10.2022 um 21:53 schrieb bartbutenaers @.***>:

@guenter-ms https://github.com/guenter-ms, Sorry for the long delay. Complete lack of free time lately ...

I see two different issues in your console log:

The Syntax Error. Not sure if this is related to the Blockly node or to another node. You can find it out using the following steps:

Open your browser developer tools, where you see the syntax error. Refresh your browser screen where the Node-RED flow editor is running (while your developer tools window is still open) Now click on the link at the end of your Syntax error line in the console log: Take a screenshot of the code where the syntax error is and share it here. Also have a look at which node this is. You can see that normally in the file with the syntax error, because there should be somewhere a statement like RED.nodes.registerType('some_node_id',. The workspace-search is not defined. That is loaded from the server in this https://github.com/bartbutenaers/node-red-contrib-blockly/blob/master/blockly.html#L42 code snippet. Since this cannot be loaded from your Node-RED server, there should be somewhere in the "Network" tabsheet of your developer tools an error. Please share a screenshot of that also.

I assume there is something in the HomeAssistant integration that this node doesn't like for some reason...

— Reply to this email directly, view it on GitHub https://github.com/bartbutenaers/node-red-contrib-blockly/issues/101#issuecomment-1279822154, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPWWAZXUCIUGCYY3REFXD3WDMDTZANCNFSM6AAAAAARBDE74M. You are receiving this because you were mentioned.

bartbutenaers commented 1 year ago

Hi @guenter-ms, Don't see any images? I assume it is because you have replied via mail.

Regarding HomeAssistant: I have nearly no time for Node-RED, so installing/learning/testing other tools is no option for me...

guenter-ms commented 1 year ago

see images below!

Bildschirmfoto 2022-10-17 um 14 44 40 Bildschirmfoto 2022-10-17 um 14 56 09 Bildschirmfoto 2022-10-17 um 14 58 16

guenter-ms commented 1 year ago

Hi @bartbutenaers

Can you please comment?

Thank you! Guenter

bartbutenaers commented 1 year ago

Morning @guenter-ms,

Sorry for the delay! Six days already, that is not my common response time...
Complete lack of free time lately ;-(

Had already a series of issues about similar problems where people could not reach their endpoint, since there are a number of different settings available. So I made yesterday evening a diagram, to hopefully be able to troubleshoot this easier in the future:

https://discourse.nodered.org/t/add-a-diagram-drawing-to-the-configuration-documentation/69449

The fact that you use HomeAssistant makes it also more difficult for me to find the root cause. Because I don't have time to spend on HomeAssistant, so that is a big black box for me.

Did you adjust any of these parameters yourself (in the Node-RED settings.js file) to change the root url? Or do you know if the HomeAssistant plugin for Node-RED does change any of those?

From here on you can start troubleshooting it completely separate from Node-RED or HomeAssistant. The blue url's you see in your browser console log, you need to be able to copy and paste them in your browser. But they won't work of course, because they are (incorrectly) generated by this node. IF you could somehow be able to determine the real URL's then I can fix my node to generate the correct ul.

This node launches a http endpoint, where the browser can download all those files (index.js, date_compressed.js, ...). But the url's to that endpoint are not entirly correct. But not sure at the moment if you can easily determine the real url of the endpoint.

Any ideas are welcome! Will try to answer faster from here on...

bartbutenaers commented 1 year ago

BTW can you share a screenshot of a blockly-node related url that was accessed by your browser without problems. I assume in your "network" tab you can see - when you type 'blockly' in the filter field - you will see url's that had no issue. Those might help us troubleshooting...

guenter-ms commented 1 year ago

Morning bartbutenaers and thanks!

Below you will find the /config/node-red/settings.js

The comments may be interesting concerning path informations:

/**

/**

module.exports = { // Retry time in milliseconds for MQTT connections mqttReconnectTime: 15000,

// Retry time in milliseconds for Serial port connections serialReconnectTime: 15000,

// Retry time in milliseconds for TCP socket connections //socketReconnectTime: 10000,

// Timeout in milliseconds for TCP server socket connections // defaults to no timeout //socketTimeout: 120000,

// Timeout in milliseconds for HTTP request connections // defaults to 120 seconds //httpRequestTimeout: 120000,

// The maximum length, in characters, of any message sent to the debug sidebar tab debugMaxLength: 1000,

// The maximum number of messages nodes will buffer internally as part of their // operation. This applies across a range of nodes that operate on message sequences. // defaults to no limit. A value of 0 also means no limit is applied. //nodeMaxMessageBufferLength: 0,

// To disable the option for using local files for storing keys and certificates in the TLS configuration // node, set this to true //tlsConfigDisableLocalFiles: true,

// By default, the Node-RED UI is available at http://localhost:1880/ // The following property can be used to specify a different root path. // If set to false, this is disabled. //httpAdminRoot: '/admin',

// The following property can be used in place of 'httpAdminRoot' and 'httpNodeRoot', // to apply the same root to both parts. //httpRoot: '/red',

// When httpAdminRoot is used to move the UI to a different root path, the // following property can be used to identify a directory of static content // that should be served at http://localhost:1880/. //httpStatic: '/home/nol/node-red-static/',

// The maximum size of HTTP request that will be accepted by the runtime api. // Default: 5mb //apiMaxLength: '5mb',

// If you installed the optional node-red-dashboard you can set it's path // relative to httpRoot //ui: { path: "ui" },

// The following property can be used to disable the editor. The admin API // is not affected by this option. To disable both the editor and the admin // API, use either the httpRoot or httpAdminRoot properties //disableEditor: false,

// The following property can be used to configure cross-origin resource sharing // in the HTTP nodes. // See https://github.com/troygoode/node-cors#configuration-options for // details on its contents. The following is a basic permissive set of options: //httpNodeCors: { // origin: "*", // methods: "GET,PUT,POST,DELETE" //},

// If you need to set an http proxy please set an environment variable // called http_proxy (or HTTP_PROXY) outside of Node-RED in the operating system. // For example - http_proxy=http://myproxy.com:8080 // (Setting it here will have no effect) // You may also specify no_proxy (or NO_PROXY) to supply a comma separated // list of domains to not proxy, eg - no_proxy=.acme.co,.acme.co.uk

// The following property can be used to add a custom middleware function // in front of all http in nodes. This allows custom authentication to be // applied to all http in nodes, or any other sort of common request processing. //httpNodeMiddleware: function(req,res,next) { // // Handle/reject the request, or pass it on to the http in node by calling next(); // // Optionally skip our rawBodyParser by setting this to true; // //req.skipRawBodyParser = true; // next(); //},

// The following property can be used to verify websocket connection attempts. // This allows, for example, the HTTP request headers to be checked to ensure // they include valid authentication information. //webSocketNodeVerifyClient: function(info) { // // 'info' has three properties: // // - origin : the value in the Origin header // // - req : the HTTP request // // - secure : true if req.connection.authorized or req.connection.encrypted is set // // // // The function should return true if the connection should be accepted, false otherwise. // // // // Alternatively, if this function is defined to accept a second argument, callback, // // it can be used to verify the client asynchronously. // // The callback takes three arguments: // // - result : boolean, whether to accept the connection or not // // - code : if result is false, the HTTP error status to return // // - reason: if result is false, the HTTP reason string to return //},

// Anything in this hash is globally available to all functions. // It is accessed as context.global. // eg: // functionGlobalContext: { os:require('os') } // can be accessed in a function block as: // context.global.os

functionGlobalContext: { // os:require('os'), // jfive:require("johnny-five"), // j5board:require("johnny-five").Board({repl:false}) },

// The following property can be used to order the categories in the editor // palette. If a node's category is not in the list, the category will get // added to the end of the palette. // If not set, the following default order is used: paletteCategories: [ "home_assistant", "home_assistant entities", "subflows", "common", "function", "network", "sequence", "parser", "storage", ],

// Configure the logging output logging: { // Only console logging is currently supported console: { // Whether or not to include metric events in the log output metrics: false, // Whether or not to include audit events in the log output audit: false, }, },

// Customising the editor editorTheme: { projects: { // To enable the Projects feature, set this value to true enabled: false, }, }, };

guenter-ms commented 1 year ago

...and see screenshots from browser (good and bad url):

Bildschirmfoto 2022-10-23 um 12 37 18 Bildschirmfoto 2022-10-23 um 12 34 28

guenter-ms commented 1 year ago

...and my 3rd hint see NR add-on log at the end (The npm package @blockly is not installed (but required via the Blockly config node))!

s6-rc: info: service s6rc-oneshot-runner: starting s6-rc: info: service s6rc-oneshot-runner successfully started s6-rc: info: service fix-attrs: starting s6-rc: info: service fix-attrs successfully started s6-rc: info: service legacy-cont-init: starting cont-init: info: running /etc/cont-init.d/00-banner.sh

Add-on: Node-RED Flow-based programming for the Internet of Things

Add-on version: 13.4.0 You are running the latest version of this add-on. System: Home Assistant OS 9.2 (aarch64 / raspberrypi3-64) Home Assistant Core: 2022.10.4 Home Assistant Supervisor: 2022.10.0

Please, share the above information when looking for help or support in, e.g., GitHub, forums or the Discord chat.

cont-init: info: /etc/cont-init.d/00-banner.sh exited 0 cont-init: info: running /etc/cont-init.d/01-log-level.sh cont-init: info: /etc/cont-init.d/01-log-level.sh exited 0 cont-init: info: running /etc/cont-init.d/customizations.sh cont-init: info: /etc/cont-init.d/customizations.sh exited 0 cont-init: info: running /etc/cont-init.d/nginx.sh cont-init: info: /etc/cont-init.d/nginx.sh exited 0 cont-init: info: running /etc/cont-init.d/node-red.sh patching file nodes/ui_base.html Hunk #1 succeeded at 1164 (offset 633 lines). up to date, audited 104 packages in 11s 2 packages are looking for funding run npm fund for details 6 moderate severity vulnerabilities To address issues that do not require attention, run: npm audit fix To address all issues (including breaking changes), run: npm audit fix --force Run npm audit for details. cont-init: info: /etc/cont-init.d/node-red.sh exited 0 s6-rc: info: service legacy-cont-init successfully started s6-rc: info: service legacy-services: starting services-up: info: copying legacy longrun nginx (no readiness notification) services-up: info: copying legacy longrun nodered (no readiness notification) s6-rc: info: service legacy-services successfully started [13:18:00] INFO: Starting Node-RED...

start node $NODE_OPTIONS node_modules/node-red/red.js "--settings" "/etc/node-red/config.js" 23 Oct 13:18:14 - [info] Welcome to Node-RED

23 Oct 13:18:14 - [info] Node-RED version: v3.0.2 23 Oct 13:18:14 - [info] Node.js version: v16.16.0 23 Oct 13:18:14 - [info] Linux 5.15.61-v8 arm64 LE 23 Oct 13:18:21 - [info] Loading palette nodes 23 Oct 13:18:40 - [info] Dashboard version 3.2.0 started at /endpoint/ui 23 Oct 13:18:43 - [info] Settings file : /etc/node-red/config.js 23 Oct 13:18:43 - [info] Context store : 'default' [module=memory] 23 Oct 13:18:43 - [info] User directory : /config/node-red/ 23 Oct 13:18:43 - [warn] Projects disabled : editorTheme.projects.enabled=false 23 Oct 13:18:43 - [info] Flows file : /config/node-red/flows.json 23 Oct 13:18:44 - [warn] Encrypted credentials not found 23 Oct 13:18:44 - [info] Server now running at http://127.0.0.1:46836/ 23 Oct 13:18:44 - [info] Starting flows 23 Oct 13:18:44 - [info] Started flows [13:18:44] INFO: Starting NGinx... 23 Oct 13:18:49 - [info] [server:Home Assistant] Connecting to http://supervisor/core 23 Oct 13:18:49 - [info] [server:Home Assistant] Connected to http://supervisor/core 23 Oct 13:30:43 - [info] Stopping flows 23 Oct 13:30:43 - [info] [server:Home Assistant] Closing connection to http://supervisor/core 23 Oct 13:30:43 - [info] Stopped flows 23 Oct 13:30:43 - [info] Updated flows 23 Oct 13:30:43 - [info] Starting flows 23 Oct 13:30:43 - [info] Started flows 23 Oct 13:30:48 - [info] [server:Home Assistant] Connecting to http://supervisor/core 23 Oct 13:30:48 - [info] [server:Home Assistant] Connected to http://supervisor/core The npm package @blockly is not installed (but required via the Blockly config node) The npm package @blockly is not installed (but required via the Blockly config node) The npm package @blockly is not installed (but required via the Blockly config node) The npm package @blockly is not installed (but required via the Blockly config node) The npm package @blockly is not installed (but required via the Blockly config node) The npm package @blockly is not installed (but required via the Blockly config node) The npm package @blockly is not installed (but required via the Blockly config node) The npm package @blockly is not installed (but required via the Blockly config node)

bartbutenaers commented 1 year ago

That is weird. Is there a @blockly directory:

  1. In the same directory where you can find the node-red-contrib-blockly directory?
  2. As a subfolder of the node-red-contrib-blockly directory?

This node is going to search automatically all those directories, but perhaps it cannot find them.

Don't know wheere the HA addon installs the npm modules.

BTW very clever how the HA guys managed all of this...

guenter-ms commented 1 year ago

I found @blockly DIR there:

Bildschirmfoto 2022-10-23 um 14 02 08

bartbutenaers commented 1 year ago

That should be fine... And which subfolders does the @blockly folder has?

guenter-ms commented 1 year ago

Bildschirmfoto 2022-10-23 um 14 11 13

bartbutenaers commented 1 year ago

So everything seems yo be installed correctly. But I think my node cannot find these folders for reason. Need to think about that...

guenter-ms commented 1 year ago

This message makes me nervous: The npm package @blockly is not installed (but required via the Blockly config node)

bartbutenaers commented 1 year ago

Yes that is indeed the missing clue. It seems that this require.resolve statement doesn't return the directory of the specified blockly plugin for some reason.

Normally it works like this:

  1. You double click the node in the flow to open its config screen.

  2. The node will call the endpoint N times to load all the blockly libraries

  3. In the http request, the endpoint will receive the name of the blockly plugin as an http paremeter:

    image

  4. Then the endpoint on your server will ask npm in which directory that npm module is being installed:

    image

  5. The endpoint will load the file from that directory and pass it to your flow editor, where the plugin is loaded in this node's config screen.

The weird thing is that the message says that @blockly is not installed, while it should say instead something like:

image

bartbutenaers commented 1 year ago

So I am getting the impression that the npm package name is not arriving correctly in your endpoint (because it logs @blockly instead of @blockly/plugin-workspace-search) but I might be mistaken.

I need your help again to get this solved:

  1. Open my .../node-red-contrib-blockly/blockly.js file

  2. Add the following two console log statements in the endpoint code.

    // Make all the static NPM modules resources from this node public available
    RED.httpAdmin.get('/blockly-contrib/npm/:package/*', function(req, res) {
        var requestedFilePath;
    
        console.log("req.url = " + req.url);
        console.log("req.params.package = " + req.params.package);
    
        // Try to get the npm package path from the cache
        var npmPackagePath = blocklyNpmPaths.get(req.params.package);
  3. Make sure Node-RED is restarted. Don't know how you need to do that via HA....

  4. Copy those log statements from your Node-RED log and share them here.

Thanks!! I will have a look at the result tonight...

guenter-ms commented 1 year ago

Done, see below!

23 Oct 16:41:25 - [info] Linux 5.15.61-v8 arm64 LE 23 Oct 16:41:28 - [info] Loading palette nodes 23 Oct 16:41:36 - [info] node-red-contrib-ccu version: 3.6.2 23 Oct 16:41:41 - [info] Dashboard version 3.2.0 started at /endpoint/ui 23 Oct 16:41:44 - [info] Settings file : /etc/node-red/config.js 23 Oct 16:41:44 - [info] Context store : 'default' [module=memory] 23 Oct 16:41:44 - [info] User directory : /config/node-red/ 23 Oct 16:41:44 - [warn] Projects disabled : editorTheme.projects.enabled=false 23 Oct 16:41:44 - [info] Flows file : /config/node-red/flows.json 23 Oct 16:41:45 - [warn] Encrypted credentials not found 23 Oct 16:41:45 - [info] Server now running at http://127.0.0.1:46836/ 23 Oct 16:41:45 - [info] Starting flows [16:41:45] INFO: Starting NGinx... 23 Oct 16:41:46 - [info] [ccu-connection:Raspberrymatic local CCU-Jack] paramsets loaded from /config/node-red/paramsets.json 23 Oct 16:41:46 - [warn] [ccu-connection:Raspberrymatic local CCU-Jack] metadata new empty 23 Oct 16:41:46 - [info] [ccu-connection:Raspberrymatic local CCU-Jack] regadata loaded from /config/node-red/ccu_rega_192.168.178.81.json 23 Oct 16:41:46 - [info] [ccu-connection:Raspberrymatic local CCU-Jack] values loaded from /config/node-red/ccu_values_192.168.178.81.json 23 Oct 16:41:46 - [info] [ccu-connection:Raspberrymatic local CCU-Jack] Interfaces: CCU-Jack 23 Oct 16:41:46 - [info] Started flows 23 Oct 16:41:46 - [info] [ccu-connection:Raspberrymatic local CCU-Jack] init CCU-Jack http://192.168.178.81:2054 nr_5d1db5_CCU-Jack 23 Oct 16:41:46 - [info] [ccu-connection:Raspberrymatic local CCU-Jack] xmlrpc server listening on http://192.168.178.81:2054 23 Oct 16:41:47 - [error] [ccu-connection:Raspberrymatic local CCU-Jack] < CCU-Jack init Error: connect ECONNREFUSED 192.168.178.81:2121 23 Oct 16:41:47 - [error] [ccu-connection:Raspberrymatic local CCU-Jack] init CCU-Jack Error: connect ECONNREFUSED 192.168.178.81:2121 23 Oct 16:41:47 - [info] [ccu-connection:Raspberrymatic local CCU-Jack] Interface CCU-Jack disconnected 23 Oct 16:41:50 - [info] [server:Home Assistant] Connecting to http://supervisor/core 23 Oct 16:41:50 - [info] [server:Home Assistant] Connected to http://supervisor/core req.url = /blockly-contrib/npm/blockly/blockly_compressed.js req.params.package = blockly req.url = /blockly-contrib/npm/blockly/msg/en.js req.params.package = blockly req.url = /blockly-contrib/npm/blockly/blocks_compressed.js req.params.package = blockly req.url = /blockly-contrib/npm/blockly/javascript_compressed.js req.params.package = blockly req.url = /blockly-contrib/npm/@blockly/plugin-workspace-search/dist/index.js req.params.package = @blockly The npm package @blockly is not installed (but required via the Blockly config node) req.url = /blockly-contrib/npm/@blockly/zoom-to-fit/dist/index.js req.params.package = @blockly The npm package @blockly is not installed (but required via the Blockly config node) req.url = /blockly-contrib/npm/@blockly/workspace-backpack/dist/index.js req.params.package = @blockly The npm package @blockly is not installed (but required via the Blockly config node) req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/messages/en.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/@blockly/field-date/dist/date_compressed.js req.params.package = @blockly The npm package @blockly is not installed (but required via the Blockly config node) req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml req.params.package = node-red-contrib-blockly req.url = /blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml req.params.package = node-red-contrib-blockly

guenter-ms commented 1 year ago

Whats mysterious:

@blockly is bold in opposite to blocky. See screenshot.

Bildschirmfoto 2022-10-23 um 15 48 04

bartbutenaers commented 1 year ago

I will add the same console statements tonight in my setup, and then we can compare and figure out a fix.

bartbutenaers commented 1 year ago

I think I found it...

As you can see here, all the url's for the Blockly plugins contain an encoded forward slash (i.e. %2F)

"blockly-contrib/npm/@blockly%2Fplugin-workspace-search/dist/index.js",

The ExpressJs webserver ignores the encoded slash for routing, so normally the req.params.package is filled with the value "@blockly%2Fplugin-workspace-search".

However it seems that the NGinx reverse proxy (in the Home Assistant setup) has been configured to automatically decode these forward slashes. Which means that the decoded forward slash arrives at the Node-RED ExpressJs webserver, which will do its routing now differently (due to the extra forward slash). As a result the req.params.package will be filled with the value "@blockly" only.

Since I don't have time to rewrite a part of my frontend code, I would like to try this workaround. Using this workaround it still works fine in my setup.

  1. Can you change the url's of the four plugins in the blockly.html file to this:

    "blockly-contrib/npm/@blockly%252Fplugin-workspace-search/dist/index.js",
    "blockly-contrib/npm/@blockly%252Fzoom-to-fit/dist/index.js",
    "blockly-contrib/npm/@blockly%252Fworkspace-backpack/dist/index.js",
    "blockly-contrib/npm/@blockly%252Fworkspace-backpack/dist/index.js"

    So we replace the encoded forward slash %2F by %252F, which means we simply escape it. As a result your Nginx will ignore the escaped (encoded) forward slash, so the routing of your Express.js webserver should be unchanged.

  2. Can you add the unescape line to the endpoint code in the blockly.js file:

    RED.httpAdmin.get('/blockly-contrib/npm/:package/*', function(req, res) {
        var requestedFilePath;
    
        req.params.package = require("querystring").unescape(req.params.package);
    
        // Try to get the npm package path from the cache
        var npmPackagePath = blocklyNpmPaths.get(req.params.package);

    So now we unescape the req.params.package value, which means we get the original encoded slash again.

  3. Restart you Node-RED to activate the updated endpoint code.

  4. Refresh your browser window so your flow editor code will be reloaded to use the updated url's.

Fingers crossed this works in your case...

bartbutenaers commented 1 year ago

@guenter-ms, I assume there is something not correct in the HA NGinx config. But I am not really familiar with NGinx, so I cannot give you any advice. In this Stackoverflow discussion, you will see that they advice to use proxy_pass ìn the NGinx config. If you want the HA team to change something about that, be my guest. But I have no time to digg into this further.

guenter-ms commented 1 year ago

Thank you,

unfortunately it will not work...

I changed your advise #1:

Bildschirmfoto 2022-10-24 um 11 37 40

and then #2:

Bildschirmfoto 2022-10-24 um 11 38 36

and got this result:

Bildschirmfoto 2022-10-24 um 11 35 53

What can I do?

bartbutenaers commented 1 year ago

Damn...

Can you show the 'before' and 'after' log lines for the "plugin-workspace-search" please? I have no clue anymore which values Express.js parses in your setup ;-(

Because currently you show the date_compressed.js log lines, but we haven't changed the url for that file yet. As soon as the plugins can be loaded without problem, it indeed seems that need to be fixed the date_compressed url also for Home Assistant...

bartbutenaers commented 1 year ago

Because currently you show the date_compressed.js log lines, but we haven't changed the url for that file yet. As soon as the plugins can be loaded without problem, it indeed seems that need to be fixed the date_compressed url also for Home Assistant...

To do that we also need to change the url in the blockly_config.html file to this:

"blockly-contrib/npm/@blockly%252Ffield-date/dist/date_compressed.js",

Then I think we have updated all url's that your Nginx corrupts. But now we only need to know why your Express.js still doesn't parse the %252F incorrectly. Hopefully your log reveals enough information. Otherwise we need to add some extra console.log statements...

guenter-ms commented 1 year ago

Hi, have changed the blocky_config.html, see below Bildschirmfoto 2022-10-24 um 17 33 08 Bildschirmfoto 2022-10-24 um 17 33 48

the plugin-workspace-search is NOT in the debug window or console...

bartbutenaers commented 1 year ago

Ok, we are both running out of time. Let's try a workaround...

Suppose we replace all the %252F by ___SEPARATOR___ in the client side. For example (and also for the date_compressed url):

"blockly-contrib/npm/@blockly___SEPARATOR___plugin-workspace-search/dist/index.js",
"blockly-contrib/npm/@blockly___SEPARATOR___zoom-to-fit/dist/index.js",
"blockly-contrib/npm/@blockly___SEPARATOR___workspace-backpack/dist/index.js",
"blockly-contrib/npm/@blockly___SEPARATOR___workspace-backpack/dist/index.js"

And in the endpoint we replace this again by the path separator of your operating system:

    RED.httpAdmin.get('/blockly-contrib/npm/:package/*', function(req, res) {
        var requestedFilePath;

        req.params.package = req.params.package.replace("___SEPARATOR___", path.sep);

        // Try to get the npm package path from the cache
        var npmPackagePath = blocklyNpmPaths.get(req.params.package);

Does this work for you?

guenter-ms commented 1 year ago

YEAH! That works!

Can you implement?

Best from Muenster, Germany

Bildschirmfoto 2022-10-25 um 11 10 46

bartbutenaers commented 1 year ago

I thought you were never going to say those words ;-)

Version 2.0.1 is now available in the Node-RED palette:

image

Could you please install it and close this issue if it works fine!

P.S. make sure to restart Node-RED and refresh your browser window, so we are sure that you are not running your manually installed fix...

guenter-ms commented 1 year ago

Works fine! Good job! Take care!

bartbutenaers commented 1 year ago

Glad I could help. Enjoy it!