server.js: Cannot read property 'length' of undefined

[plaksnor]

TL;DR: After labelling a selected period in Grafana, hastic-server shows an http 400 error: Client sent an HTTP request to an HTTPS server.

Context: Running RPi4 (arm7l architecture) with Hass.io (Home Assistant in docker) with addons Influxdb and Grafana, all in separate docker containers. All secured with SSL by LetsEncrypt. Succesfully managed to build hastic-server. No errors during the tests in the development section on this page. Also succesfully managed to install the hastic-grafana-app in the grafana docker add-on of Hass.io.

I created and tested my Hastic datasource in Grafana, which refers to a HTTP (instead of HTTPS) url, with port 8000. When I add an AnalyticUnit and start labelling data according to this page, the hastic-server (started with npm start) shows an error:

LEARNING started for XAZ11o4lxTl0V7XP
query time range: from Sun Aug 18 2019 16:06:13 GMT+0000 (Coordinated Universal Time) to Sun Aug 18 2019 18:13:37 GMT+0000 (Coordinated Universal Time)
Data kit: fail while request data: Request failed with status code 400 query url: "/api/datasources/proxy/1/query"
Response:         status: 400,         response data: "Client sent an HTTP request to an HTTPS server.\n",         headers: {}

This is my config.json:

{
  "HASTIC_PORT": 8000,
  "HASTIC_API_KEY": "my_grafana_api_key_made_for_hastic",
  "HASTIC_WEBHOOK_URL": "http://localhost:8080",
  "HASTIC_WEBHOOK_TYPE": "application/x-www-form-urlencoded",
  "HASTIC_WEBHOOK_SECRET": "mysecret",
  "GRAFANA_URL": "https://granafa_docker_instance_with_ssl:3000",
  "LEARNING_TIMEOUT": 120
}

I'm aware the webhook part should probably not work, but I'm not even close to get it working at all. Question: does anyone know if hastic-server support SSL, or is there something configured wrong in my setup?

Whole log of hastic-server:

Analytics creating successful, pid: 5765
Start analytics pinger...
Analytics pinger started
2019-08-23 07:10:38,931 [Analytics] [INFO ]  Ok
2019-08-23 07:10:38,931 [Analytics] [INFO ]  Starting services...
2019-08-23 07:10:38,932 [Analytics] [INFO ]  Server...
2019-08-23 07:10:38,935 [Analytics] [INFO ]  Ok
2019-08-23 07:10:38,935 [Analytics] [INFO ]  Data service...
2019-08-23 07:10:38,935 [Analytics] [INFO ]  Ok
2019-08-23 07:10:38,936 [Analytics] [INFO ]  Analytic unit manager...
2019-08-23 07:10:38,936 [Analytics] [DEBUG]  Using selector: EpollSelector
2019-08-23 07:10:38,936 [Analytics] [INFO ]  Ok
Analytics process is running
2019-08-23 07:10:38,938 [Analytics] [INFO ]  Binding to ipc:///tmp/hastic/5752.ipc ...

(node:5752) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
Analytics is up
Can't send notification to http://localhost:8080. Error: connect ECONNREFUSED 127.0.0.1:8080
LEARNING started for XAZ11o4lxTl0V7XP
query time range: from Sun Aug 18 2019 16:06:13 GMT+0000 (Coordinated Universal Time) to Sun Aug 18 2019 18:13:37 GMT+0000 (Coordinated Universal Time)
Data kit: fail while request data: Request failed with status code 400 query url: "/api/datasources/proxy/1/query"
Response:         status: 400,         response data: "Client sent an HTTP request to an HTTPS server.\n",         headers: {}

[plaksnor]

Seems like a wrong config.json was loaded (the one in the docker container, not the one which was mapped to my customized config.json outside the docker container). The one in the docker container has the line:

  "GRAFANA_URL": "http://localhost:3000",

which I now changed for testing purposes to

  "GRAFANA_URL": "https://external_wan_ip_address:3000",

And this seems to work, but leads me into some other issue.

Restarted hastic-server with npm start gives me the following error:

LEARNING started for wM9Yoluht3luTIPY
query time range: from Fri Aug 16 2019 10:12:48 GMT+0000 (Coordinated Universal Time) to Fri Aug 23 2019 10:12:48 GMT+0000 (Coordinated Universal Time)
Error: Cannot read property 'length' of undefined
    at /hastic-server/server/dist/server.js:9:92782
    at Generator.next (<anonymous>)
    at a (/hastic-server/server/dist/server.js:9:87908)

In order to make any sense of it, I used the server-dev.js file instead of server.js, which gives me exact line and pos specifications of the location of the code which is causing the error:

LEARNING started for wM9Yoluht3luTIPY
query time range: from Fri Aug 16 2019 10:18:59 GMT+0000 (Coordinated Universal Time) to Fri Aug 23 2019 10:18:59 GMT+0000 (Coordinated Universal Time)
Error: Cannot read property 'length' of undefined
    at /hastic-server/server/dist/server.js:19337:19
    at Generator.next (<anonymous>)
    at fulfilled (/hastic-server/server/dist/server.js:19041:32)

I think there's an issue with the runLearning function, which is not able to define a 'segments' var:

                    let segments = yield Segment.findMany(id, { labeled: true });
                    if (segments.length === 0) {
                        throw new Error('Need at least 1 labeled segment');
                    }

segments is probably not defined. Any thoughts?

[plaksnor]

I think

        if (res.data === undefined || res.data.results.length < 1) {
            console.log('datasource return empty response, no data');
            return emptyResult;
        }

should be changed to:

        if (res.data === undefined || res.data.results === undefined || res.data.results.length < 1) {
            console.log('datasource return empty response, no data');
            return emptyResult;
        }

which at least causes the code to give back a decent message:

Error: Empty data to detect on
    at /hastic-server/server/dist/server.js:19341:19
    at Generator.next (<anonymous>)
    at fulfilled (/hastic-server/server/dist/server.js:19041:32)

API could not been accessed, but finally got the right grafana_url (it's complicated when using Hass.io) and got another error.

LEARNING started for BRl1jNQ8K104WG51
query time range: from Sun Aug 18 2019 15:20:09 GMT+0000 (Coordinated Universal Time) to Sun Aug 18 2019 17:52:48 GMT+0000 (Coordinated Universal Time)
Error: e is not defined
    at /hastic-server/server/dist/server.js:19339:19
    at Generator.next (<anonymous>)
    at fulfilled (/hastic-server/server/dist/server.js:19041:32)

[plaksnor]

Problem solved when going back from server-dev.js to server.js.

So in short: It works in SSL, just use the right address in order to communicate with the API.

[jonyrock]

@plaksnor thank you a lot! Sorry for being slow.