Runtime Catalogue issues

[vitormsilva]

Hi @Endebert,

With the last runtime-catalogue upgrade you did, we have some issues, and doubts with what happen when the nodejs try to contact the catalogue..

Issues:

• console.debug and console.warn aren't support by nodejs, could you please remove or change to console.log or console.info; (this happens on RuntimeCatalogue and ReThinkObject)
• when we try to contact the catalogue, all the requests have success, but when the descriptor have the sourcePackage with sourceCode included,. In this case it never finds the descriptor.sourcePackage, and tries to contact /sourcePackage placed in the sourcePackageURL, and is not valid.
• the same happens when the sourcePackageURL is a real url like "sourcePackageURL": "hyperty-catalogue://catalogue.hybroker.rethink.ptinovacao.pt/.well-known/sourcepackage/Runtime_sp", generated by the remote catalogue.. we have the same problem, doesn't work..
• see this issue too: https://github.com/reTHINK-project/dev-runtime-core/issues/120

Can you help me to try figure out what is happening? you can try to test it.. Steps to reproduce:

1. clone the dev-runtime-nodejs repository, and change to branch develop-update
2. clone the or change the branch of dev-hyperty-toolkit to tookit-node
3. on toolkit run npm run start:node
4. on runtime-nodejs run npm run demo

For each repository you need run npm install.. NOTE this only happen on the nodejs, on browser looks work fine..

Could you help us?

[jboulmal]

We confirm that, using develop branch of #service-framework we got an error in console.debug, etc.

[Endebert]

Still analyzing problems, but one of the errors is caused by the nodejs runtime already parsing the result of a http(s) request, which it should not do. The RuntimeCatalogue also tries to parse the result of the request, which causes the error.

JSON parsing in Request.js is not present in the master branch of the node js runtime and the browser runtime. Why was this introduced?

[vitormsilva]

I try to do some tests, but without success.. but i can remove it.. if help you debugging the problem.

[Endebert]

It seems I got it running, at least after starting the node runtime with npm run demo it stops with runtime core exited. Or is this an error?

What I did:

1. added one line to core.js: console.debug = console.log;
2. removed json parsing from Request.js

[vitormsilva]

Yes.. is that error.. You have a get undefined before the catalogue.getRuntimeDescriptor resolve the then step.

[Endebert]

did some more testing: The cause of the error is that when the RuntimeCatalogue requests the sourcePackage, the complete sourcePackage is not returned, only a part of it. This leads to SyntaxError: Unexpected end of JSON input (I added a try-catch block for testing)

The implementation of https.get() is incorrect in Request.js.

[vitormsilva]

Why? Could be more specific?

[Endebert]

This is the relevant part in Request.js (slightly adapted):

let req = https.get(url, (response) => {
    console.log('statusCode:', response.statusCode);
    response.on('data', (data) => {
        let buffer = data.toString('utf8');
        resolve(buffer);
    });
});

req.end();

req.on('error', (e) => {
    console.error('Error:', e);
    reject(e);
});

So you create a buffer object, set it to the incoming data, then immediately resolve the promise with the buffer.

But there is a reason why there is a buffer, Because response.on('data') might be called several times and you have to wait for the request to be completed before you resolve the promise with the buffer.

See this StackOverflow answer on how to handle it properly: http://stackoverflow.com/a/21953201

[vitormsilva]

Ok.. make sense.. i will try. Thank you for the help.

You will push to the service-framework, the changes from console.debug to console.log, or we will use the "hack" you did?

[Endebert]

changed console.debug to console.log, see referenced commit.

[vitormsilva]

Thank you.. i can close this issue..

[vitormsilva]

@Endebert, Sorry for reopen this issue, but we have other problem. Is related with the atob. The method createSourcePacakge(sp) is calling the atob, but need call this.atob because of the implementation of nodejs.. this implementation is passed on the RuntimeCatalogue constructor.

Could you fix that?

[Endebert]

Well, I just pushed it because you asked me to, but in the future please fix such small issues yourself.