Issue using node-red flow with IBM Conversation service

[dennisnotojr]

converstation_service_problem_with_nodered.docx

When preparing a message for the IBM Watson conversation service, you can force a new conversation_id by omitting the context object. After discussing with IBM Watson Bluemix dev, they say their service is stateless and they always pass back a new conversation_id. I can only resolve the issue by restarting the flow. So either node-red is holding the service open by design or the wrapper for the service has a caching issue. Test via rest web services does not cause this problem. A debug description is attached as well as the flow json for review. flowjson.docx

[dennisnotojr]

Sorry, the flowjson is actually the IBM Watson Conversation dialog flow. See below for the node-red flow.

[{"id":"c7d826d6.3827d8","type":"http in","z":"12a890ec.ed576f","name":"Mobile HTTP in","url":"/mobileV2-1","method":"post","swaggerDoc":"","x":133.3333282470703,"y":121.1111068725586,"wires":[["6d404baf.92bfb4"]]},{"id":"5873982c.a78c68","type":"function","z":"12a890ec.ed576f","name":"Watson Response Msg","func":"\n\nvar Watson_msg = msg.payload;\n\nvar Watson_response = JSON.stringify(msg.payload.output.text);\nvar Watson_context = JSON.stringify(msg.payload.context);\nnode.log(Watson_response)\n//global.set(\"Watson_mobile_context\", msg.payload.context);\n\nMyresponse = Watson_response.substring(2,Watson_response.length-2);\nif (Myresponse.match(\"\",\"\")) {\n Myresponse = Myresponse.substring(3,Myresponse.length)\n node.log(\"Found a comma\")\n node.log(Myresponse)\n}\n \n\nvar response = {\n // text: Watson_response.substring(2,Watson_response.length-2),\n text: Myresponse,\n username: \"Watson\",\n context: Watson_context\n};\n\n//var Watson_gvar = global.get(\"Watson_mobile_context\"); \nnode.log(JSON.stringify(response));\n\nvar newMsg = {payload: response};\nnewMsg.req = msg.req;\nnewMsg.res = msg.res;\n\nreturn newMsg;","outputs":1,"noerr":0,"x":678.3333282470703,"y":228.1111068725586,"wires":[["653ce262.9ac31c"]]},{"id":"223cc0f3.ddc34","type":"watson-conversation-v1","z":"12a890ec.ed576f","name":"Conversation_Starter","workspaceid":"71fb6c7d-730c-45ea-94e5-e7704b0eb4ca","x":668.3333282470703,"y":121.1111068725586,"wires":[["5873982c.a78c68"]]},{"id":"6d404baf.92bfb4","type":"function","z":"12a890ec.ed576f","name":"Make String for Conversation","func":"var temp_msg = msg.payload.input;\nvar workspace_id = msg.payload.workspace_id;\nvar fname = msg.payload.fname;\nvar lname = msg.payload.lname;\nvar nname = msg.payload.nname;\nvar cvalue1 = msg.payload.cvalue1;\nvar cvalue2 = msg.payload.cvalue2;\nvar cvalue3 = msg.payload.cvalue3;\n\nvar test = JSON.stringify(msg.payload.context);\nnode.log(test)\nnode.log(test.length)\nif (test.length > 5) {\n var context = JSON.parse(msg.payload.context);\n //adding outbound data from iphone contact\n context.fname = fname\n context.lname = lname\n context.nname = nname\n context.cvalue1 = cvalue1\n context.cvalue2 = cvalue2\n context.cvalue3 = cvalue3 }\n \n//else { var context = msg.payload.context} \n\n//node.log(JSON.stringify(msg.payload));\nmsg.payload = temp_msg;\n\n//var Watson_gvar = global.get(\"Watson_mobile_context\");\n//Conversation_Starter.msg.params.context = Watson_gvar;\n\nnode.log(workspace_id);\n\nif (test.length > 5) {\n var params = {\n context: context,\n workspace_id : workspace_id};}\nelse {\n \n var params = {\n workspace_id : workspace_id }\n}\n\n\n\n//return msg;\n\nreturn {\n payload: temp_msg,\n params : params,\n req : msg.req,\n res : msg.res\n};\n","outputs":1,"noerr":0,"x":377.3333282470703,"y":121.1111068725586,"wires":[["223cc0f3.ddc34"]]},{"id":"653ce262.9ac31c","type":"http response","z":"12a890ec.ed576f","name":"Http Response","x":904.8333282470703,"y":229.1111068725586,"wires":[]}]

[dennisnotojr]

Also, the debug analysis was directed towards IBM development. which they responded with Dennis,

IBM Dev ................ Sorry to be difficult, but I cannot reproduce this behavior using simple API calls. The logic is very clear in the API (if you are familiar with NodeJS!):

if (! _.has(wip, 'message.context.conversationid') ) { .set(wip, 'message.context.conversation_id', uuid.v4()); }

i.e. it does what you are asking - if its not set - create a new one. Additionally, the Conversational API (and Dialog fwiw) is completely stateless - we have no caching/etc/etc so its just not possible for us to "remember" a previous conversation. All of this information has to be provided by the client.

The only variable here is Node-Red....

It would be extremely helpful if you could set up a packet capture on your outgoing network traffic. I would imagine Node-Red is doing something "smart" around query caching - its seeing you send an identical request so its not physically making a new call to the Conversational API and returning the previous response...(and notice everything works when you restart Node-Red - which would have the effect of clearing any in-memory request cache).

This could also maybe be validated by you adding a "dynamic" component to your request payload (and/or request headers) - perhaps add a "timestamp" attribute that defaults to the current ms since epoch. However, the network traffic capture would be a more clear cut definitive answer on what is going on.

Andy

[dennisnotojr]

Really important issue. If it is a matter of caching as IBM dev is suggesting, what should I do? If it's the service wrapper for node-red, who manages that? I extensively use node-red for my demos and I meet with CIO's at fortune 500 companies every week talking up your product. I'm an old ESB guy, Talent, Mule, Tibco, etc. Love what you have done!

[chughts]

I am unable to import your flow which implies there are errors in it. So I am going to base my answers on similarly related questions I have seen, assuming that you are falling into the same error.

Take a look at your flow - end to end. In your flow msg.params.context gets set, where does it get reset or initialised to an empty value? If it is not reset, or emptied out then it will retain its value. This is basic node-RED.

I will also be closing this issue, as this is not the correct forum to raise questions like this. Raise this question on DeveloperWorks - https://developer.ibm.com/answers/topics/watson/ and I, and every other node-RED expert will respond and give you as much advise as you need. First though you need to clean up your flow, so that when you post in your question, it can be imported.

[dennisnotojr]

First, the flow was exported from a working flow. All I did was highlight all the nodes and export. Not possible to have errors because it been running months error free. If your export function has a problem, I can fix that. I do know that you must set the password in the convo node because it does not copy from the export.

Second. I initialize a new message every time to conversation and I have verified that I don't send the context block on new conversations. I work for IBM and I'm working with Watson developers at this moment. I fail to see how to post it to yet another forum. I wanted to work with a node red developer. Who runs the node-red team from a management prospective??