watson-developer-cloud / node-red-node-watson

A collection of nodes for the IBM Watson services
Apache License 2.0
82 stars 86 forks source link

Can't train on Natural Language Classifier #442

Closed tmuraki closed 5 years ago

tmuraki commented 5 years ago

Hi there,

The following error message is returned when I tried to train the Natural Language Classifier.

"Lost connection to server, reconnecting..."

2019-01-21 11 42 35

But I can do the following things with same instance of Natural Language Classifier.

Here is the flow with japanese training data.

[{"id":"75694604.f6e1a8","type":"tab","label":"フロー 1","disabled":false,"info":""},{"id":"471e84a8.80dbfc","type":"inject","z":"75694604.f6e1a8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":160,"wires":[["910f7147.60462"]]},{"id":"910f7147.60462","type":"watson-natural-language-classifier","z":"75694604.f6e1a8","name":"List Classifiers","mode":"list","language":"en","classifier":"","collections-off":true,"default-endpoint":true,"service-endpoint":"https://gateway.watsonplatform.net/natural-language-classifier/api","x":350,"y":160,"wires":[["d6e3d402.0db1b8"]]},{"id":"d6e3d402.0db1b8","type":"debug","z":"75694604.f6e1a8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":670,"y":160,"wires":[]},{"id":"f909fe1e.07422","type":"inject","z":"75694604.f6e1a8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":240,"wires":[["68036ff8.e4523"]]},{"id":"498777d5.f704f8","type":"watson-natural-language-classifier","z":"75694604.f6e1a8","name":"Training","mode":"create","language":"ja","classifier":"","collections-off":true,"default-endpoint":true,"service-endpoint":"https://gateway.watsonplatform.net/natural-language-classifier/api","x":500,"y":240,"wires":[["dda278da.7bb008"]]},{"id":"dda278da.7bb008","type":"debug","z":"75694604.f6e1a8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":670,"y":240,"wires":[]},{"id":"68036ff8.e4523","type":"template","z":"75694604.f6e1a8","name":"Training Data","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"iPhone X予約完了,positive\nサンローラン買えなかったから大容量のiPhone X 256GB予約したよ😇ピンクないし柄にもなくスペースグレーにしようかと思ったけど指紋目立つからやっぱりシルバー😅,positive\niPhoneXはシルバーの256GBで今回はソフトバンクで予約しときました😌,positive\n何みんなiPhone X買おうとしてんだ金あるな あんなん買うならMacBook Pro買うわ,negative\nコレ、ホント楽しいです。あと、まとめサイトの件はちっとも気にしていません…|iPhone Xに勝てた? 海外版Galaxy Note8用 サムスン純正ケース「LED View Cover」が楽しすぎてザンネン……,positive\niPhone X「あいふぉんばつ」と読んじゃって。「あいふぉんえっくす」か、と気づいて訂正して。「あいふぉんてん」だった時の私の気持ち誰かわかる…?,negative\n転売ヤーに激震、iPhone Xが未だに4~5週間後の出荷予定。年内絶望どころか余裕で買える事態に,negative\nもうソフバンでiPhone X予約してしまおうか…,negative\niPhone XのXってwwwwwwwwwwwwwテンって読み方だったのかwwwwwwwwwwwwwwwwwwwwwwww俺ずっとwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwお前の事が好きだった,positive\n[ITmedia News] iPhone Xを安く買った ソフトバンクのMNP引き止めポイントで,positive\niPhone Xをエディオン様で予約したでござる٩( 'ω' )و,positive\n間もなく予約開始! 「iPhone X」購入を迷っている人向け“総復習”,neutral\niPhoneXって7とそんなにスペック変わんなそうだけど、何か大きな違いあるんかな?バッテリー増えてるのはいいな。,positive\niPhoneXほしいのだけどandroidから離れられるかどうか更新時期に調子悪くなるのやめてXperiaちゃん(´;ω;`),negative\niPhoneX予約完了!色はシルバーにしました!,positive\niPhoneX予約完了。,positive\nYup. #iPhoneX is coming next week. (@ Meguro Station in 品川区,neutral\nぜひ最高のiPhoneXからの記念ツイートを…!,positive\n緊急プレゼント企画🎁任天堂 スイッチ マリオオデッセイセット 1名様にプレゼント❤︎応募方法→RTとツイッターフォロー(PUSHにて応募で高当選確率)応募締め切りは 11月5日まで🙏🙏今回はiPhone Xも予定では出すので…,neutral","output":"str","x":330,"y":240,"wires":[["498777d5.f704f8"]]},{"id":"71e0af38.d801e","type":"inject","z":"75694604.f6e1a8","name":"","topic":"","payload":"こんにちは","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":320,"wires":[["6e93c178.8cb55"]]},{"id":"6e93c178.8cb55","type":"watson-natural-language-classifier","z":"75694604.f6e1a8","name":"Classify","mode":"classify","language":"en","classifier":"cdf8a7x488-nlc-2752","collections-off":true,"default-endpoint":true,"service-endpoint":"https://gateway.watsonplatform.net/natural-language-classifier/api","x":330,"y":320,"wires":[["a814b76f.279848"]]},{"id":"a814b76f.279848","type":"debug","z":"75694604.f6e1a8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":670,"y":320,"wires":[]}]

chughts commented 5 years ago

The node so that it doesn't process data, if it is of invalid format like this example. Currently it gets a TypeError: Expected the input argument to be of type Uint8Array or Buffer, got number, but it shouldn't allow the flow to get that far.

The node is expecting input as a CSV stream. The easiest way to do this is to use the file inject node, when running in the cloud and the file input node when running on your own machine.

chughts commented 5 years ago

The node information panel documentation also needs updating.

tatsuru-toda commented 5 years ago

The training method of the NLC node is utterly broken. I got multiple reports from my colleagues, partners and clients on this same issue.

The node so that it doesn't process data, if it is of invalid format like this example.

Do you mean the format of data @tmuraki has shared is invalid? I have created the simplest flow I can think of to recreate the issue. Could you import this and see what happens?

[ { "id": "2ae0e82f.8e49b8", "type": "tab", "label": "Flow 1", "disabled": false, "info": "" }, { "id": "b1bb191b.2c4b18", "type": "watson-natural-language-classifier", "z": "2ae0e82f.8e49b8", "name": "", "mode": "create", "language": "en", "classifier": "", "collections-off": true, "default-endpoint": true, "service-endpoint": "https://gateway.watsonplatform.net/natural-language-classifier/api", "x": 490, "y": 220, "wires": [ [ "2fd66ca0.87dae4" ] ] }, { "id": "35862b47.e65fb4", "type": "template", "z": "2ae0e82f.8e49b8", "name": "", "field": "payload", "fieldType": "msg", "format": "handlebars", "syntax": "mustache", "template": "exampleA1,classA\nexampleA2,classA\nexampleA3,classA\nexampleA4,classA\nexampleA5,classA\nexampleB1,classB\nexampleB2,classB\nexampleB3,classB\nexampleB4,classB\nexampleB5,classB", "output": "str", "x": 280, "y": 220, "wires": [ [ "b1bb191b.2c4b18" ] ] }, { "id": "2fd66ca0.87dae4", "type": "debug", "z": "2ae0e82f.8e49b8", "name": "", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "false", "x": 710, "y": 220, "wires": [] }, { "id": "e690644f.aa66e8", "type": "inject", "z": "2ae0e82f.8e49b8", "name": "", "topic": "", "payload": "", "payloadType": "date", "repeat": "", "crontab": "", "once": false, "onceDelay": 0.1, "x": 120, "y": 220, "wires": [ [ "35862b47.e65fb4" ] ] } ]

chughts commented 5 years ago

You are right, which is why it is recorded as a bug which needs to be fixed. If you supply the css file though a file inject or a http post it will work -

[{"id":"751d25df.1465bc","type":"watson-natural-language-classifier","z":"2ec77256.d67abe","name":"","mode":"create","language":"en","classifier":"","collections-off":true,"default-endpoint":true,"service-endpoint":"https://gateway.watsonplatform.net/natural-language-classifier/api","x":450,"y":220,"wires":[["d4c36997.d069a8"]]},{"id":"d4c36997.d069a8","type":"debug","z":"2ec77256.d67abe","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":710,"y":220,"wires":[]},{"id":"f57dea1e.60d428","type":"fileinject","z":"2ec77256.d67abe","name":"","x":160,"y":220,"wires":[["751d25df.1465bc"]]}]

chughts commented 5 years ago

Fixed in Fixed in 0.7.6 awaiting release