axa-group / nlp.js

An NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more
MIT License
6.23k stars 617 forks source link

I can't save model to a file #766

Closed GuillermoPena closed 3 years ago

GuillermoPena commented 3 years ago

Hi Jesus. I can't save model to a file:

await nlp.train()
nlp.save()

I receive this error:

(node:23551) UnhandledPromiseRejectionWarning: Error: File cannot be written in web
    at /home/guiller/mindsaic/nlpjs/node_modules/@nlpjs/core/src/mock-fs.js:32:12
    at new Promise (<anonymous>)
    at Object.writeFile (/home/guiller/mindsaic/nlpjs/node_modules/@nlpjs/core/src/mock-fs.js:31:10)
    at Nlp.save (/home/guiller/mindsaic/nlpjs/node_modules/@nlpjs/nlp/src/nlp.js:714:14)
    at Nlp.train (/home/guiller/mindsaic/nlpjs/node_modules/@nlpjs/nlp/src/nlp.js:475:18)

Old similar issue 353

Versions: Package versions: "@nlpjs/basic": "^4.17.5", "@nlpjs/lang-all": "^4.17.1", "@nlpjs/lang-es": "^4.17.0", "@nlpjs/language": "^4.17.0" Node version: v14.15.0

GuillermoPena commented 3 years ago

Some simple code to reproduce issue:

const { containerBootstrap } = require('@nlpjs/core');
const { Nlp } = require('@nlpjs/nlp');

const language = 'Es'
const Language = require('@nlpjs/lang-' + language.toLowerCase())

const go = async () => {
  const container = await containerBootstrap();
  container.use(Nlp);
  container.use(Language['Lang' + language])
  const nlp = container.get('nlp')

  nlp.nluManager.settings.log = false
  nlp.settings.forceNER = true
  nlp.settings.autoSave = false
  nlp.settings.calculateSentiment = false
  nlp.addLanguage('es')

  // Adds the utterances and intents for the NLP
  nlp.addDocument('es', 'adios por ahora', 'greetings.bye');
  nlp.addDocument('es', 'adios y ten cuidado', 'greetings.bye');
  nlp.addDocument('es', 'muy bien nos vemos luego', 'greetings.bye');
  nlp.addDocument('es', 'debo irme', 'greetings.bye');
  nlp.addDocument('es', 'hola', 'greetings.hello');

  await nlp.train();

  const text = 'me tengo que ir'
  const response = await nlp.process( language.toLowerCase(), text)
  console.log('Text to process: "' + text + '"')
  console.log('Intent: ' + response.intent + ' - Score: ' + response.score)
}

go()

this code works but, if I chage nlp.settings.autoSave property, fails

jesus-seijas-sp commented 3 years ago

You are calling containerBootstrap of the @nlpjs/core. This is the basic container, that is used for web, so it does not register the plugin for using the filesystem, because in a web you don't have a filesystem.

First suggestion: use the containerBootstrap of the library @nlpjs/core-loader. This library is similar to @nlpjs/core, but when you know that you target node.js and not javascript in the browser.

Second suggestion: follow the quickstart, the quickstart shows how to use @nlpjs/basic, that contains the basic plugins for backend. https://github.com/axa-group/nlp.js/blob/master/docs/v4/quickstart.md

Third suggestion: if you don't want to use the basic library because you want to focus AI in the browser, then you cannot save, because there is no filesystem in the browser. Perhaps you can implement your own system to store in localStorage.

GuillermoPena commented 3 years ago

ouch! It was really evident... Sorry! Thanks Jesus.

Macilias commented 2 years ago

@jesus-seijas-sp actually, your second suggestion does not work for react-native, since the dock is not available there:

error: Error: Unable to resolve module readline from /Users/mac/Programming/fak10/packages/app/node_modules/@nlpjs/console-connector/src/console-connector.js: readline could not be found within the project or in these directories:
  node_modules
  ../../node_modules

Any advices how to train and save the model in react-native?

Thx.