dachev / node-cld

Language detection for Javascript (Node). Based on the CLD2 (Compact Language Detector) library from Google.
Apache License 2.0
316 stars 55 forks source link

support the bestEffort flag #84

Closed slackhappy closed 3 months ago

slackhappy commented 3 months ago

Support CLD2::kCLDFlagBestEffort for shorter text

Helps with issues like #78 for example:

without bestEffort

> console.log((await cld.detect("J'aime me beurrer la biscote")))
Uncaught Error: Failed to identify language
    at Object.detect (/Users/jgallagher/extsrc/slackhappy.gh/node-cld/index.js:76:15)
    at async REPL6:1:46

with bestEffort:

> console.log((await cld.detect("J'aime me beurrer la biscote", {bestEffort:true})))
{
  reliable: true,
  textBytes: 30,
  languages: [ { name: 'FRENCH', code: 'fr', percent: 96, score: 176 } ],
  chunks: []
}

Or #31

> console.log((await cld.detect("beija-flor: laíla defende maior número de parcerias e critica altos gastos")))
Uncaught Error: Failed to identify language
    at Object.detect (/Users/jgallagher/extsrc/slackhappy.gh/node-cld/index.js:76:15)
    at async REPL9:1:46

with bestEffort:

> console.log((await cld.detect("beija-flor: laíla defende maior número de parcerias e critica altos gastos", {bestEffort:true})))
{
  reliable: true,
  textBytes: 77,
  languages: [ { name: 'PORTUGUESE', code: 'pt', percent: 98, score: 215 } ],
  chunks: []
}

33

> console.log(await cld.detect('Hi there'))
Uncaught Error: Failed to identify language
    at Object.detect (/Users/jgallagher/extsrc/slackhappy.gh/node-cld/index.js:76:15)
    at async REPL3:1:45
> 

with bestEffort:

> console.log(await cld.detect('Hi there', {bestEffort:true}))
{
  reliable: true,
  textBytes: 10,
  languages: [ { name: 'ENGLISH', code: 'en', percent: 90, score: 796 } ],
  chunks: []
}
dachev commented 3 months ago

@slackhappy thank you!

slackhappy commented 3 months ago

oops sorry - one more update to the typescript defs and the docs

slackhappy commented 3 months ago

@dachev, ready for re-review, I added documentation and updated the ts defs. Thanks so much for considering this PR!

dachev commented 3 months ago

@slackhappy thanks for the contribution. I pushed the changes to npm: https://www.npmjs.com/package/cld?activeTab=versions

slackhappy commented 3 months ago

@dachev, thank you so much!