-
```sh
cd lang-mrj
make distclean
./autogen.sh && ./configure --enable-tokenisers --enable-morpher
make
```
For some odd reason the analyzer introduces and +Ex/... tag, which is something lan…
-
_From [QiaoLian...@gmail.com](https://code.google.com/u/107115759537984949554/) on October 22, 2013 16:44:49_
use datastore to store log info with proper format, instead of tokenising and parsing a l…
-
This issue was created automatically with bugzilla2github
# Bugzilla Bug 2659
Date: 2020-04-20T13:31:57+02:00
From: Trond Trosterud <>
To: Sjur Nørstebø Moshagen <>
Last updated: 2020-0…
-
_From [QiaoLian...@gmail.com](https://code.google.com/u/107115759537984949554/) on October 22, 2013 16:44:49_
use datastore to store log info with proper format, instead of tokenising and parsing a l…
-
### Describe the bug
We are seeing huge CPU and load spikes, which causes the entire application to crash and the api to be unavailable.
Even with load distributed across 2 containers we are…
-
https://realpython.com/nltk-nlp-python/
Learning how to do NLP from this website.
So far I have learnt how to tokenise words and also how to filter out stop_words such as "I" , "am" ," not" , "is…
-
Removing most of the checklist as I'm not using atom, I'm using the grammar file directly.
### Description
The syntax grammar does not correctly tokenise properties with hashes inside them:
`…
-
I went to run `yarn` on the `demo` folder and ran into an issue with the `yarn.lock` file:
```
[mark@mark-pc demo]$ yarn
yarn install v1.22.19
warning package.json: No license field
error An un…
Shard updated
8 months ago
-
We'd like `'` (and possibly others) to tokenise as part of unknown-word-tokens, but not as part of known-word-tokens.
So if the lexicon contains only
```
ja:ja CC
':' PUNCT
```
then `ja'ja` s…
-
The current implementation of the API caller function doesn't include any form of tokenisation. With all the tags and normal punctuation still present in the job description (it _is_ rich text), we ne…