aboSamoor / polyglot

Multilingual text (NLP) processing toolkit
http://polyglot-nlp.com
Other
2.3k stars 337 forks source link

Report Bugs: 'pos2.ja' not found in index but I installed TASK:pos2 and polyglot doesn't have pos2.ja (Windows 10 Bash on Ubuntu) #133

Open I-himawari opened 6 years ago

I-himawari commented 6 years ago

When I do tokenization Japanese, displayed "ValueError: Package 'pos2.ja' not found in index". But, I installed TASK:post and TASK:unipos and pos2.ja package is not found

t = "あ、ちなみにどっちの意味で適齢期外なの?上?下?いやべつに下だったらま ぁいっかなーとかそういうことではなく" tokens = Text(t) for token in tokens.pos_tags: ... print("{0:20s}{1}".format(token[0], token[1])) ... Traceback (most recent call last): File "/usr/lib/python3.5/code.py", line 91, in runcode exec(code, self.locals) File "", line 1, in File "/usr/local/lib/python3.5/dist-packages/polyglot/decorators.py", line 20, in get value = obj.dict[self.func.name] = self.func(obj) File "/usr/local/lib/python3.5/dist-packages/polyglot/text.py", line 155, in pos_tags for word,t in self.pos_tagger.annotate(self.words): File "/usr/local/lib/python3.5/dist-packages/polyglot/decorators.py", line 20, in get value = obj.dict[self.func.name] = self.func(obj) File "/usr/local/lib/python3.5/dist-packages/polyglot/text.py", line 104, in pos_tagger return get_pos_tagger(lang=self.language.code) File "/usr/local/lib/python3.5/dist-packages/polyglot/decorators.py", line 30, in memoizer cache[key] = obj(*args, *kwargs) File "/usr/local/lib/python3.5/dist-packages/polyglot/tag/base.py", line 181, in get_pos_tagger return POSTagger(lang=lang) File "/usr/local/lib/python3.5/dist-packages/polyglot/tag/base.py", line 131, in init super(POSTagger, self).init(lang=lang) File "/usr/local/lib/python3.5/dist-packages/polyglot/tag/base.py", line 40, in init self.predictor = self._load_network() File "/usr/local/lib/python3.5/dist-packages/polyglot/tag/base.py", line 138, in _load_network self.model = load_pos_model(lang=self.lang, version=2) File "/usr/local/lib/python3.5/dist-packages/polyglot/decorators.py", line 30, in memoizer cache[key] = obj(args, **kwargs) File "/usr/local/lib/python3.5/dist-packages/polyglot/load.py", line 115, in load_pos_model p = locate_resource(src_dir, lang) File "/usr/local/lib/python3.5/dist-packages/polyglot/load.py", line 43, in locate_resource if downloader.status(package_id) != downloader.INSTALLED: File "/usr/local/lib/python3.5/dist-packages/polyglot/downloader.py", line 737, in status info = self._info_or_id(info_or_id) File "/usr/local/lib/python3.5/dist-packages/polyglot/downloader.py", line 507, in _info_or_id return self.info(info_or_id) File "/usr/local/lib/python3.5/dist-packages/polyglot/downloader.py", line 933, in info raise ValueError('Package %r not found in index' % id) ValueError: Package 'pos2.ja' not found in index

I tried to install 'pos2.ka' by cli. But not found.

[polyglot_data] Error loading pos2.ja: Package 'pos2.ja' not found in [polyglot_data] index Error installing package. Retry? [n/y/e] y Traceback (most recent call last): File "/usr/local/bin/polyglot", line 11, in sys.exit(main()) File "/usr/local/lib/python3.5/dist-packages/polyglot/main.py", line 280, in main args.func(args) File "/usr/local/lib/python3.5/dist-packages/polyglot/main.py", line 110, in download halt_on_error=args.halt_on_error) File "/usr/local/lib/python3.5/dist-packages/polyglot/downloader.py", line 676, in download if not self.download(msg.package.id, download_dir, AttributeError: 'NoneType' object has no attribute 'id'

Please help.

shyambhu-mukherjee commented 2 years ago

the issue is actually that parts of speech tagging for japanese is not supported by polyglot. The error misleads you to believe that pos2.ja exists and is not downloaded. See this documentation for pos page to see which all language is supported for pos.