OpenVoiceOS / skill-ovos-wikipedia

wikipedia skill for mycroft
Apache License 2.0
7 stars 8 forks source link

NLTK should be from core, not from a skill? #19

Closed goldyfruit closed 9 months ago

goldyfruit commented 1 year ago

Not sure why this skill is returning this error when it's ovos-core that should handle NLTK.

ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia 2023-09-03 01:30:59.498 - skill-ovos-wikipedia.openvoiceos - ERROR - 
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia **********************************************************************
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   Resource punkt not found.
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   Please use the NLTK Downloader to obtain the resource:
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia 
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   >>> import nltk
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   >>> nltk.download('punkt')
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   For more information see: https://www.nltk.org/data.html
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia 
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   Attempted to load tokenizers/punkt/PY3/english.pickle
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia 
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   Searched in:
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/home/ovos/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/home/ovos/.venv/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/home/ovos/.venv/share/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/home/ovos/.venv/lib/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/usr/share/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/usr/local/share/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/usr/lib/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/usr/local/lib/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - ''
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia **********************************************************************
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia 
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia 2023-09-03 01:30:59.505 - OVOS - ovos_workshop.skills.common_query_skill:__get_cq:138 - ERROR - error matching Do I need an umbrella with skill-ovos-wikipedia.openvoiceos
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia Traceback (most recent call last):
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/ovos_workshop/skills/common_query_skill.py", line 136, in __get_cq
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     result = self.CQS_match_query_phrase(search_phrase)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/skill_ovos_wikipedia/__init__.py", line 201, in CQS_match_query_phrase
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     title, summary = self.ask_the_wiki(phrase)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia                      ^^^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/skill_ovos_wikipedia/__init__.py", line 227, in ask_the_wiki
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     self.image = self.wiki.get_image(query)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/skill_ovos_wikipedia/__init__.py", line 100, in get_image
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     data = self.extract_and_search(query, context)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/skill_ovos_wikipedia/__init__.py", line 55, in extract_and_search
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     query = self.extract_keyword(query, lang)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/skill_ovos_wikipedia/__init__.py", line 39, in extract_keyword
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     return keyword_extractor.extract_subject(query, lang)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/ovos_classifiers/heuristics/keyword_extraction.py", line 58, in extract_subject
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     tagged_tokens = NltkPostag().tag(query)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia                     ^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/ovos_classifiers/heuristics/postag.py", line 149, in tag
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     sentence = nltk.word_tokenize(sentence)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/nltk/tokenize/__init__.py", line 129, in word_tokenize
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     sentences = [text] if preserve_line else sent_tokenize(text, language)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/nltk/tokenize/__init__.py", line 106, in sent_tokenize
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     tokenizer = load(f"tokenizers/punkt/{language}.pickle")
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/nltk/data.py", line 750, in load
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     opened_resource = _open(resource_url)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia                       ^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/nltk/data.py", line 876, in _open
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     return find(path_, path + [""]).open()
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia            ^^^^^^^^^^^^^^^^^^^^^^^^
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   File "/home/ovos/.venv/lib/python3.11/site-packages/nltk/data.py", line 583, in find
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     raise LookupError(resource_not_found)
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia LookupError: 
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia **********************************************************************
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   Resource punkt not found.
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   Please use the NLTK Downloader to obtain the resource:
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia 
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   >>> import nltk
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   >>> nltk.download('punkt')
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   For more information see: https://www.nltk.org/data.html
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia 
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   Attempted to load tokenizers/punkt/PY3/english.pickle
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia 
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia   Searched in:
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/home/ovos/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/home/ovos/.venv/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/home/ovos/.venv/share/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/home/ovos/.venv/lib/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/usr/share/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/usr/local/share/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/usr/lib/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - '/usr/local/lib/nltk_data'
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia     - ''
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia **********************************************************************
ovos-skills-b8954466c-mjkwg ovos-skill-wikipedia 
JarbasAl commented 1 year ago

the skill uses one of the ovos-classifier plugins internally that needs nltk

the plugin itself should try to download any missing nltk resource (and doesnt), for now i suggest you share the nltk data container across containers, at least wiki and ddg skills need it

austin-b commented 9 months ago

I am new to using Docker and am experiencing this same issue with the skill, how does one "share the nltk data container across containers" either with ovos-core or some other fashion? Any direction would be helpful, thank you

austin-b commented 9 months ago

I figured it out after some investigation, here's the gist of it for anyone who may come across this:

add the top-level volume block

volumes:
  ovos_nltk:
    name: ovos_nltk
    driver: local

to docker-compose.skills.yml as well as the line - ovos_nltk:/home/${OVOS_USER}/nltk_data under the volumes tag for ovos_skill_wikipedia in the same file, then fully stop and start the services. This fixed the wiki skill for me

goldyfruit commented 9 months ago

Fixed thanks @austin-b :+1: