run-llama / llama_index

LlamaIndex is a data framework for your LLM applications
https://docs.llamaindex.ai
MIT License
35.53k stars 5.02k forks source link

[Bug]: "OSError: No such file or directory PY3_tab" after upgrade llama_index and nltk #16142

Open you-healthtap opened 3 hours ago

you-healthtap commented 3 hours ago

Bug Description

I started to see many test cases failing after I switch from llama_index==0.10.40 nltk==3.8.1 to llama_index==0.11.10 nltk==3.9.1 with below error OSError: No such file or directory: '/home/runner/.local/lib/python3.10/site-packages/llama_index/core/_static/nltk_cache/tokenizers/punkt/PY3_tab'

Attached reference log. Not sure how to solve this. Please advise. Thanks.

Version

0.11.10

Steps to Reproduce

pytest tests/unit/

although the tests are from private repo

Relevant Logs/Tracbacks

___________ ERROR collecting tests/unit/webserver/test_web_server.py ___________
tests/unit/webserver/test_web_server.py:59: in <module>
    from webserver.web_server import (
core/llm/webserver/web_server.py:31: in <module>
    from llama_index.core import (
../../../.local/lib/python3.10/site-packages/llama_index/core/__init__.py:10: in <module>
    from llama_index.core.base.response.schema import Response
../../../.local/lib/python3.10/site-packages/llama_index/core/base/response/schema.py:9: in <module>
    from llama_index.core.schema import NodeWithScore
../../../.local/lib/python3.10/site-packages/llama_index/core/schema.py:27: in <module>
    from llama_index.core.utils import SAMPLE_TEXT, truncate_text
../../../.local/lib/python3.10/site-packages/llama_index/core/utils.py:89: in <module>
    globals_helper = GlobalsHelper()
../../../.local/lib/python3.10/site-packages/llama_index/core/utils.py:65: in __init__
    nltk.data.find("tokenizers/punkt_tab")
../../../.local/lib/python3.10/site-packages/nltk/data.py:537: in find
    return FileSystemPathPointer(p)
../../../.local/lib/python3.10/site-packages/nltk/compat.py:41: in _decorator
    return init_func(*args, **kwargs)
../../../.local/lib/python3.10/site-packages/nltk/data.py:312: in __init__
    raise OSError("No such file or directory: %r" % _path)
E   OSError: No such file or directory: '/home/runner/.local/lib/python3.10/site-packages/llama_index/core/_static/nltk_cache/tokenizers/punkt/PY3_tab'
dosubot[bot] commented 3 hours ago

Hello, @you-healthtap! I'm here to help you with your bug. If you need assistance, feel free to ask.

Hey @you-healthtap!

I see you're dealing with some pesky test failures after updating your dependencies. Unfortunately, I can't help with this specific issue. I'm a bot designed to help solve bugs, answer technical questions, and gather engineering knowledge.

If you think I should have tackled this, feel free to reach out to the Dosu Team.

Good luck with your debugging!

To continue talking to Dosu, mention @dosu.