Get a AttributeError: torch._inductor.config.fx_graph_cache does not exist ERROR

tuobay commented 4 months ago

System Info

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| +-----------------------------------------------------------------------------+

SORFTWARE: PyTorch version 2.1.2+cu118+infinity_emb0.0.52

❯ infinity_emb v2 --help
INFO     2024-07-24 09:53:12,021 datasets INFO: PyTorch version 2.1.2+cu118 available.                                                                                                       config.py:58
Traceback (most recent call last):
  File "/home/tuo/conda/envs/minicpmv/bin/infinity_emb", line 5, in <module>
    from infinity_emb.infinity_server import cli
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/__init__.py", line 20, in <module>
    from infinity_emb.engine import AsyncEmbeddingEngine, AsyncEngineArray  # noqa: E402
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/engine.py", line 7, in <module>
    from infinity_emb.inference import (
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/inference/__init__.py", line 1, in <module>
    from infinity_emb.inference.batch_handler import BatchHandler
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/inference/batch_handler.py", line 30, in <module>
    from infinity_emb.transformer.utils import get_lengths_with_tokenize
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/transformer/utils.py", line 10, in <module>
    from infinity_emb.transformer.embedder.ct2 import CT2SentenceTransformer
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/transformer/embedder/ct2.py", line 18, in <module>
    from infinity_emb.transformer.embedder.sentence_transformer import (
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/transformer/embedder/sentence_transformer.py", line 41, in <module>
    torch._inductor.config.fx_graph_cache = True
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/torch/_dynamo/config_utils.py", line 72, in __setattr__
    raise AttributeError(f"{self.__name__}.{name} does not exist")
AttributeError: torch._inductor.config.fx_graph_cache does not exist
❯ infinity_emb --help
INFO     2024-07-24 10:00:05,828 datasets INFO: PyTorch version 2.1.2+cu118 available.                                                                                                       config.py:58
Traceback (most recent call last):
  File "/home/togodpypy/conda/envs/minicpmv/bin/infinity_emb", line 5, in <module>
    from infinity_emb.infinity_server import cli
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/__init__.py", line 20, in <module>
    from infinity_emb.engine import AsyncEmbeddingEngine, AsyncEngineArray  # noqa: E402
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/engine.py", line 7, in <module>
    from infinity_emb.inference import (
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/inference/__init__.py", line 1, in <module>
    from infinity_emb.inference.batch_handler import BatchHandler
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/inference/batch_handler.py", line 30, in <module>
    from infinity_emb.transformer.utils import get_lengths_with_tokenize
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/transformer/utils.py", line 10, in <module>
    from infinity_emb.transformer.embedder.ct2 import CT2SentenceTransformer
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/transformer/embedder/ct2.py", line 18, in <module>
    from infinity_emb.transformer.embedder.sentence_transformer import (
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/infinity_emb/transformer/embedder/sentence_transformer.py", line 41, in <module>
    torch._inductor.config.fx_graph_cache = True
  File "/home/togodpypy/conda/envs/minicpmv/lib/python3.10/site-packages/torch/_dynamo/config_utils.py", line 72, in __setattr__
    raise AttributeError(f"{self.__name__}.{name} does not exist")
AttributeError: torch._inductor.config.fx_graph_cache does not exist

Information

[ ] Docker
[X] The CLI directly via pip

Tasks

[X] An officially supported command
[ ] My own modifications

Reproduction

Just type infinity_emb v2 --help can get the error

Expected behavior

Just show the help command output

greptile-apps[bot] commented 4 months ago

To resolve the AttributeError: torch._inductor.config.fx_graph_cache does not exist error, follow these steps:

Check PyTorch Version Compatibility: Ensure that the PyTorch version you are using is compatible with the infinity_emb library. The error might be due to an incompatibility between the PyTorch version and the library.
Update infinity_emb Library: Make sure you are using the latest version of the infinity_emb library. Update it if necessary.

Modify test_infinity_server.py: Update the test cases in /libs/infinity_emb/tests/unit_test/test_infinity_server.py to ensure they are compatible with the current PyTorch version.

@pytest.mark.skipif(sys.platform == "win32", reason="does not run on windows")
def test_cli_v2_help():
   log = subprocess.run(["infinity_emb", "v2", "--help"])
   assert log.returncode == 0

Check for Deprecated Attributes: Verify if torch._inductor.config.fx_graph_cache has been deprecated or moved in the latest PyTorch version. If so, update the code to use the new attribute or method.
Debugging: Add debugging statements to identify where the attribute is being accessed and ensure it exists in the current PyTorch version.

If the issue persists, consider reaching out to the maintainers of the infinity_emb library for further assistance.

References

/.github/ISSUE_TEMPLATE/bug-report.yml /docs/assets/openapi.json /.github/ISSUE_TEMPLATE/new-model-addition.yml /libs/infinity_emb/tests/unit_test/test_infinity_server.py /libs/infinity_emb /docs/docs/index.md /README.md /.github/ISSUE_TEMPLATE /libs/infinity_emb/tests/unit_test /libs/infinity_emb/Dockerfile /libs/infinity_emb/poetry.toml /docs/docs/deploy.md /libs/infinity_emb/Makefile /libs/infinity_emb/tests/unit_test/fastapi_schemas /libs/infinity_emb/pyproject.toml /libs/infinity_emb/tests/data/benchmark /docs/benchmarks/simple_app.py /libs /libs/infinity_emb/tests/end_to_end/test_optimum_embedding.py /docs /libs/infinity_emb/infinity_emb/fastapi_schemas /docs/assets /libs/infinity_emb/tests /docs/docs

#### About Greptile

This response provides a starting point for your research, not a precise solution. Help us improve! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. [Ask Greptile](https://app.greptile.com/chat/github/michaelfeil/infinity/main) · [Edit Issue Bot Settings](https://app.greptile.com/apps/github)

michaelfeil commented 4 months ago

Try torch>=2.2 . infinity_emb==0.0.52 -> pip should automatically install that, or at least you skipped some example

michaelfeil / infinity