search

google-research / perch

Apache License 2.0
188 stars 42 forks source link

installation woes #702

Closed sammlapp closed 2 weeks ago

sammlapp commented 2 weeks ago

When trying to use perch/chirp/projects/agile2 on Ubuntu 20.04.6 LTS:

during imports, from chirp.projects.zoo import models leads to an error regarding GLIBCXX_3.4.30' not found, full trace below. This occurs whether I pip install the chirp git repo, clone the git repo and add to my python path, or run poetry install.

following a seemingly similar issue's solution I tried conda install -c conda-forge gcc=12.1.0, which updates some packages but then I get a new error ImportError: /lib/x86_64-linux-gnu/libc.so.6: versionGLIBC_2.32' not found (required by /home/sml161/miniconda3/envs/perch/lib/python3.10/site-packages/ai_edge_litert/_pywrap_tensorflow_interpreter_wrapper.so)`.

I'm not sure what the GLIBC package is all about, but it seems to be central to the operating system as some call it the "soul" of the os. I can see that I have a prior version: ldd --version gives

ldd (Ubuntu GLIBC 2.31-0ubuntu9.16) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.

Does this mean an OS update is the best way to be compatible with ai_edge_litert?

Full stack trace of original error:

ImportError                               Traceback (most recent call last)
Cell In[2], [line 5](vscode-notebook-cell:?execution_count=2&line=5)
      [2](vscode-notebook-cell:?execution_count=2&line=2) from IPython.display import display
      [3](vscode-notebook-cell:?execution_count=2&line=3) import ipywidgets as widgets
----> [5](vscode-notebook-cell:?execution_count=2&line=5) from chirp.projects.agile2 import colab_utils
      [6](vscode-notebook-cell:?execution_count=2&line=6) from chirp.projects.agile2 import embed
      [7](vscode-notebook-cell:?execution_count=2&line=7) from chirp.projects.agile2 import source_info

File ~/perch/chirp/projects/agile2/colab_utils.py:20
     [16](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/agile2/colab_utils.py:16) """Utility functions for sqlite-backed Agile modeling notebooks."""
     [18](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/agile2/colab_utils.py:18) import dataclasses
---> [20](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/agile2/colab_utils.py:20) from chirp.projects.agile2 import embed
     [21](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/agile2/colab_utils.py:21) from chirp.projects.agile2 import source_info
     [22](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/agile2/colab_utils.py:22) from chirp.projects.hoplite import db_loader

File ~/perch/chirp/projects/agile2/embed.py:26
     [24](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/agile2/embed.py:24) from chirp.projects.agile2 import source_info
     [25](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/agile2/embed.py:25) from chirp.projects.hoplite import interface as hoplite_interface
---> [26](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/agile2/embed.py:26) from chirp.projects.zoo import model_configs
     [27](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/agile2/embed.py:27) from chirp.projects.zoo import zoo_interface
     [28](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/agile2/embed.py:28) from ml_collections import config_dict

File ~/perch/chirp/projects/zoo/model_configs.py:21
     [18](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/model_configs.py:18) import enum
     [20](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/model_configs.py:20) from chirp.models import frontend
---> [21](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/model_configs.py:21) from chirp.projects.zoo import models
     [22](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/model_configs.py:22) from chirp.projects.zoo import taxonomy_model_tf
     [23](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/model_configs.py:23) from chirp.projects.zoo import zoo_interface

File ~/perch/chirp/projects/zoo/models.py:36
     [33](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/models.py:33) import tensorflow.compat.v1 as tf1
     [34](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/models.py:34) import tensorflow_hub as hub
---> [36](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/models.py:36) from ai_edge_litert import interpreter as tfl_interpreter  # pylint: disable=g-direct-tensorflow-import
     [39](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/models.py:39) @dataclasses.dataclass
     [40](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/models.py:40) class SeparateEmbedModel(zoo_interface.EmbeddingModel):
     [41](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/models.py:41)   """Wrapper for separate separation and embedding models.
     [42](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/models.py:42) 
     [43](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/models.py:43)   Note: Use the separation model's sample rate. The embedding model's sample
   (...)
     [51](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/models.py:51)       will have shape [T, C+1, D], with the raw audio embedding on channel 0.
     [52](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/perch/chirp/projects/zoo/models.py:52)   """

File ~/miniconda3/envs/perch/lib/python3.10/site-packages/ai_edge_litert/interpreter.py:33
     [30](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/miniconda3/envs/perch/lib/python3.10/site-packages/ai_edge_litert/interpreter.py:30)   from tensorflow.python.util.tf_export import tf_export as _tf_export
     [31](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/miniconda3/envs/perch/lib/python3.10/site-packages/ai_edge_litert/interpreter.py:31) else:
     [32](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/miniconda3/envs/perch/lib/python3.10/site-packages/ai_edge_litert/interpreter.py:32)   # This file is part of ai_edge_litert package.
---> [33](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/miniconda3/envs/perch/lib/python3.10/site-packages/ai_edge_litert/interpreter.py:33)   from ai_edge_litert import _pywrap_tensorflow_interpreter_wrapper as _interpreter_wrapper
     [34](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/miniconda3/envs/perch/lib/python3.10/site-packages/ai_edge_litert/interpreter.py:34)   from ai_edge_litert import metrics_portable as metrics
     [36](https://vscode-remote+ssh-002dremote-002bsnow.vscode-resource.vscode-cdn.net/home/sml161/eccoXX_hawaii/2_chirp_active_nb/~/miniconda3/envs/perch/lib/python3.10/site-packages/ai_edge_litert/interpreter.py:36)   def _tf_export(*x, **kwargs):

ImportError: /home/sml161/miniconda3/envs/perch/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/sml161/miniconda3/envs/perch/lib/python3.10/site-packages/ai_edge_litert/_pywrap_tensorflow_interpreter_wrapper.so)
sdenton4 commented 2 weeks ago

Hey, Samm!

This is a known issue with the new version of TF Lite - we have rolled back to the older version, so it should be fixed for now. Could you give it a shot and check that things are working for you now?

sammlapp commented 2 weeks ago

I pulled changes from main, ran poetry install, and still get:

----> [1](vscode-notebook-cell:?execution_count=3&line=1) from ai_edge_litert import _pywrap_tensorflow_interpreter_wrapper as _interpreter_wrapper

ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /home/sml161/miniconda3/envs/perch/lib/python3.10/site-packages/ai_edge_litert/_pywrap_tensorflow_interpreter_wrapper.so)

Should I be manually downgrading a package version?

relevant package versions:

Python 3.10.14
tensorflow                            2.16.2
tensorflow-estimator                  2.15.0
tensorflow-hub                        0.16.1
tensorflow-io-gcs-filesystem          0.37.1
nvidia-cudnn-cu12                     8.9.2.26
ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.16) 2.31
sdenton4 commented 2 weeks ago

Could you provide a full stack trace for the ImportError? We rolled back the ai_edge_litert imports yesterday, so will be helpful to know where the error is arising.

sammlapp commented 2 weeks ago

ah I take it back sorry, It's working fine!

(I was only still getting an error because I had added from ai_edge_litert import _pywrap_tensorflow_interpreter_wrapper as _interpreter_wrapper when trying to debug original error)

sdenton4 commented 2 weeks ago

OK! I was really worried there that it had slipped in higher up the toolchain...