akolesnikov
commented
1 year ago I assume that source install.sh did not return any errors.
What is the output of pip show tensorflow ?
What version of Python do you have on your system?
Closed ap1438 closed 1 year ago
akolesnikov
commented
1 year ago I assume that source install.sh did not return any errors.
What is the output of pip show tensorflow ?
What version of Python do you have on your system?
ap1438
commented
1 year ago Sorry for the late reply . I was not feeling well.
This time i tried in HPC cluster
(deepcon) aparida@grover:/prj/pflaphy-pacbio/Softwares/deepcon$ /homes/aparida/.local/bin/deepconsensus run
2023-01-19 11:01:05.720575: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/thinlinc/lib64:/opt/thinlinc/lib
2023-01-19 11:01:05.720609: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2023-01-19 11:01:09.927521: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/thinlinc/lib64:/opt/thinlinc/lib
2023-01-19 11:01:09.927567: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2023-01-19 11:01:09.927616: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (grover): /proc/driver/nvidia/version does not exist
Traceback (most recent call last):
File "/homes/aparida/.local/bin/deepconsensus", line 8, in
There are 2 python versions Python 3.8.10 (python -V) and Python 2.7.18 (python -V)
(deepcon) aparida@grover:/prj/pflaphy-pacbio/Softwares/deepcon$ pip show tensorflow /homes/aparida/.local/lib/python3.8/site-packages/pkg_resources/init.py:123: PkgResourcesDeprecationWarning: 0.1.36ubuntu1 is an invalid version and will not be supported in a future release warnings.warn( /homes/aparida/.local/lib/python3.8/site-packages/pkg_resources/init.py:123: PkgResourcesDeprecationWarning: 0.23ubuntu1 is an invalid version and will not be supported in a future release warnings.warn( /homes/aparida/.local/lib/python3.8/site-packages/pkg_resources/init.py:123: PkgResourcesDeprecationWarning: 1.13.1-unknown is an invalid version and will not be supported in a future release warnings.warn( Name: tensorflow Version: 2.11.0 Summary: TensorFlow is an open source machine learning framework for everyone. Home-page: https://www.tensorflow.org/ Author: Google Inc. Author-email: packages@tensorflow.org License: Apache 2.0 Location: /homes/aparida/.local/lib/python3.8/site-packages Requires: absl-py, astunparse, flatbuffers, gast, google-pasta, grpcio, h5py, keras, libclang, numpy, opt-einsum, packaging, protobuf, setuptools, six, tensorboard, tensorflow-estimator, termcolor, typing-extensions, wrapt, tensorflow-io-gcs-filesystem Required-by: tf-models-official, tensorflow-text
The machine i am using is a HPC cluster
I ran commands
git clone https://github.com/google/deepconsensus.git cd deepconsensus source install.sh ./run_all_tests.sh
deepcon/deepconsensus$ ./run_all_tests.sh 2023-01-12 16:45:20.924360: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/thinlinc/lib64:/opt/thinlinc/lib 2023-01-12 16:45:20.924408: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/prj/pflaphy-pacbio/Softwares/deepcon/deepconsensus/deepconsensus/inference/quick_inference_test.py", line 36, in
from deepconsensus.inference import quick_inference
File "/prj/pflaphy-pacbio/Softwares/deepcon/deepconsensus/deepconsensus/inference/quick_inference.py", line 60, in
from deepconsensus.models import model_utils
File "/prj/pflaphy-pacbio/Softwares/deepcon/deepconsensus/deepconsensus/models/model_utils.py", line 41, in
from deepconsensus.models import losses_and_metrics
File "/prj/pflaphy-pacbio/Softwares/deepcon/deepconsensus/deepconsensus/models/losses_and_metrics.py", line 37, in
class PerExampleAccuracy(tf.keras.metrics.Accuracy):
File "/homes/aparida/.local/lib/python3.8/site-packages/tensorflow/python/util/lazy_loader.py", line 62, in getattr
module = self._load()
File "/homes/aparida/.local/lib/python3.8/site-packages/tensorflow/python/util/lazy_loader.py", line 45, in _load
module = importlib.import_module(self.name)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/homes/aparida/.local/lib/python3.8/site-packages/keras/init.py", line 21, in
from keras import models
File "/homes/aparida/.local/lib/python3.8/site-packages/keras/models/init.py", line 18, in
from keras.engine.functional import Functional
File "/homes/aparida/.local/lib/python3.8/site-packages/keras/engine/functional.py", line 26, in
from keras import backend
File "/homes/aparida/.local/lib/python3.8/site-packages/keras/backend.py", line 34, in
from keras.engine import keras_tensor
File "/homes/aparida/.local/lib/python3.8/site-packages/keras/engine/keras_tensor.py", line 19, in
from keras.utils import object_identity
File "/homes/aparida/.local/lib/python3.8/site-packages/keras/utils/init.py", line 48, in
from keras.utils.layer_utils import get_source_inputs
File "/homes/aparida/.local/lib/python3.8/site-packages/keras/utils/layer_utils.py", line 26, in
from keras import initializers
File "/homes/aparida/.local/lib/python3.8/site-packages/keras/initializers/init.py", line 22, in
from keras.initializers import initializers_v2
File "/homes/aparida/.local/lib/python3.8/site-packages/keras/initializers/initializers_v2.py", line 23, in
from keras.dtensor import utils
File "/homes/aparida/.local/lib/python3.8/site-packages/keras/dtensor/init.py", line 22, in
from tensorflow.compat.v2.experimental import dtensor as dtensor_api
File "/homes/aparida/.local/lib/python3.8/site-packages/tensorflow/_api/v2/compat/v2/experimental/dtensor/init.py", line 8, in
from tensorflow.dtensor.python.accelerator_util import initialize_accelerator_system
File "/homes/aparida/.local/lib/python3.8/site-packages/tensorflow/dtensor/python/accelerator_util.py", line 24, in
from tensorflow.dtensor.python import api
File "/homes/aparida/.local/lib/python3.8/site-packages/tensorflow/dtensor/python/api.py", line 22, in
from tensorflow.dtensor.python import dtensor_device
File "/homes/aparida/.local/lib/python3.8/site-packages/tensorflow/dtensor/python/dtensor_device.py", line 27, in
from tensorflow.dtensor.python import layout as layout_lib
File "/homes/aparida/.local/lib/python3.8/site-packages/tensorflow/dtensor/python/layout.py", line 24, in
from tensorflow.python import _pywrap_dtensor_device
ImportError: /homes/aparida/.local/lib/python3.8/site-packages/tensorflow/python/_pywrap_dtensor_device.so: undefined symbol: _ZN10tensorflow7dtensor4Mesh10CreateMeshERKSsRKSt6vectorISsSaISsEERKS4_IlSaIlEESC_S8_SCS8, version tensorflow
Can you let me know whats the problem.