Ayuei / DeBEIR

Dense Bi-Encoder Retrieval for Rapid Experimentation
GNU General Public License v3.0
5 stars 0 forks source link

error with hparam tuning example #10

Closed KonradHoeffner closed 1 year ago

KonradHoeffner commented 1 year ago
(venv) hparam_tuning$ python hparam_tuning_from_config.py 
2023-02-28 13:33:52.220491: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-02-28 13:33:52.301716: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-02-28 13:33:52.304477: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2023-02-28 13:33:52.304491: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2023-02-28 13:33:52.826877: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2023-02-28 13:33:52.826932: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2023-02-28 13:33:52.826940: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Traceback (most recent call last):
  File "/home/konrad/tmp/debeir/examples/hparam_tuning/hparam_tuning_from_config.py", line 53, in <module>
    hparam_config = HparamConfig.from_json(
  File "/home/konrad/.local/lib/python3.10/site-packages/debeir/training/hparm_tuning/config.py", line 39, in from_json
    return HparamConfig(json.load(open(fp)))
FileNotFoundError: [Errno 2] No such file or directory: './configs/hparam/trec2021_tuning.json'
Ayuei commented 1 year ago

I updated the hyperparameter tuning example and fixed the paths so that the user won't need to go looking for the configuration file https://github.com/Ayuei/DeBEIR/commit/91c921a17e409e2af56c4963b374b9422954bcfc.

KonradHoeffner commented 1 year ago

Can confirm, it now seems to run fine. I cancelled it after a short while because it showed an estimated time of ~ 4:20 hours on my Core i9-12900k without external GPU but it looks like it works. By the way, is it possible to drastically reduce the runtime to something like 5 minutes by simplifying the example?