koursaros-ai / nboost

NBoost is a scalable, search-api-boosting platform for deploying transformer models to improve the relevance of search results on different platforms (i.e. Elasticsearch)
Apache License 2.0
675 stars 69 forks source link

Tensorflow Drone Build #20

Closed colethienes closed 4 years ago

colethienes commented 4 years ago

Here's the stack from the drone:

WARNING:tensorflow:From /drone/src/nboost/model/bert_model/__init__.py:26: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.
--
97 |  
98 | WARNING:tensorflow:From /drone/src/nboost/model/bert_model/__init__.py:26: The name tf.logging.ERROR is deprecated. Please use tf.compat.v1.logging.ERROR instead.
99 |  
100 | 2019-12-02 18:45:47.464940: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
101 | 2019-12-02 18:45:47.493108: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1996200000 Hz
102 | 2019-12-02 18:45:47.495342: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f036d87ea20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
103 | 2019-12-02 18:45:47.495363: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
104 | 2019-12-02 18:45:47.497434: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
105 | 2019-12-02 18:45:47.497454: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (-1)
106 | 2019-12-02 18:45:47.497473: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (5fd1963b10ef): /proc/driver/nvidia/version does not exist

Not sure if this is the sole issue, or there is another issue. Regardless, the drone is timing out.

pertschuk commented 4 years ago

this is just standard tf1.0 output unfortunately, not related to it stalling out

pertschuk commented 4 years ago

It shows up on passing builds as well:

Screen Shot 2019-12-02 at 11 37 51 AM
colethienes commented 4 years ago

Fixed.