-
We have seen TF2 Albert pretraining crashes intermittently every 1 out of ~3 runs using latest Horovod training on 8 nodes; the crash happens around 3000 steps
Error message:
```
Loss: 6.436, MLM…
-
Please go to Stack Overflow for help and support:
http://stackoverflow.com/questions/tagged/tensorflow
Also, please understand that many of the models included in this repository are experimenta…
-
I run convert.py to convert albert tensorhub model to TF2.0 model with following commands
```shell
MODEL_DIR=albert-base
SIZE=base
# Converting weights to TF 2.0
python converter.py --tf_hub_…
-
I am doing pre-training from scratch. It seems that training is started as gpu's are being used but nothing is on terminal except this:
```
***** Number of cores used : 4
I0227 09:00:31.841020 14…
-
I have generated pretraining data using [https://github.com/kamalkraj/ALBERT-TF2.0](url)
because this supports training with multi GPU. I am doing this for the Hindi language with 22gb of data. Gener…
-
'''
WARNING:root:bert_config not exists. will load model from huggingface checkpoint.
Traceback (most recent call last):
File "run_weibo_ner_cws.py", line 31, in
train_bert_multitask(proble…
-
-
Running the cola script returns:
```sh
2020-01-15 17:53:21.504699: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:163] no NVIDIA GPU device is present: /dev/nvidia0 does not exist
2020-01-…
-
提问时请尽可能提供如下信息:
### 基本信息
- 你使用的**操作系统**: win10
- 你使用的**Python**版本: 3.6
- 你使用的**Tensorflow**版本: 2.0.1
- 你使用的**Keras**版本: 2.3.1
- 你使用的**bert4keras**版本: 0.10.1
- 你使用纯**keras**还是**tf.keras**: …
-
## ❓ Questions & Help
Good job, thanks for sharing your code.
My system has 2 x NVIDIA 1080Ti. Running data parallel doesn't work for me with the current transformers, and I'd prefer to run dis…