search

MiraldiLab / maxATAC

Transcription Factor Binding Prediction from ATAC-seq and scATAC-seq with Deep Neural Networks
Apache License 2.0
25 stars 8 forks source link

Issue with Loading Models during Prediction. Clash between keras and tensorflow.keras #41

Closed dlab-arp closed 3 years ago

dlab-arp commented 3 years ago

` Lmod is automatically replacing "intel/19.0.5" with "gnu/9.1.0".

Due to MODULEPATH changes, the following have been reloaded: 1) mvapich2/2.3.3

/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/sklearn/ensemble/weight_boosting.py:29: DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and should not be imported. It will be removed in a future NumPy release. from numpy.core.umath_tests import inner1d [2021-07-14 16:42:25,447] Set up model parameters [2021-07-14 16:42:27,473] Import training regions [2021-07-14 16:42:37,597] Initialize data generator [2021-07-14 16:42:37,597] Fit model

                      /\|__   __|/\   / ____|

_ / \ | | / \ | |
| ' ` \ / ` \ \/ / / /\ \ | | / /\ | |
| | | | | | (| |> < / __ | |/ \ | || || ||_,//_\// __// __|

Epoch 1/5

1/10 [==>...........................] - ETA: 12:59 - loss: 1.5907 - dice_coef: 0.0208 - acc: 0.0080 - precision: 0.0098 - recall: 0.8344 - pearson: 0.0242 - spearman: 0.1251 2/10 [=====>........................] - ETA: 6:03 - loss: 1.4849 - dice_coef: 0.0218 - acc: 0.0110 - precision: 0.0103 - recall: 0.8344 - pearson: 0.0332 - spearman: 0.1304  3/10 [========>.....................] - ETA: 3:42 - loss: 1.3926 - dice_coef: 0.0189 - acc: 0.0110 - precision: 0.0090 - recall: 0.7954 - pearson: 0.0242 - spearman: 0.1314 4/10 [===========>..................] - ETA: 2:29 - loss: 1.2897 - dice_coef: 0.0191 - acc: 0.0137 - precision: 0.0091 - recall: 0.7614 - pearson: 0.0292 - spearman: 0.1328 5/10 [==============>...............] - ETA: 1:44 - loss: 1.2038 - dice_coef: 0.0209 - acc: 0.0144 - precision: 0.0098 - recall: 0.7034 - pearson: 0.0297 - spearman: 0.1321 6/10 [=================>............] - ETA: 1:12 - loss: 1.1333 - dice_coef: 0.0206 - acc: 0.0152 - precision: 0.0099 - recall: 0.6793 - pearson: 0.0319 - spearman: 0.1306 7/10 [====================>.........] - ETA: 48s - loss: 1.0706 - dice_coef: 0.0204 - acc: 0.0154 - precision: 0.0101 - recall: 0.6479 - pearson: 0.0318 - spearman: 0.1297  8/10 [=======================>......] - ETA: 29s - loss: 1.0145 - dice_coef: 0.0208 - acc: 0.0166 - precision: 0.0105 - recall: 0.6073 - pearson: 0.0316 - spearman: 0.1295 9/10 [==========================>...] - ETA: 13s - loss: 0.9641 - dice_coef: 0.0208 - acc: 0.0178 - precision: 0.0106 - recall: 0.5607 - pearson: 0.0287 - spearman: 0.1291 10/10 [==============================] - 201s 20s/step - loss: 0.9187 - dice_coef: 0.0210 - acc: 0.0192 - precision: 0.0109 - recall: 0.5224 - pearson: 0.0284 - spearman: 0.1301 - val_loss: 0.3826 - val_dice_coef: 0.0127 - val_acc: 0.5362 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_pearson: -0.0527 - val_spearman: 0.3206 Epoch 2/5

1/10 [==>...........................] - ETA: 40s - loss: 0.4617 - dice_coef: 0.0163 - acc: 0.0220 - precision: 0.0180 - recall: 0.1602 - pearson: 0.0182 - spearman: 0.1291 2/10 [=====>........................] - ETA: 35s - loss: 0.4395 - dice_coef: 0.0183 - acc: 0.0350 - precision: 0.0261 - recall: 0.1712 - pearson: 0.0198 - spearman: 0.1279 3/10 [========>.....................] - ETA: 30s - loss: 0.4264 - dice_coef: 0.0208 - acc: 0.0330 - precision: 0.0304 - recall: 0.1532 - pearson: 0.0249 - spearman: 0.1314 4/10 [===========>..................] - ETA: 26s - loss: 0.4087 - dice_coef: 0.0237 - acc: 0.0330 - precision: 0.0380 - recall: 0.1484 - pearson: 0.0308 - spearman: 0.1332 5/10 [==============>...............] - ETA: 21s - loss: 0.3931 - dice_coef: 0.0236 - acc: 0.0338 - precision: 0.0390 - recall: 0.1356 - pearson: 0.0305 - spearman: 0.1326 6/10 [=================>............] - ETA: 17s - loss: 0.3772 - dice_coef: 0.0237 - acc: 0.0350 - precision: 0.0401 - recall: 0.1287 - pearson: 0.0327 - spearman: 0.1353 7/10 [====================>.........] - ETA: 13s - loss: 0.3643 - dice_coef: 0.0233 - acc: 0.0370 - precision: 0.0421 - recall: 0.1272 - pearson: 0.0341 - spearman: 0.1345 8/10 [=======================>......] - ETA: 8s - loss: 0.3530 - dice_coef: 0.0246 - acc: 0.0382 - precision: 0.0446 - recall: 0.1187 - pearson: 0.0363 - spearman: 0.1378  9/10 [==========================>...] - ETA: 4s - loss: 0.3392 - dice_coef: 0.0253 - acc: 0.0384 - precision: 0.0470 - recall: 0.1182 - pearson: 0.0414 - spearman: 0.1393 10/10 [==============================] - 101s 10s/step - loss: 0.3281 - dice_coef: 0.0263 - acc: 0.0392 - precision: 0.0477 - recall: 0.1090 - pearson: 0.0416 - spearman: 0.1404 - val_loss: 0.4410 - val_dice_coef: 0.0126 - val_acc: 0.3121 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_pearson: -0.1249 - val_spearman: 0.2094 Epoch 3/5

1/10 [==>...........................] - ETA: 41s - loss: 0.2193 - dice_coef: 0.0327 - acc: 0.0400 - precision: 0.1186 - recall: 0.0806 - pearson: 0.0685 - spearman: 0.1316 2/10 [=====>........................] - ETA: 37s - loss: 0.2007 - dice_coef: 0.0292 - acc: 0.0395 - precision: 0.0895 - recall: 0.0710 - pearson: 0.0634 - spearman: 0.1382 3/10 [========>.....................] - ETA: 32s - loss: 0.1927 - dice_coef: 0.0295 - acc: 0.0460 - precision: 0.0914 - recall: 0.0702 - pearson: 0.0647 - spearman: 0.1378 4/10 [===========>..................] - ETA: 27s - loss: 0.1873 - dice_coef: 0.0309 - acc: 0.0440 - precision: 0.0953 - recall: 0.0701 - pearson: 0.0684 - spearman: 0.1420 5/10 [==============>...............] - ETA: 22s - loss: 0.1822 - dice_coef: 0.0332 - acc: 0.0428 - precision: 0.1001 - recall: 0.0672 - pearson: 0.0714 - spearman: 0.1427 6/10 [=================>............] - ETA: 18s - loss: 0.1783 - dice_coef: 0.0321 - acc: 0.0433 - precision: 0.0975 - recall: 0.0629 - pearson: 0.0710 - spearman: 0.1457 7/10 [====================>.........] - ETA: 13s - loss: 0.1723 - dice_coef: 0.0314 - acc: 0.0414 - precision: 0.0910 - recall: 0.0586 - pearson: 0.0674 - spearman: 0.1460 8/10 [=======================>......] - ETA: 8s - loss: 0.1697 - dice_coef: 0.0331 - acc: 0.0408 - precision: 0.0979 - recall: 0.0597 - pearson: 0.0712 - spearman: 0.1450  9/10 [==========================>...] - ETA: 4s - loss: 0.1654 - dice_coef: 0.0328 - acc: 0.0399 - precision: 0.0947 - recall: 0.0554 - pearson: 0.0695 - spearman: 0.1458 10/10 [==============================] - 94s 9s/step - loss: 0.1609 - dice_coef: 0.0320 - acc: 0.0395 - precision: 0.0934 - recall: 0.0537 - pearson: 0.0670 - spearman: 0.1454 - val_loss: 0.3214 - val_dice_coef: 0.0142 - val_acc: 0.4117 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_pearson: -0.1173 - val_spearman: 0.2017 Epoch 4/5

1/10 [==>...........................] - ETA: 41s - loss: 0.1261 - dice_coef: 0.0423 - acc: 0.0360 - precision: 0.0980 - recall: 0.0293 - pearson: 0.0842 - spearman: 0.1578 2/10 [=====>........................] - ETA: 36s - loss: 0.1278 - dice_coef: 0.0445 - acc: 0.0325 - precision: 0.1327 - recall: 0.0386 - pearson: 0.0892 - spearman: 0.1615 3/10 [========>.....................] - ETA: 31s - loss: 0.1242 - dice_coef: 0.0425 - acc: 0.0323 - precision: 0.1010 - recall: 0.0298 - pearson: 0.0801 - spearman: 0.1659 4/10 [===========>..................] - ETA: 26s - loss: 0.1201 - dice_coef: 0.0417 - acc: 0.0330 - precision: 0.0870 - recall: 0.0265 - pearson: 0.0766 - spearman: 0.1707 5/10 [==============>...............] - ETA: 22s - loss: 0.1179 - dice_coef: 0.0410 - acc: 0.0328 - precision: 0.0844 - recall: 0.0250 - pearson: 0.0753 - spearman: 0.1684 6/10 [=================>............] - ETA: 17s - loss: 0.1169 - dice_coef: 0.0417 - acc: 0.0323 - precision: 0.0796 - recall: 0.0230 - pearson: 0.0741 - spearman: 0.1685 7/10 [====================>.........] - ETA: 13s - loss: 0.1167 - dice_coef: 0.0432 - acc: 0.0316 - precision: 0.0774 - recall: 0.0211 - pearson: 0.0763 - spearman: 0.1666 8/10 [=======================>......] - ETA: 8s - loss: 0.1183 - dice_coef: 0.0438 - acc: 0.0342 - precision: 0.0775 - recall: 0.0201 - pearson: 0.0762 - spearman: 0.1678  9/10 [==========================>...] - ETA: 4s - loss: 0.1157 - dice_coef: 0.0450 - acc: 0.0340 - precision: 0.0758 - recall: 0.0205 - pearson: 0.0784 - spearman: 0.1683 10/10 [==============================] - 83s 8s/step - loss: 0.1141 - dice_coef: 0.0449 - acc: 0.0350 - precision: 0.0705 - recall: 0.0193 - pearson: 0.0768 - spearman: 0.1702 - val_loss: 0.2257 - val_dice_coef: 0.0198 - val_acc: 0.0417 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_pearson: -0.0402 - val_spearman: 0.2000 Epoch 5/5

1/10 [==>...........................] - ETA: 40s - loss: 0.1050 - dice_coef: 0.0515 - acc: 0.0340 - precision: 0.0685 - recall: 0.0158 - pearson: 0.0813 - spearman: 0.1857 2/10 [=====>........................] - ETA: 36s - loss: 0.1030 - dice_coef: 0.0502 - acc: 0.0315 - precision: 0.0548 - recall: 0.0126 - pearson: 0.0813 - spearman: 0.1843 3/10 [========>.....................] - ETA: 32s - loss: 0.1129 - dice_coef: 0.0511 - acc: 0.0313 - precision: 0.0568 - recall: 0.0117 - pearson: 0.0775 - spearman: 0.1837 4/10 [===========>..................] - ETA: 27s - loss: 0.1088 - dice_coef: 0.0527 - acc: 0.0330 - precision: 0.0671 - recall: 0.0154 - pearson: 0.0819 - spearman: 0.1810 5/10 [==============>...............] - ETA: 22s - loss: 0.1090 - dice_coef: 0.0514 - acc: 0.0298 - precision: 0.0663 - recall: 0.0148 - pearson: 0.0789 - spearman: 0.1775 6/10 [=================>............] - ETA: 17s - loss: 0.1081 - dice_coef: 0.0506 - acc: 0.0300 - precision: 0.0592 - recall: 0.0135 - pearson: 0.0771 - spearman: 0.1776 7/10 [====================>.........] - ETA: 13s - loss: 0.1090 - dice_coef: 0.0512 - acc: 0.0291 - precision: 0.0578 - recall: 0.0137 - pearson: 0.0766 - spearman: 0.1780 8/10 [=======================>......] - ETA: 8s - loss: 0.1083 - dice_coef: 0.0517 - acc: 0.0296 - precision: 0.0589 - recall: 0.0141 - pearson: 0.0773 - spearman: 0.1779  9/10 [==========================>...] - ETA: 4s - loss: 0.1072 - dice_coef: 0.0533 - acc: 0.0294 - precision: 0.0625 - recall: 0.0154 - pearson: 0.0804 - spearman: 0.1778 10/10 [==============================] - 77s 8s/step - loss: 0.1072 - dice_coef: 0.0535 - acc: 0.0302 - precision: 0.0630 - recall: 0.0152 - pearson: 0.0804 - spearman: 0.1782 - val_loss: 0.1637 - val_dice_coef: 0.0268 - val_acc: 0.0212 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_pearson: 0.0844 - val_spearman: 0.2211 [2021-07-14 16:52:00,754] Plot and save results [2021-07-14 16:52:02,524] Results are saved to: /fs/project/PES0738/maxatac_predictions/20210714_11TF_Dense_DCNN_binary/ARNT/training_results/ARNT_binaryarchs{epoch}.h5 [2021-07-14 16:52:02,524] Total training time: 0:9:37.

/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/sklearn/ensemble/weight_boosting.py:29: DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and should not be imported. It will be removed in a future NumPy release. from numpy.core.umath_tests import inner1d

                      /\|__   __|/\   / ____|

_ / \ | | / \ | |
| ' ` \ / ` \ \/ / / /\ \ | | / /\ | |
| | | | | | (| |> < / __ | |/ \ | || || ||_,//_\// __// __|

Traceback (most recent call last): File "/users/PES0698/ucn2213/.conda/envs/tf1_env/bin/maxatac", line 7, in exec(compile(f.read(), file, 'exec')) File "/users/PES0698/ucn2213/Project-Repos/maxATAC/maxatac/bin/maxatac", line 24, in sys.exit(main(sys.argv[1:])) File "/users/PES0698/ucn2213/Project-Repos/maxATAC/maxatac/bin/maxatac", line 18, in main setup_logger(args.loglevel, LOG_FORMAT) AttributeError: 'Namespace' object has no attribute 'loglevel' /users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/sklearn/ensemble/weight_boosting.py:29: DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and should not be imported. It will be removed in a future NumPy release. from numpy.core.umath_tests import inner1d [2021-07-14 16:52:38,980] Prediction Parameters Output filename: /fs/project/PES0738/maxatac_predictions/20210714_11TF_Dense_DCNN_binary/ARNT/prediction_results/ARNT_binary_revcomp99_RR30.bw Target signal: /fs/project/PES0738/maxATAC_inputs/ATAC/hg38/normalization/GM12878__minmax_percentile99.bw Sequence data: /fs/project/PES0738/maxATAC_inputs/genome_inf/hg38.2bit Models:

[2021-07-14 16:52:38,980] Create prediction regions [2021-07-14 16:53:25,707] Make prediction on forward strand [2021-07-14 16:53:25,709] Load pre-trained model

                      /\|__   __|/\   / ____|

_ / \ | | / \ | |
| ' ` \ / ` \ \/ / / /\ \ | | / /\ | |
| | | | | | (| |> < / __ | |/ \ | || || ||_,//_\// __// __|

Traceback (most recent call last): File "/users/PES0698/ucn2213/.conda/envs/tf1_env/bin/maxatac", line 7, in exec(compile(f.read(), file, 'exec')) File "/users/PES0698/ucn2213/Project-Repos/maxATAC/maxatac/bin/maxatac", line 24, in sys.exit(main(sys.argv[1:])) File "/users/PES0698/ucn2213/Project-Repos/maxATAC/maxatac/bin/maxatac", line 20, in main args.func(args) File "/users/PES0698/ucn2213/Project-Repos/maxATAC/maxatac/analyses/predict.py", line 89, in run_prediction use_complement=False) File "/users/PES0698/ucn2213/Project-Repos/maxATAC/maxatac/utilities/prediction_tools.py", line 299, in make_stranded_predictions nn_model = load_model(models, compile=False) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/engine/saving.py", line 492, in load_wrapper return load_function(args, kwargs) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/engine/saving.py", line 584, in load_model model = _deserialize_model(h5dict, custom_objects, compile) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/engine/saving.py", line 274, in _deserialize_model model = model_from_config(model_config, custom_objects=custom_objects) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/engine/saving.py", line 627, in model_from_config return deserialize(config, custom_objects=custom_objects) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/layers/init.py", line 168, in deserialize printable_module_name='layer') File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 147, in deserialize_keras_object list(custom_objects.items()))) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/engine/network.py", line 1056, in from_config process_layer(layer_data) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/engine/network.py", line 1042, in process_layer custom_objects=custom_objects) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/layers/init.py", line 168, in deserialize printable_module_name='layer') File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 149, in deserialize_keras_object return cls.from_config(config['config']) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/engine/base_layer.py", line 1179, in from_config return cls(config) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(args, kwargs) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/layers/convolutional.py", line 353, in init kwargs) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/layers/convolutional.py", line 117, in init self.kernel_initializer = initializers.get(kernel_initializer) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/initializers.py", line 515, in get return deserialize(identifier) File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/initializers.py", line 510, in deserialize printable_module_name='initializer') File "/users/PES0698/ucn2213/.conda/envs/tf1_env/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 140, in deserialize_keras_object ': ' + class_name) ValueError: Unknown initializer: GlorotUniform`

dlab-arp commented 3 years ago

Import load_model from tensorflow.keras instead of keras use below import line from tensorflow.keras.models import load_model