wenbostar
commented
3 years ago It seems tensorflow version is not the right one. Could you install tensorflow 2.6?
Closed WinkelsK closed 1 year ago
wenbostar
commented
3 years ago It seems tensorflow version is not the right one. Could you install tensorflow 2.6?
WinkelsK
commented
3 years ago Hey - thanks for your quick answer. I changed the requirements to
tensorflow==2.6.0
matplotlib
pandas
scikit-learn
numpyAnd run
pip install -r requirements.txtIt seemed to work as the output suggests
(base) C:\AutoRT_master\AutoRT_master>pip install -r requirements.txt
Requirement already satisfied: tensorflow==2.6.0 in c:\programdata\anaconda3\lib
\site-packages (from -r requirements.txt (line 1)) (2.6.0)
Requirement already satisfied: matplotlib in c:\programdata\anaconda3\lib\site-p
ackages (from -r requirements.txt (line 2)) (3.3.4)
Requirement already satisfied: pandas in c:\programdata\anaconda3\lib\site-packa
ges (from -r requirements.txt (line 3)) (1.2.4)
Requirement already satisfied: scikit-learn in c:\programdata\anaconda3\lib\site
-packages (from -r requirements.txt (line 4)) (0.24.1)
Requirement already satisfied: numpy in c:\programdata\anaconda3\lib\site-packag
es (from -r requirements.txt (line 5)) (1.19.5)
Requirement already satisfied: wheel~=0.35 in c:\programdata\anaconda3\lib\site-
packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (0.36.2)
Requirement already satisfied: six~=1.15.0 in c:\programdata\anaconda3\lib\site-
packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (1.15.0)
Requirement already satisfied: flatbuffers~=1.12.0 in c:\programdata\anaconda3\l
ib\site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (1.12)
Requirement already satisfied: absl-py~=0.10 in c:\programdata\anaconda3\lib\sit
e-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (0.14.1)
Requirement already satisfied: keras~=2.6 in c:\programdata\anaconda3\lib\site-p
ackages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (2.6.0)
Requirement already satisfied: typing-extensions~=3.7.4 in c:\programdata\anacon
da3\lib\site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (3.
7.4.3)
Requirement already satisfied: tensorflow-estimator~=2.6 in c:\programdata\anaco
nda3\lib\site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (2
.6.0)
Requirement already satisfied: opt-einsum~=3.3.0 in c:\programdata\anaconda3\lib
\site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (3.3.0)
Requirement already satisfied: clang~=5.0 in c:\programdata\anaconda3\lib\site-p
ackages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (5.0)
Requirement already satisfied: gast==0.4.0 in c:\programdata\anaconda3\lib\site-
packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (0.4.0)
Requirement already satisfied: wrapt~=1.12.1 in c:\programdata\anaconda3\lib\sit
e-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (1.12.1)
Requirement already satisfied: h5py~=3.1.0 in c:\programdata\anaconda3\lib\site-
packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (3.1.0)
Requirement already satisfied: keras-preprocessing~=1.1.2 in c:\programdata\anac
onda3\lib\site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (
1.1.2)
Requirement already satisfied: google-pasta~=0.2 in c:\programdata\anaconda3\lib
\site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (0.2.0)
Requirement already satisfied: astunparse~=1.6.3 in c:\programdata\anaconda3\lib
\site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (1.6.3)
Requirement already satisfied: protobuf>=3.9.2 in c:\programdata\anaconda3\lib\s
ite-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (3.18.1)
Requirement already satisfied: tensorboard~=2.6 in c:\programdata\anaconda3\lib\
site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (2.7.0)
Requirement already satisfied: grpcio<2.0,>=1.37.0 in c:\programdata\anaconda3\l
ib\site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (1.41.0)
Requirement already satisfied: termcolor~=1.1.0 in c:\programdata\anaconda3\lib\
site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (1.1.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in c:\programdat
a\anaconda3\lib\site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requ
irements.txt (line 1)) (0.4.6)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in c:\progr
amdata\anaconda3\lib\site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r
requirements.txt (line 1)) (0.6.1)
Requirement already satisfied: google-auth<3,>=1.6.3 in c:\programdata\anaconda3
\lib\site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.tx
t (line 1)) (2.3.0)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in c:\programdata\a
naconda3\lib\site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r require
ments.txt (line 1)) (1.8.0)
Requirement already satisfied: setuptools>=41.0.0 in c:\programdata\anaconda3\li
b\site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (
line 1)) (52.0.0.post20210125)
Requirement already satisfied: markdown>=2.6.8 in c:\programdata\anaconda3\lib\s
ite-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (lin
e 1)) (3.3.4)
Requirement already satisfied: requests<3,>=2.21.0 in c:\programdata\anaconda3\l
ib\site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt
(line 1)) (2.25.1)
Requirement already satisfied: werkzeug>=0.11.15 in c:\programdata\anaconda3\lib
\site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (l
ine 1)) (1.0.1)
Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\programdata\anaconda3
\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2.
6.0->-r requirements.txt (line 1)) (0.2.8)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in c:\programdata\anaconda
3\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2
.6.0->-r requirements.txt (line 1)) (4.2.4)
Requirement already satisfied: rsa<5,>=3.1.4 in c:\programdata\anaconda3\lib\sit
e-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2.6.0->-r
requirements.txt (line 1)) (4.7.2)
Requirement already satisfied: requests-oauthlib>=0.7.0 in c:\programdata\anacon
da3\lib\site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->
tensorflow==2.6.0->-r requirements.txt (line 1)) (1.3.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in c:\programdata\anaconda3\
lib\site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboar
d~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (0.4.8)
Requirement already satisfied: chardet<5,>=3.0.2 in c:\programdata\anaconda3\lib
\site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0->-
r requirements.txt (line 1)) (4.0.0)
Requirement already satisfied: certifi>=2017.4.17 in c:\programdata\anaconda3\li
b\site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0->
-r requirements.txt (line 1)) (2020.12.5)
Requirement already satisfied: idna<3,>=2.5 in c:\programdata\anaconda3\lib\site
-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0->-r req
uirements.txt (line 1)) (2.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\programdata\anaconda3
\lib\site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.
0->-r requirements.txt (line 1)) (1.26.4)
Requirement already satisfied: oauthlib>=3.0.0 in c:\programdata\anaconda3\lib\s
ite-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->t
ensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (3.1.1)
Requirement already satisfied: cycler>=0.10 in c:\programdata\anaconda3\lib\site
-packages (from matplotlib->-r requirements.txt (line 2)) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\programdata\anaconda3\lib
\site-packages (from matplotlib->-r requirements.txt (line 2)) (1.3.1)
Requirement already satisfied: pillow>=6.2.0 in c:\programdata\anaconda3\lib\sit
e-packages (from matplotlib->-r requirements.txt (line 2)) (8.2.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in c:\pr
ogramdata\anaconda3\lib\site-packages (from matplotlib->-r requirements.txt (lin
e 2)) (2.4.7)
Requirement already satisfied: python-dateutil>=2.1 in c:\programdata\anaconda3\
lib\site-packages (from matplotlib->-r requirements.txt (line 2)) (2.8.1)
Requirement already satisfied: pytz>=2017.3 in c:\programdata\anaconda3\lib\site
-packages (from pandas->-r requirements.txt (line 3)) (2021.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in c:\programdata\anaconda3\
lib\site-packages (from scikit-learn->-r requirements.txt (line 4)) (2.1.0)
Requirement already satisfied: joblib>=0.11 in c:\programdata\anaconda3\lib\site
-packages (from scikit-learn->-r requirements.txt (line 4)) (1.0.1)
Requirement already satisfied: scipy>=0.19.1 in c:\programdata\anaconda3\lib\sit
e-packages (from scikit-learn->-r requirements.txt (line 4)) (1.6.2)Could you run AutoRT example again?
WinkelsK
commented
3 years ago Sure. It comes up with a similar issue (to my opinion)
(base) C:\AutoRT_master\AutoRT_master\example>python ../autort.py train -i data/
28CPTAC_COprospective_W_VU_20150810_05CO037_f01_normal_train.tsv -o tf_model/ -e
40 -b 64 -u m -m ../models/general_base_model/model.json -rlr -n 10
2021-10-19 15:19:48.982615: W tensorflow/stream_executor/platform/default/dso_lo
ader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64
_110.dll not found
2021-10-19 15:19:48.987616: I tensorflow/stream_executor/cuda/cudart_stub.cc:29]
Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Scaling method: min_max
Step 1:
Load aa coding data from file ../models/general_base_model/aa.tsv
AA types: 25
Longest peptide in training data: 41
Use test file tf_model//step1/validation.tsv
Longest peptide in test data: 30
['1', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'K', 'L', 'M', 'N', 'P', 'Q', 'R',
'S', 'T', 'V', 'W', 'Y']
RT range: 0 - 98
X_train shape:
(4626, 60)
X_test shape:
(515, 60)
Modeling start ...
Use one GPU or no GPU!
2021-10-19 15:19:52.894839: W tensorflow/stream_executor/platform/default/dso_lo
ader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64
_110.dll not found
2021-10-19 15:19:52.899839: I tensorflow/stream_executor/cuda/cudart_stub.cc:29]
Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Model training: 0
2021-10-19 15:19:56.472044: W tensorflow/stream_executor/platform/default/dso_lo
ader.cc:64] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not
found
2021-10-19 15:19:56.477044: W tensorflow/stream_executor/cuda/cuda_driver.cc:269
] failed call to cuInit: UNKNOWN ERROR (303)
2021-10-19 15:19:56.482044: I tensorflow/stream_executor/cuda/cuda_diagnostics.c
c:169] retrieving CUDA diagnostic information for host: MININT-L73FV72
2021-10-19 15:19:56.486044: I tensorflow/stream_executor/cuda/cuda_diagnostics.c
c:176] hostname: MININT-L73FV72
2021-10-19 15:19:56.490045: I tensorflow/core/platform/cpu_feature_guard.cc:142]
This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (on
eDNN) to use the following CPU instructions in performance-critical operations:
AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate comp
iler flags.
60
Perform transfer learning ...
The number of layers: 11
<keras.optimizer_v2.adam.Adam object at 0x00000000500D3FA0>
Use optimizer: Adam from saved model
Used optimizer:
<keras.optimizer_v2.adam.Adam object at 0x00000000500CAD00>
Batch size: 64
Epoch: 40
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\multiprocessing\pool.py", line 125, in work
er
result = (True, func(*args, **kwds))
File "C:\ProgramData\Anaconda3\lib\multiprocessing\pool.py", line 48, in mapst
ar
return list(map(*args))
File "C:\AutoRT_master\AutoRT_master\autort\ModelT.py", line 14, in run_model_
t
model_t.run()
File "C:\AutoRT_master\AutoRT_master\autort\ModelT.py", line 153, in run
new_model.fit(ModelT.x_train, ModelT.y_train, batch_size=ModelT.batch_size,
epochs=ModelT.n_epoch,
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\training.py", li
ne 1134, in fit
data_handler = data_adapter.get_data_handler(
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\data_adapter.py"
, line 1383, in get_data_handler
return DataHandler(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\data_adapter.py"
, line 1137, in __init__
adapter_cls = select_data_adapter(x, y)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\data_adapter.py"
, line 976, in select_data_adapter
raise ValueError(
ValueError: Failed to find data adapter that can handle input: <class 'NoneType'
>, <class 'NoneType'>
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "../autort.py", line 158, in <module>
main()
File "../autort.py", line 109, in main
two_step_ensemble_models(input_data=input_file, nb_epoch=epochs, batch_size=
batch_size,
File "C:\AutoRT_master\AutoRT_master\autort\RTModels.py", line 675, in two_ste
p_ensemble_models
ensemble_models(models_file=models_file, input_data=input_data, ensemble_met
hod=ensemble_method,
File "C:\AutoRT_master\AutoRT_master\autort\RTModels.py", line 814, in ensembl
e_models
trained_model = p.map(run_model_t, [model_t])
File "C:\ProgramData\Anaconda3\lib\multiprocessing\pool.py", line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "C:\ProgramData\Anaconda3\lib\multiprocessing\pool.py", line 771, in get
raise self._value
File "C:\ProgramData\Anaconda3\lib\multiprocessing\pool.py", line 125, in work
er
result = (True, func(*args, **kwds))
File "C:\ProgramData\Anaconda3\lib\multiprocessing\pool.py", line 48, in mapst
ar
return list(map(*args))
File "C:\AutoRT_master\AutoRT_master\autort\ModelT.py", line 14, in run_model_
t
model_t.run()
File "C:\AutoRT_master\AutoRT_master\autort\ModelT.py", line 153, in run
new_model.fit(ModelT.x_train, ModelT.y_train, batch_size=ModelT.batch_size,
epochs=ModelT.n_epoch,
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\training.py", li
ne 1134, in fit
data_handler = data_adapter.get_data_handler(
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\data_adapter.py"
, line 1383, in get_data_handler
return DataHandler(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\data_adapter.py"
, line 1137, in __init__
adapter_cls = select_data_adapter(x, y)
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\data_adapter.py"
, line 976, in select_data_adapter
raise ValueError(
ValueError: Failed to find data adapter that can handle input: <class 'NoneType'
>, <class 'NoneType'>I'm trying to reproduce this error on a Windows PC. I Will let you know how this can be fixed soon.
WinkelsK
commented
3 years ago Great - thanks for the support :)
wenbostar
commented
3 years ago This was caused by different behaviors of the multiprocessing package between Windows and Linux as described here: https://www.pythonforthelab.com/blog/differences-between-multiprocessing-windows-and-linux/. I just updated the code and it should work well on Windows now. Please try again.
WinkelsK
commented
3 years ago Great - it is running now! Thanks!
wenbostar
commented
1 year ago @hetaofan , could you open a new issue about your question?
Hey all - I just downloaded AutoRT. It sounds like its exactly what I want :) Unfortunately I wasn't able to run the example files. It comes up with the following error. As I don't have much experience in Python, I am looking forward to your input! Thanks!