search

totti0223 / deep_learning_for_biologists_with_keras

tutorials made for biologists to learn deep learning
105 stars 28 forks source link

Dependency list and Python version(s)? #1

Closed arpieb closed 5 years ago

arpieb commented 5 years ago

Hi there! I really like the motivation behind this project, and have already shared it with some colleagues in the biology field. One question though - I import the notebooks into my existing Jupyter installation but am getting missing dependencies, but don't see any kind of requirements.txt or similar file listing dependencies. Is it possible to export the dependency list for those of us who don't use Colab? Also, which version of Python are the notebooks written for?

Thanks!

totti0223 commented 5 years ago

hi there. thankyou for understanding the concepts behind this project. :) First of all, the codes were written in Colab environment. The priority of this tutorial was intended to run in Google Colab only and did not expect users to run offline at respective jupyter notebooks, however, it is reasonable that people who wants to run offline exists in certain extent, so may create a requirements.txt to fully run all the notebooks offline in the near future.

For now, colab is run in python 3.(6?).

1.Taking a quick look at the library used in my current notebooks, installing the bellow packages is probably sufficient (excuse if I had missed something)

numpy matplotlib scipy scikit-image sklearn pandas keras tensorflow pyyaml (see bellow for versions)

2.Alternative (did not check myself) After running the notebooks in colab (executing all the library cells), Run the following command in the cell as in the cited url. https://stackoverflow.com/questions/4858100/how-to-list-imported-modules

import types
def imports():
    for name, val in globals().items():
        if isinstance(val, types.ModuleType):
            yield val.__name__
list(imports())

This may list all the libraries required.

3.Alternative2. The easiest (but possibly pretty time consuming) way is to install all the libraries listed bellow in a new isolated environment as outputted by Colab notebook in GPU runtime mode.

!pip freeze
absl-py==0.6.1
alabaster==0.7.12
albumentations==0.1.9
altair==2.3.0
astor==0.7.1
astropy==3.0.5
atari-py==0.1.7
atomicwrites==1.2.1
attrs==18.2.0
audioread==2.1.6
autograd==1.2
Babel==2.6.0
backports.tempfile==1.0
backports.weakref==1.0.post1
beautifulsoup4==4.6.3
bleach==3.0.2
bokeh==1.0.3
boto==2.49.0
boto3==1.9.75
botocore==1.12.75
bs4==0.0.1
bz2file==0.98
cachetools==3.0.0
certifi==2018.11.29
cffi==1.11.5
chainer==5.0.0
chardet==3.0.4
Click==7.0
cloudpickle==0.6.1
cmake==3.12.0
colorlover==0.2.1
community==1.0.0b1
contextlib2==0.5.5
convertdate==2.1.3
coverage==3.7.1
coveralls==0.5
crcmod==1.7
cufflinks==0.14.6
cupy-cuda92==5.0.0
cvxopt==1.2.2
cvxpy==1.0.11
cycler==0.10.0
cymem==2.0.2
Cython==0.29.2
cytoolz==0.9.0.1
daft==0.0.4
dask==0.20.2
datascience==0.10.6
decorator==4.3.0
defusedxml==0.5.0
dill==0.2.8.2
distributed==1.25.2
Django==2.1.5
dlib==19.16.0
dm-sonnet==1.23
docopt==0.6.2
docutils==0.14
dopamine-rl==1.0.5
easydict==1.9
ecos==2.0.7.post1
editdistance==0.5.2
en-core-web-sm==2.0.0
entrypoints==0.3
ephem==3.7.6.0
et-xmlfile==1.0.1
fa2==0.2
fancyimpute==0.4.2
fastcache==1.0.2
fastdtw==0.3.2
fastrlock==0.4
fbprophet==0.4
featuretools==0.4.1
filelock==3.0.10
fix-yahoo-finance==0.0.22
Flask==1.0.2
folium==0.2.1
future==0.16.0
gast==0.2.1.post0
GDAL==2.2.2
gdown==3.6.4
gensim==3.6.0
geographiclib==1.49
geopy==1.17.0
gevent==1.4.0
gin-config==0.1.2
glob2==0.6
google==2.0.1
google-api-core==1.7.0
google-api-python-client==1.6.7
google-auth==1.4.2
google-auth-httplib2==0.0.3
google-auth-oauthlib==0.2.0
google-cloud-bigquery==1.1.0
google-cloud-core==0.28.1
google-cloud-language==1.0.2
google-cloud-storage==1.8.0
google-cloud-translate==1.3.3
google-colab==0.0.1a1
google-resumable-media==0.3.2
googleapis-common-protos==1.5.5
googledrivedownloader==0.3
graph-nets==1.0.2
graphviz==0.10.1
greenlet==0.4.15
grpcio==1.15.0
gspread==3.0.1
gspread-dataframe==3.0.2
gunicorn==19.9.0
gym==0.10.9
h5py==2.8.0
HeapDict==1.0.0
holidays==0.9.9
html5lib==1.0.1
httpimport==0.5.16
httplib2==0.11.3
humanize==0.5.1
hyperopt==0.1.1
ideep4py==2.0.0.post3
idna==2.6
image==1.5.27
imageio==2.4.1
imagesize==1.1.0
imbalanced-learn==0.4.3
imblearn==0.0
imgaug==0.2.6
imutils==0.5.2
inflect==2.1.0
intel-openmp==2019.0
intervaltree==2.1.0
ipykernel==4.6.1
ipython==5.5.0
ipython-genutils==0.2.0
ipython-sql==0.3.9
ipywidgets==7.4.2
itsdangerous==1.1.0
jdcal==1.4
jieba==0.39
Jinja2==2.10
jmespath==0.9.3
joblib==0.13.0
jpeg4py==0.1.4
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.2.4
jupyter-console==6.0.0
jupyter-core==4.4.0
kaggle==1.5.1.1
kapre==0.1.3.1
Keras==2.2.4
Keras-Applications==1.0.6
Keras-Preprocessing==1.0.5
keras-vis==0.4.1
knnimpute==0.1.0
librosa==0.6.2
lightgbm==2.2.2
llvmlite==0.27.0
lmdb==0.94
lucid==0.3.8
lunardate==0.2.0
lxml==4.2.6
magenta==0.3.19
Markdown==3.0.1
MarkupSafe==1.1.0
matplotlib==2.1.2
matplotlib-venn==0.11.5
mesh-tensorflow==0.0.5
mido==1.2.6
mir-eval==0.5
missingno==0.4.1
mistune==0.8.4
mkl==2019.0
mlxtend==0.14.0
more-itertools==5.0.0
moviepy==0.2.3.5
mpi4py==3.0.0
mpmath==1.1.0
msgpack==0.5.6
msgpack-numpy==0.4.3.2
multiprocess==0.70.6.1
multitasking==0.0.7
murmurhash==1.0.1
music21==5.5.0
natsort==5.5.0
nbconvert==5.4.0
nbformat==4.4.0
networkx==2.2
nibabel==2.3.2
nltk==3.2.5
nose==1.3.7
notebook==5.2.2
np-utils==0.5.9.0
numba==0.40.1
numexpr==2.6.9
numpy==1.14.6
oauth2client==4.1.3
oauthlib==3.0.0
okgrade==0.4.3
olefile==0.46
opencv-contrib-python==3.4.3.18
opencv-python==3.4.5.20
openpyxl==2.5.9
osqp==0.5.0
packaging==18.0
pandas==0.22.0
pandas-datareader==0.7.0
pandas-gbq==0.4.1
pandas-profiling==1.4.1
pandocfilters==1.4.2
pathlib==1.0.1
patsy==0.5.1
pexpect==4.6.0
pickleshare==0.7.5
Pillow==4.0.0
plac==0.9.6
plotly==1.12.12
pluggy==0.8.0
portpicker==1.2.0
prefetch-generator==1.0.1
preshed==2.0.1
pretty-midi==0.2.8
prettytable==0.7.2
progressbar2==3.38.0
promise==2.2.1
prompt-toolkit==1.0.15
protobuf==3.6.1
psutil==5.4.8
psycopg2==2.7.6.1
ptyprocess==0.6.0
py==1.7.0
pyasn1==0.4.5
pyasn1-modules==0.2.3
pycocotools==2.0.0
pycparser==2.19
pydot==1.3.0
pydot-ng==2.0.0
pydotplus==2.0.2
pyemd==0.5.1
pyglet==1.3.2
Pygments==2.1.3
pygobject==3.26.1
pymc3==3.6
pymongo==3.7.2
pymystem3==0.2.0
PyOpenGL==3.1.0
pyparsing==2.3.0
pysndfile==1.3.2
PySocks==1.6.8
pystache==0.5.4
pystan==2.18.1.0
pytest==3.10.1
python-apt==1.6.3
python-chess==0.23.11
python-dateutil==2.5.3
python-louvain==0.13
python-rtmidi==1.1.2
python-slugify==2.0.1
python-utils==2.3.0
pytz==2018.9
PyWavelets==1.0.1
PyYAML==3.13
pyzmq==17.0.0
qtconsole==4.4.3
regex==2018.1.10
requests==2.18.4
requests-oauthlib==1.0.0
resampy==0.2.1
rsa==4.0
s3fs==0.2.0
s3transfer==0.1.13
scikit-image==0.13.1
scikit-learn==0.20.2
scipy==1.1.0
screen-resolution-extra==0.0.0
scs==2.0.2
seaborn==0.7.1
simplegeneric==0.8.1
six==1.11.0
sklearn==0.0
smart-open==1.7.1
snowballstemmer==1.2.1
sortedcontainers==2.1.0
spacy==2.0.18
Sphinx==1.8.3
sphinxcontrib-websupport==1.1.0
SQLAlchemy==1.2.15
sqlparse==0.2.4
stable-baselines==2.2.1
statsmodels==0.8.0
sympy==1.1.1
tables==3.4.4
tabulate==0.8.2
tblib==1.3.2
tensor2tensor==1.11.0
tensorboard==1.12.2
tensorboardcolab==0.0.22
tensorflow==1.12.0
tensorflow-hub==0.2.0
tensorflow-metadata==0.9.0
tensorflow-probability==0.5.0
termcolor==1.1.0
terminado==0.8.1
testpath==0.4.2
textblob==0.15.2
textgenrnn==1.4.1
tfds-nightly==0.0.2.dev201901080014
tflearn==0.3.2
Theano==1.0.3
thinc==6.12.1
toolz==0.9.0
tornado==4.5.3
tqdm==4.28.1
traitlets==4.3.2
tweepy==3.6.0
ujson==1.35
umap-learn==0.3.7
Unidecode==1.0.23
uritemplate==3.0.0
urllib3==1.22
vega-datasets==0.7.0
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.14.1
widgetsnbextension==3.4.2
wordcloud==1.5.0
wrapt==1.10.11
xarray==0.11.2
xgboost==0.7.post4
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yellowbrick==0.9
zict==0.1.3
zmq==0.0.0
arpieb commented 5 years ago

@totti0223 thank you for the thorough reply! I haven't worked with Colab, and it was easier (that was the plan!) to drop the notebooks into my current JupyterHub server with Keras and most of the popular libs (tf-gpu, numpy, scipy, pandas, etc ad nauseum). I think I have enough to work from in getting this repo to work on JupyterHub, thanks!