evilsocket / pwnagotchi

(⌐■_■) - Deep Reinforcement Learning instrumenting bettercap for WiFi pwning.
https://pwnagotchi.ai/
Other
7.81k stars 1.16k forks source link

Generate the "requirements.txt" file with "pip freeze" #797

Open hectorm opened 4 years ago

hectorm commented 4 years ago

Describe the solution you'd like Now that #759 and #784 are merged I think that to improve the reproducibility of the project we should generate the requirements.txt file with pip freeze.

This file should be periodically updated, maybe in every Pwnagotchi version?

If you agree with this I can open a PR.

dadav commented 4 years ago

We can try this, BUT it was a pita to get the dependencies right (god bless @caquino) and I don't think this will be as easy as you might think^^

hectorm commented 4 years ago

You're right, it's a mess. Especially supporting both ARM32v6 and AMD64.

For some reason TensorFlow 1.13.1 in piwheels have defined different constraints than in PyPI (piwheels/packages#66). And it isn't by chance, since the 1.13.1 version in piwheels imports tensorflow_estimator.python.estimator.tpu, which only exists since TensorFlow Estimator 1.14.0.

I've also tried to update TensorFlow to 1.14.0, but for some reason in piwheels it is built without tensorflow.contrib.

Fortunately I'm building Pwnagotchi with Docker and venv, so it hasn't been complicated to quickly try different dependency versions until I got it right (with some drawbacks).

After some basic tests (AI, display, led, grid, session stats and memtemp working fine), these are the latest dependency versions that seems to work on ARM32v6 (to make it work on AMD64, the TensorFlow Estimator version must be changed to 1.13.0).

absl-py==0.9.0
astor==0.8.1
certifi==2019.11.28
chardet==3.0.4
Click==7.0
cloudpickle==1.2.2
cycler==0.10.0
dbus-python==1.2.16
file-read-backwards==2.0.0
Flask==1.1.1
Flask-Cors==3.0.8
Flask-WTF==0.14.2
future==0.18.2
gast==0.2.2
google-pasta==0.1.8
grpcio==1.26.0
gym==0.15.4
h5py==2.10.0
idna==2.8
inky==0.0.5
itsdangerous==1.1.0
Jinja2==2.10.3
joblib==0.14.1
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
kiwisolver==1.1.0
Markdown==3.1.1
MarkupSafe==1.1.1
matplotlib==3.1.2
mock==3.0.5
numpy==1.18.1
oauthlib==3.1.0
opencv-python==4.1.1.26
pandas==0.25.3
Pillow==7.0.0
protobuf==3.11.2
pycryptodome==3.9.4
pyglet==1.3.2
pyparsing==2.4.6
PySocks==1.7.1
python-dateutil==2.8.1
pytz==2019.3
PyYAML==5.3
requests==2.22.0
requests-oauthlib==1.3.0
RPi.GPIO==0.7.0
scapy==2.4.3
scipy==1.3.3
six==1.14.0
smbus2==0.3.0
spidev==3.4
stable-baselines==2.9.0
tensorboard==1.13.1
tensorflow==1.13.1
tensorflow-estimator==1.14.0
termcolor==1.1.0
toml==0.10.0
tweepy==3.8.0
urllib3==1.25.8
Werkzeug==0.16.0
wrapt==1.11.2
WTForms==2.2.1

Until the discrepancies between piwheels and PyPI are resolved, maintaining this list would be tricky.