search

TechWorksHub / ManySecured-D3DB

1 stars 0 forks source link

Turris Omnia python3 packages #16

Open mereacre opened 2 years ago

mereacre commented 2 years ago

Below is the list of default python3 packages installed on Turris Omnia:

python3 - 3.7.13-3.7-1
python3-asyncio - 3.7.13-3.7-1
python3-attrs - 19.3.0-3.7-1
python3-babel - 2.9.1-3.7-1
python3-base - 3.7.13-3.7-1
python3-cachelib - 0.1-3.7-1
python3-certifi - 2019.11.28-3.7-1
python3-cffi - 1.13.2-3.7-1
python3-cgi - 3.7.13-3.7-1
python3-cgitb - 3.7.13-3.7-1
python3-chardet - 3.0.4-3.7-2
python3-click - 7.0-3.7-1
python3-codecs - 3.7.13-3.7-1
python3-cryptography - 2.8-3.7-1
python3-ctypes - 3.7.13-3.7-1
python3-dbm - 3.7.13-3.7-1
python3-decimal - 3.7.13-3.7-1
python3-distro - 1.5.0-3.7-1
python3-distutils - 3.7.13-3.7-1
python3-email - 3.7.13-3.7-1
python3-flask - 1.1.2-3.7-1
python3-flask-babel - 2.0.0-3.7-1
python3-flask-seasurf - 0.3.0-3.7-1
python3-flup - 1.0.3-3.7-3
python3-gdbm - 3.7.13-3.7-1
python3-idna - 2.9-3.7-1
python3-ifaddr - 0.1.7-3.7-1
python3-importlib-metadata - 1.5.0-3.7-1
python3-itsdangerous - 1.1.0-3.7-1
python3-jinja2 - 2.10.1-3.7-1
python3-jsonschema - 3.2.0-3.7-2
python3-light - 3.7.13-3.7-1
python3-logging - 3.7.13-3.7-1
python3-lzma - 3.7.13-3.7-1
python3-markupsafe - 1.1.1-3.7-1
python3-more-itertools - 7.2.0-3.7-1
python3-multiprocessing - 3.7.13-3.7-1
python3-ncurses - 3.7.13-3.7-1
python3-openssl - 3.7.13-3.7-1
python3-paho-mqtt - 1.5.1-3.7-1
python3-pbkdf2 - 1.3-3.7-4
python3-pkg-resources - 47.1.0-1
python3-ply - 3.11-3.7-1
python3-prctl - 1.7-3.7-2
python3-pycparser - 2.20-3.7-1
python3-pydoc - 3.7.13-3.7-1
python3-pyrsistent - 0.15.3-3.7-1
python3-python - 1.0.0-1
python3-pytz - 2019.3-3.7-1
python3-requests - 2.23.0-3.7-1
python3-setuptools - 47.1.0-1
python3-simplejson - 3.16.0-3.7-2
python3-six - 1.13.0-3.7-1
python3-slugify - 3.0.3-3.7-1
python3-sqlite3 - 3.7.13-3.7-1
python3-text-unidecode - 1.2-3.7-1
python3-ubus - 0.1-3.7-2
python3-uci - 0.8.1-3.7-1
python3-unittest - 3.7.13-3.7-1
python3-urllib - 3.7.13-3.7-1
python3-urllib3 - 1.25.10-3.7-1
python3-websockets - 9.1-3.7-1
python3-werkzeug - 0.16.0-3.7-1
python3-xml - 3.7.13-3.7-1
python3-zeroconf - 0.28.0-3.7-1
python3-zipp - 0.6.0-3.7-2

Ideally for code running on Turris we should use the packages that come with the device. We could use other packages but it creates complication with the disk space, etc.

cbrafter commented 2 years ago

Thanks for this, if the scripts build the YAML to CSV it should be straightforward to read in Python using the built-in file IO, and sqlite3 will cover updating the database. I don't think we'll need to install any extra packages on the Turris.

aloisklink commented 2 years ago

python3-jsonschema - 3.2.0-3.7-2

Nice, the Turris even comes with the jsonschema package pre-installed. That's great.

I don't think we'll need to install any extra packages on the Turris.

100% agree. YAML parsing is probably the only extra package we need, but if we have json/csv D3 claims, that can be removed.

(although, PyYAML is pretty heavy since it's a C-compiled Python lib, but even then, it's probably only around 1 MB in size https://pypi.org/project/PyYAML/#files)

I'm guessing it's probably worth splitting up the Python code into 3 different packages, d3-importer, d3-convert, d3-lint (names not set it stone). The d3-lint and d3-convert can be designed to run on CI/server and use almost any dependencies, while the d3-importer will be the one that must run on the Turris Omnia.