search

aegisbigdata / documentation

0 stars 0 forks source link

Pre-install necessary libraries used by AEGIS notebook tools in every newly created project #153

Closed pitsios-s closed 5 years ago

pitsios-s commented 5 years ago

Since all AEGIS notebook tools are installed automatically in every newly created project, it would make sense to also have all the necessary python libraries that those tools are using, being installed automatically, instead of letting the users install them manually each time.

KTH team can fix that, but they need input from each involved partner about the dependencies.

pitsios-s commented 5 years ago

For the visualizer the libraries needed are:

Those are already available by default in every project with python 2.7, but they are missing for python 3.6.

pitsios-s commented 5 years ago

@cbotsikas , @skous could you please provide the necessary input for the QB and the AEC as well?

skous commented 5 years ago

No libraries necessary for AEC, as all are part of hops Will see if some are necessary for the Python3 upgrade (and UI improvement)

cbotsikas commented 5 years ago

Query builder doesn't require any special libraries and it's working on both 2.7 and 3.6 versions. We use:

o-alex commented 5 years ago

The required libraries for visualizations are now installed by default in new projects under both python 2.7 and 3.6. The libraries are currently set so they can be uninstalled or upgraded by users. We have another setting that makes them immutable. @pitsios-s Let me know if you want me to make the libraries immutable. Please test if the current version is working as you expect it to and if yes this issue can be set to resolved.

pitsios-s commented 5 years ago

I tested this functionality with new projects for both python 2.7 and 3.6 and all the basic operations of the visualizer seem to be working as expected.

Personally, I would opt for leaving those libraries as modifiable. If a user is expect enough to uninstall a library, then she is also expert enough to re-install it if she sees a message in Jupyter saying that the X library is missing. After all, this issue is just meant to help users getting started faster with our tools instead of having to configure the libraries each time.

o-alex commented 5 years ago

The fix is: For future installations - automatic through chef - conda+tensorflow:

  1. conda Set the visualisation libraries as default provided libraries: https://github.com/aegisbigdata/conda-chef/commit/842ac3908b79ee0c1e198149bbad435da93e86b4

  2. tensorflow Install the default libraries in the prototype python 2.7 and 3.6 environments: https://github.com/aegisbigdata/tensorflow-chef/commit/6b7631a0cbef302b60462f6d2a118857afb6f628 https://github.com/aegisbigdata/tensorflow-chef/commit/07876743f3d0e5889de86e23e61e99c466082e8a

For an existing installation:

  1. Manually(command line - pip) install the required libraries in the /srv/hops/anaconda/anaconda/envs/pythonX environments, where X is 27 and 36
  2. Go to admin ui -> variables -> provided_python_lib_names and append your new default library to this variable. This will ensure that the python settings page will show the default library as being installed and it will also pick the correct version to show.
pitsios-s commented 5 years ago

Since this issue has been addressed properly, I will proceed by closing it. Thanks @o-alex for your support.