Provide AIDeveloper as a Docker Container

[andko76]

Hello Maik,

i had some problems to create the right environment with dependencies under linux to run AIDeveloper from source code, i tried to use your .yml file, but it was not successful.

My wish is to provide AIDeveloper additionally also as Docker container, so it is easy to use it with windows, mac and linux without the trouble to create the right environment with all libraries.

Additional question: Had you already included the heatmaps?

Thanks again for your very good "AIDeveloper" and the big improvements André

[maikherbig]

Hi, I guess docker could be useful, but I have no experience. Could you please explain me the advantages briefly? ->(...or I guess we are all learning here,... so why not just go ahead and create it to show its potential!) Currently, I can provide standalone executables of AID for Windows, Mac and Linux, which means people just need to download, unzip and run. That's it (see the release section https://github.com/maikherbig/AIDeveloper/releases). There are no Linux standalones right now because there were no requests for it and I was lazy :) (I created a Linux version for AID_0.0.6).

I studied for a while on the topic of the heatmaps, and could not find a solution that works reliably every time. Apparently, I need a GUI with a couple of levers and knobs to make heatmaps useful.

Here is a "recipe" to install an environment for AIDeveloper, that worked for me on several machines: -Install miniconda or anaconda -open anaconda command prompt and type: conda create -n aid_envi python=3.5 activate aid_envi pip install --upgrade setuptools pip install keras==2.2.4

----for the CPU only version:----

pip install tensorflow==1.12.3

----for the GPU+CPU version:----

pip install tensorflow-gpu==1.12.3

pip install scikit-learn==0.20.0 pip install dclab==0.22.1 pip install Pillow==5.4.1 pip install pandas==0.24.0 psutil==5.4.7 pip install mkl==2019.0 pip install pyqt5==5.9.2 pip install pyqtgraph==0.11.0rc0 pip install imageio==2.4.1 pip install opencv-contrib-python-headless==4.1.1.26 pip install openpyxl==2.5.6 pip install xlrd==1.1.0 pip install keras2onnx==1.4.0 pip install libopencv pip install ffmpeg==1.4 pip install tf2onnx==1.4.1 pip install numpy==1.16.4

The folders coremltools, keras_metrics, and mmdnn (contained in my repository under AIDeveloper) have to be present in the same directory as your AIDeveloper.py. Furthermore, there are a couple of files (see https://github.com/maikherbig/AIDeveloper/tree/master/AIDeveloper) which have to be present in the same directory as AIDeveloper. Best, mAIk :)

[andko76]

Hello Maik,

thank you for your detailed answer & help. My answer need a bit longer, because i tried to solve my two issues.

1. to install the Conda AIDeveloper environment on a windows pc: (after a lot of frustration now i managed it, the solution was the installation from the building
   tools from visual studio, because this fixed the building problem (wheels) by the packages dclap and psutils) Before i tried to update / upgrade / reinstall the wheel package and a lot other solutions, but without success, but now it's running :-))

2. i liked to install the AIDeveloper Environment on a EC2 Instance in AWS Cloud, after some problems with the graphics now i also finished it and it is running, the GPU support i still have to test

And i tested a little bit with the docker containers, it was no problem to install the environment on a standard ubuntu container, but the problem is to find a solution for the graphic output. With my basic docker / container knowledge i was not able to solve this, but the container can very helpful to deploy a trained model, for example in a tensorflow serving container with a endpoint, where you can send your images and get back the classification result. But i am just getting started.

I understand that you are not happy with the realization from the heatmaps but i think that this feature is very helpful for understanding from the training results and quality check. I would be happy, if you will implement a first solution for this, I would like to support you with the implementation, but my current Python skills are not yet sufficient.

best André

[maikherbig]

That sounds dope! So you want to train a model (using AID) and then provide the model for others via some cloud-based instance built using docker. If you actually intend to let people run inference (not training), you should be fine with using CPU only. Starting the GPU for a forward pass can take some time. Typically, I use GPUs only for training.

[andko76]

Thanks for the flowers, but I think we have misunderstood each other.

I would like to use AID in the AWS Cloud, because I can book instances / processing power (GPU) there for training as needed.

The other one with the containers is only for the provision of the trained model, e.g. for use in a production environment, where only Windows computers are allowed, then you can go the way via dockers and containers, on which Linux runs and you can organize the version management of the model via the containers. Normally you don't need a GPU for this.