search

sassoftware / sas-container-recipes

A collection of recipes and other resources for building containers that include SAS Viya software.
https://www.sas.com
Apache License 2.0
89 stars 42 forks source link

SAS Viya Deployment Size #1

Closed sylus closed 6 years ago

sylus commented 6 years ago

Hi there! @hornpolish :)

I work at Statistics Canada and am super keen to help in this area as well and keen to make improvements to my workflow and align to best practices.

One thing I noticed is your installation is different then mine and I was curious what your total container size was with just Viya? I only ask because when I launch my SAS container in K8S it is a hefty 5GB which is why our jupyterhub SASpy notebook just runs sas kernel with SASpy and communicates with the separately launched SAS container via IOM. So when we have multiple launched jupyterhub users don't have such monolithic containers running.

However sometimes when working with files in the jupyter notebook it is nice to also be able to send those files to SAS without needing to use pandas so if the container size was more reduced I would no longer need to separate it from saspy. (Separately we did not separate our strata notebook with stata which was 1gb so was hoping for 1-2gb or so as a goal)

My temporary solution is to leverage blobfuse on the jupyter notebook saspy container and the SAS container as ultimately we want the data to reside on azure blob storage.

Background

a) Install steps I used: https://github.com/govcloud/docker-sas4c/blob/master/docs/install.md b) Dockerfile: https://github.com/govcloud/docker-sas4c/blob/master/studio/Dockerfile c) Jupyter SASpy Notebook: https://github.com/govcloud/jupyter-notebooks/tree/master/sas c) Jupyter SASpy Notebook: https://github.com/govcloud/jupyter-notebooks/tree/master/stata d) SAS Container deployed in cluster: https://github.com/govcloud/public-charts/tree/master/incubator/sas4c e) Relevant Issue: https://github.com/sassoftware/saspy/issues/151

hornpolish commented 6 years ago

Hi @sylus

Nice to meet you on the innertoobs.

This project is for cooking SAS Viya orders into containers; yours is based of of SAS Version 9 (SAS Analytics for containers).

The containers produced are larger, but it really depends what products are in the license/order that is input to the docker build process. I have an order with lots of products and Access Engines where if i comment out anaconda, so its "just SAS", the container is 13GB. 3GB of those are maps - they could be nfs mounted into a container, but i'm not sure what the gains from doing that are. (ie using a map over a mount versus from inside the container)

i sent you an email with a few more "du -sh" of likely folders inside the image

hornpolish commented 6 years ago

Hi @sylus ; May i close this issue? We know its a pretty big image and are working to trim it down, but its always gonna be several (maybe 10) GB.

sylus commented 6 years ago

Hi there!

Yeah that works for me :) Will take a look at the email and see how can reduce it. Mine is about 3.8GB but with more then a few things removed ^_^