Redefine software installation and packaging

[nj1973]

We need to spend some time understanding how a customer moves from a cloned github repo to a working installation.

What do they need, from the top of my head:

• A Python virtual environment with required packages
• The gluentlib egg file, not sure if this should be in the virtual env or in a lib directory which we include in PYTHONPATH
• An Oracle client which we no longer bundle. If we switch to python-oracledb then they probably don't need a client
• An OFFLOAD_HOME containing log run setup, conf and bin directories
• offload.env templates, current generated by templates/conf/make on conf directory
• An OFFLOAD_HOME containing version file
• The contents of setup and bin directories as current produced by the Makefile

I may have missed something.

We still need a make package option which produces a tarball that can be extracted to deploy GOE. There's no way customers will be prepared to run make commands on a production system, they should be able to do it once and produce a tarball which can be deployed to a server.

[nj1973]

I've reinstated Spark Standalone package. These are the steps I've been using on my VM to run Spark locally.

make package-spark-standalone
cp goe_transport_6.0.8-DEV.tar.gz $OFFLOAD_HOME/..
cd $OFFLOAD_HOME/..
tar xf goe_transport_6.0.8-DEV.tar.gz
sed -i "s:/opt/gluent:$PWD:" transport/spark/conf/spark-defaults.conf
$SPARK_HOME/sbin/start-master.sh

[nj1973]

The changes so far give us everything we need in the GOE tarball except for a Python virtual environment. I'm not convinced we should attempt to bundle that, perhaps we should just bundle a frozen requirements.txt file instead.

I think I'll call this issue complete for now and create a new issue for the Python virtual environment question.