Project byzer-build comes with tools to build
With Byzer Sandbox docker image, users are able to take a quick glance into Byzer stack. Based on spark 2.4.3:
docker run -d --name sandbox -p 9002:9002 -p 9003:9003 -e MYSQL_ROOT_PASSWORD=root byzer/byzer-sandbox:2.4.3-latest
Based on spark 3.1.1:
docker run -d --name sandbox -p 9002:9002 -p 9003:9003 -e MYSQL_ROOT_PASSWORD=root byzer/byzer-sandbox:3.1.1-latest
Based on spark 3.3.0:
docker run -d --name sandbox -p 9002:9002 -p 9003:9003 -e MYSQL_ROOT_PASSWORD=root byzer/byzer-sandbox:3.3.0-latest
docker run -d --name byzer-lang -e DRIVER_MEMORY=8g -e MASTER=local[*] -p9003:9003 byzer/byzer-lang:3.3.0-latest
This command launches byzre-lang container with
The image is built on Ubuntu 20.04, and packaged with
An example to build the image
export BYZER_LANG_VERSION=2.4.0-SNAPSHOT
export SPARK_VERSION=3.3.0
## Run the script on dev directory, or it fails
cd dev
./bin/build-byzer-lang-k8s-azure-image.sh
Directory structure
/work
├── jdk-14
├── logs
├── spark-3.1.1-bin-hadoop3.2
└── user
/home/deploy
├── byzer-lang
├── entrypoint.sh
├── logs
└── spark-warehouse
/opt/conda
├── LICENSE.txt
├── bin
├── compiler_compat
├── conda-meta
├── condabin
├── envs
├── etc
├── include
├── lib
├── [Apr 23 09:29] pkgs
├── share
├── shell
├── ssl
├── x86_64-conda-linux-gnu
└── x86_64-conda_cos6-linux-gnu
To install and run the all-in-one, please visit all-in-one doc .
Tar ball naming convention is byzer-lang-all-in-one-<os>-amd64-<byzer_spark_version>-<byzer_lang_version>.tar.gz
.
Parameter | Explanation |
---|---|
byzer_lang_version | The Byzer Lang version |
byzer_spark_version | 3.3 for Spark 3.3.0, 3.0 for Spark 3.1.1 2.4 for Spark 2.4.3 |
os | linux darwin(mac) win |
To build the Byzer all-in-one
export JUICEFS_VERSION=1.0.0
export SPARK_VERSION=3.3.0
export BYZER_LANG_VERSION=2.4.0-SNAPSHOT
export OS=linux
# darwin for MacOS, win for Windows
./dev/bin/app/build-byzer-cli-release.sh
We provide scripts to start multiple microservices at once to facilitate us to build and start docker containers flexibly. Each service is deployed in an independent Ubuntu environment, which can isolate resources and environments.