container architecture

[suvarchal]

Just my notes/issues on hecuba container

• Would be good if hecuba container can use versions of python>3.6, also might be helpful for CI.
• For the container it hecuba might be better off using different container for cassandra and hecuba (docker hub has an image that can be used), then docker-compose can be used to scale cassandra nodes to reflect realistic usecase. This also removes need for the Makefile.

A example docker-compose file as such

version: "3.3"

networks:
  cassandra_backend:
    driver: bridge

services:
  cassandra-seed:
    image: "cassandra:4.0"
   # optionally expose to outside network
    ports:
       - "7000:7000"
       - "7199:7199"
       - "9042:9042"
       - "9160:9160"
    environment:
      - CASSANDRA_SEEDS=cassandra-seed
    healthcheck:
      #test: ["CMD-SHELL", "[ $$(nodetool statusgossip) = running ]"]
      test: ["CMD", "nodetool", "status"]
      interval: 15s
      timeout: 10s
      retries: 6
    networks:
      - cassandra_backend

  cassandra:
    image: "cassandra:4.0"
    depends_on:
      - cassandra-seed
    environment:
      - CASSANDRA_SEEDS=cassandra-seed
    healthcheck:
      #test: ["CMD-SHELL", "[ $$(nodetool statusgossip) = running ]"]
      test: ["CMD-SHELL", "nodetool", "status"]
      interval: 15s
      timeout: 10s
      retries: 6
    networks:
      - cassandra_backend

  hecuba:
    build: 
      context: .
    environment:
      - CONTACT_NAMES=cassandra-seed
    networks:
      - cassandra_backend
                                                                                                                              1,1           Top

docker-compose up -d brings all services up and cassandra nodes can be scaled, for example, as docker-compose up -d --scale cassandra=2

If you wish I can make a PR or wouldn't mind you doing it either.

[suvarchal]

A container for hecuba without needing to build arrow and usable with python upto 3.8 after #375 is merged.

this leverages binaries in conda repositories, could be a template for also installing hecuba without minimal building of dependencies.

Dockerfile

FROM continuumio/miniconda3:latest
RUN apt-get update && apt-get install -y \
  automake \
  build-essential \
  curl \
  cmake \
  git \
  libtool \
  libuv1 \
  m4 \
  && rm -rf /var/lib/apt/lists/*
# only upto python 3.8 is supported for arrow=0.15.1 
RUN conda install python=3.8
RUN conda install -c conda-forge \
  pyarrow=0.15.1 \
  tbb=2020.0 \
  zlib \
  && conda clean -afy

RUN git clone https://github.com/bsc-dd/hecuba.git
WORKDIR  /hecuba
ENV LD_LIBRARY_PATH=/opt/conda/lib
ENV C_INCLUDE_PATH=/opt/conda/include
RUN python setup.py install --c_binding=/opt/conda
WORKDIR  /
RUN rm -rf /hecuba