bitnami / containers

Bitnami container images
https://bitnami.com
Other
3.03k stars 4.38k forks source link

[bitnami/scylladb] Add --smp and --memory options for ScyllaDB Docker configuration #68051

Open siddjellali opened 1 week ago

siddjellali commented 1 week ago

Name and Version

bitnami/scylladb:6.0-debian-12

What architecture are you using?

None

What steps will reproduce the bug?

We are planning to deploy ScyllaDB in a production environment using Docker via Bitnami. However, we require the --smp and --memory options to properly configure the Docker environment. Currently, these options are not available in the configuration provided by Bitnami.

According to the official ScyllaDB documentation, these options are crucial for optimizing performance and managing resources in a production Docker environment. You can find detailed information in the Best Practices for ScyllaDB on Docker documentation.

  1. Attempt to deploy ScyllaDB using Bitnami's Docker configuration.
  2. Notice that options --smp and --memory are not available.

What is the expected behavior?

Bitnami should include the --smp and --memory options in their Docker configuration for ScyllaDB to align with best practices and ensure optimal performance and resource utilization.

What do you see instead?

None

Additional information

No response

siddjellali commented 1 week ago

I have tried to include scylla_cpuset_setup and scylla_memory_setup in the Dockerfile, but encountered issues. It seems that these parameters are being overridden by the Bitnami script, and these commands require root privileges to execute.

RUN  scylla_cpuset_setup --smp 8 && \
         scylla_memory_setup --memory 4G
siddjellali commented 1 week ago

Hello everyone, I've identified the missing configuration: it appears that when you run the scyllab binary, some flags are omitted.

#/opt/bitnami/scripts/scylladb/run.sh line 35
exec "$EXEC" "${flags[@]}" --cpuset 0-1 --smp 2 --memory 8G --io-properties-file /opt/bitnami/scylladb/etc/scylla.d/io_properties.yaml

scylla_io_setup needs to be executed at least once to create the io-properties file and necessary directories. Otherwise, the command won't run. I manually created the files below. SCYLLADB_IGNORE_INITDB_SCRIPTS should be set to 'yes' because a Scylla server dry run command tries to run without the correct flag.

Please find below DOCKERFILE to reproduce issue and workarroud.

Dockerfile

FROM bitnami/scylladb:6.0-debian-12
ENV SCYLLADB_DEVELOPER_MODE no
ENV SCYLLADB_IGNORE_INITDB_SCRIPTS yes
RUN mkdir -p /bitnami/scylladb/data/data && \
    mkdir -p /bitnami/scylladb/data/commitlog &&  \
    mkdir -p /bitnami/scylladb/data/hints && \
    mkdir -p /bitnami/scylladb/data/view_hints
COPY io.conf /opt/bitnami/scylladb/etc.default/scylla.d/io.conf
COPY io_properties.yaml /opt/bitnami/scylladb/etc.default/scylla.d/io_properties.yaml
COPY run.sh /opt/bitnami/scripts/scylladb/run.sh

run.sh

#!/bin/bash

# shellcheck disable=SC1091

set -o errexit
set -o nounset
set -o pipefail
# set -o xtrace # Uncomment this line for debugging purposes

# Load libraries
. /opt/bitnami/scripts/liblog.sh
. /opt/bitnami/scripts/libscylladb.sh
. /opt/bitnami/scripts/libos.sh

# Load ScyllaDB environment variables
. /opt/bitnami/scripts/scylladb-env.sh

info "** Starting ScyllaDB **"

EXEC="${DB_BIN_DIR}/scylla"

flags=("--options-file" "$DB_CONF_FILE")
if is_boolean_yes "$SCYLLADB_DEVELOPER_MODE"; then
    flags+=("--developer-mode" "true")
fi

# Add flags passed to this script
flags+=("$@")

info "** Starting $DB_FLAVOR **"
if am_i_root; then
    exec_as_user "$DB_DAEMON_USER" "$EXEC" "${flags[@]}"
else
    exec "$EXEC" "${flags[@]}" --cpuset 0-1 --smp 2 --memory 8G --io-properties-file /opt/bitnami/scylladb/etc/scylla.d/io_properties.yaml
fi

io.conf

SEASTAR_IO="--io-properties-file=/opt/bitnami/scylladb/etc/scylla.d/io_properties.yaml"

io_properties.yaml

disks:
  - mountpoint: /
    read_iops: 915756
    read_bandwidth: 8491704832
    write_iops: 3323
    write_bandwidth: 8449316352
javsalgar commented 1 week ago

Hi!

Thank you so much for the insight. Would you like to contribute with a PR including these settings?

siddjellali commented 1 week ago

I would love to, but I'm a novice when it comes to Bitnami container scripting and related tasks. The Scylla Bitnami container seems to be a duplicate of the Cassandra container, and I'm not fully proficient in how the functions should be orchestrated and implemented.

Do you know who created it? Perhaps they can provide assistance and help with testing for the community.

javsalgar commented 6 days ago

You can create the PR and the team will guide you on which elements should be changed. It will also perform several verifications as well.

siddjellali commented 8 hours ago

I do my best :)