running via docker compose on Mac - issue with python git setup

[jexp]

I get this python git issue when running ragbuilder with docker compose up

The ragbuilder image didn't exited.

I fixed it by using GIT_PYTHON_REFRESH=quiet in the env file.

Perhaps git is missing from the image?

ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - Traceback (most recent call last):
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/git/__init__.py", line 296, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     refresh()
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/git/__init__.py", line 287, in refresh
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     if not Git.refresh(path=path):
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -            ^^^^^^^^^^^^^^^^^^^^^^
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/git/cmd.py", line 631, in refresh
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     raise ImportError(err)
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - ImportError: Bad git executable.
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - The git executable must be specified in one of the following ways:
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - be included in your $PATH
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - be set via $GIT_PYTHON_GIT_EXECUTABLE
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - explicitly set via git.refresh(<full-path-to-git-executable>)
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - All git commands will error until this is rectified.
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - This initial message can be silenced or aggravated in the future by setting the
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - $GIT_PYTHON_REFRESH environment variable. Use one of the following values:
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - quiet|q|silence|s|silent|none|n|0: for no message or exception
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - warn|w|warning|log|l|1: for a warning message (logging level CRITICAL, displayed by default)
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - error|e|exception|raise|r|2: for a raised exception
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - Example:
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     export GIT_PYTHON_REFRESH=quiet
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - The above exception was the direct cause of the following exception:
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - Traceback (most recent call last):
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/bin/ragbuilder", line 5, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from ragbuilder.ragbuilder import main
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/ragbuilder/ragbuilder.py", line 25, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from ragbuilder.executor import rag_builder, rag_builder_bayes_optmization, rag_builder_bayes_optimization_optuna, get_model_obj
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/ragbuilder/executor.py", line 59, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from ragatouille import RAGPretrainedModel
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/ragatouille/__init__.py", line 2, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from .RAGPretrainedModel import RAGPretrainedModel
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/ragatouille/RAGPretrainedModel.py", line 14, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from ragatouille.models import ColBERT, LateInteractionModel
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/ragatouille/models/__init__.py", line 2, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from .colbert import ColBERT
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/ragatouille/models/colbert.py", line 11, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from colbert import Trainer
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/colbert/__init__.py", line 1, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from .trainer import Trainer
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/colbert/trainer.py", line 1, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from colbert.infra.run import Run
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/colbert/infra/__init__.py", line 1, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from .run import *
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/colbert/infra/run.py", line 7, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from colbert.infra.config import RunConfig
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/colbert/infra/config/__init__.py", line 1, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from .config import *
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/colbert/infra/config/config.py", line 3, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from .base_config import BaseConfig
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/colbert/infra/config/base_config.py", line 13, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     from utility.utils.save_metadata import get_metadata_only
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/utility/utils/save_metadata.py", line 4, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     import git
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -   File "/usr/local/lib/python3.12/site-packages/git/__init__.py", line 298, in <module>
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     raise ImportError("Failed to initialize: {0}".format(_exc)) from _exc
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - ImportError: Failed to initialize: Bad git executable.
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - The git executable must be specified in one of the following ways:
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - be included in your $PATH
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - be set via $GIT_PYTHON_GIT_EXECUTABLE
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - explicitly set via git.refresh(<full-path-to-git-executable>)
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - All git commands will error until this is rectified.
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - This initial message can be silenced or aggravated in the future by setting the
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - $GIT_PYTHON_REFRESH environment variable. Use one of the following values:
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - quiet|q|silence|s|silent|none|n|0: for no message or exception
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - warn|w|warning|log|l|1: for a warning message (logging level CRITICAL, displayed by default)
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     - error|e|exception|raise|r|2: for a raised exception
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py - Example:
ragbuilder-1  | [INFO] 2024-10-05 22:23:20 - common.py -     export GIT_PYTHON_REFRESH=quiet
neo4j-1       | 2024-10-05 22:23:20.554+0000 INFO  ======== Neo4j 5.22.0 ========
ragbuilder-1 exited with code 1

[aravind10x]

@ashwinzyx can you please take a look once you're back?

[ibagur]

I faced the same issue when building the Docker container on Mac and I fixed it adding 'git' to the list of system dependencies in the Dockerfile:

# Install required system dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    libjpeg-dev \
    libmagic-dev \
    zlib1g-dev \
    libopenjp2-7-dev \
    libpng-dev \
    libpoppler-cpp-dev \
    pkg-config \
    gcc \
    libqpdf-dev \
    libgl1-mesa-glx \
    libglib2.0-0 \
    git \
    && rm -rf /var/lib/apt/lists/*

Also added the GIT_PYTHON_REFRESH=quiet flag in the docker-compose.yml