Conversion .bat files

[drago87]

I have made (with the help of chatGPT) some .bat files to make it easier to convert the models Convert the initial source model (.safetensors file). Just drop the model on the .bat file

@echo off
REM Check if a file has been dropped
if "%~1"=="" (
    echo Please drag and drop a .safetensors model file onto this .bat file.
    pause
    exit /b
)

REM Extract the file path from the dropped file
set model_path=%~1

REM Use %~dp0 to get the directory of the .bat file (same as convert.py)
set script_path=%~dp0convert.py

REM Run the Python script with the model file
python "%script_path%" --src "%model_path%"

pause

quantize your model to Q4_K_S format (Drop the .GGUF file made from the above script onto this .bat file)

@echo off
REM Check if a file has been dropped
if "%~1"=="" (
    echo Please drag and drop a .GGUF model file onto this .bat file.
    pause
    exit /b
)

REM Extract the file path and name from the dropped file
set model_path=%~1

REM Extract the directory path and filename without extension
for %%f in ("%model_path%") do (
    set model_dir=%%~dpf
    set model_name=%%~nf
)

REM Check if the model name contains '-FP16', '-F16', or '-BF16'
echo %model_name% | findstr /c:"-FP16" /c:"-F16" /c:"-BF16" >nul
if %errorlevel% neq 0 (
    echo The model file must have '-FP16', '-F16', or '-BF16' in the name. Exiting...
    pause
    exit /b
)

REM Remove '-FP16', '-F16', or '-BF16' from the filename
set model_name_no_suffix=%model_name:-FP16=%
set model_name_no_suffix=%model_name_no_suffix:-F16=%
set model_name_no_suffix=%model_name_no_suffix:-BF16=%

REM Set the source and target model paths
set source_model=%model_dir%%model_name%.gguf
set target_model=%model_dir%%model_name_no_suffix%-Q4_K_S.gguf

REM Use %~dp0 to get the directory of the .bat file and construct path to llama-quantize.exe
set script_dir=%~dp0llama.cpp\build\bin\Debug\

REM Run the llama-quantize.exe command
"%script_dir%llama-quantize.exe" "%source_model%" "%target_model%" Q4_K_S

pause

Both files need to be in the \custom_nodes\ComfyUI-GGUF\tools folder

This is a experimental version of the second script that lets you select the quantize format (The formats was recommended by ChatGPT as i don't know what formats is usable)

@echo off
REM Check if a file has been dropped
if "%~1"=="" (
    echo Please drag and drop a .GGUF model file onto this .bat file.
    pause
    exit /b
)

REM Extract the file path and name from the dropped file
set model_path=%~1

REM Extract the directory path and filename without extension
for %%f in ("%model_path%") do (
    set model_dir=%%~dpf
    set model_name=%%~nf
)

REM Check if the model name contains '-FP16', '-F16', or '-BF16'
echo %model_name% | findstr /c:"-FP16" /c:"-F16" /c:"-BF16" >nul
if %errorlevel% neq 0 (
    echo The model file must have '-FP16', '-F16', or '-BF16' in the name. Exiting...
    pause
    exit /b
)

REM Remove '-FP16', '-F16', or '-BF16' from the filename
set model_name_no_suffix=%model_name:-FP16=%
set model_name_no_suffix=%model_name_no_suffix:-F16=%
set model_name_no_suffix=%model_name_no_suffix:-BF16=%

REM Display the list of quantize formats and get user selection
echo Please select a quantize format:
echo 1. Q4_K_S
echo 2. Q5_K_S
echo 3. Q8_K_S
set /p format_choice=Enter the number of your choice (1-3): 

REM Map user choice to quantize format
if "%format_choice%"=="1" set quantize_format=Q4_K_S
if "%format_choice%"=="2" set quantize_format=Q5_K_S
if "%format_choice%"=="3" set quantize_format=Q8_K_S

REM Check if the user provided a valid choice
if "%quantize_format%"=="" (
    echo Invalid choice. Exiting...
    pause
    exit /b
)

REM Set the source and target model paths
set source_model=%model_dir%%model_name%.gguf
set target_model=%model_dir%%model_name_no_suffix%-%quantize_format%.gguf

REM Use %~dp0 to get the directory of the .bat file and construct path to llama-quantize.exe
set script_dir=%~dp0llama.cpp\build\bin\Debug\

REM Run the llama-quantize.exe command
"%script_dir%llama-quantize.exe" "%source_model%" "%target_model%" %quantize_format%

pause

[drago87]

I have made a .bat file for installing the llama.cpp/gguf-py and applying the lcpp.patch then it compiles the llama-quantize binary. All .bat files should be in the ComfyUI-GGUF/tools folder to work

@echo off
REM Clone the repository
git clone https://github.com/ggerganov/llama.cpp
if %ERRORLEVEL% neq 0 (
    echo Git clone failed.
    exit /b %ERRORLEVEL%
)

REM Install the required Python package
pip install llama.cpp/gguf-py
if %ERRORLEVEL% neq 0 (
    echo Pip install failed.
    exit /b %ERRORLEVEL%
)

REM Change to the llama.cpp directory
cd llama.cpp
if not exist llama.cpp (
    echo Directory llama.cpp does not exist.
    exit /b 1
)

REM Checkout the specified tag
git checkout tags/b3600
if %ERRORLEVEL% neq 0 (
    echo Git checkout failed.
    exit /b %ERRORLEVEL%
)

REM Apply the patch
git apply ..\lcpp.patch
if %ERRORLEVEL% neq 0 (
    echo Git apply patch failed.
    exit /b %ERRORLEVEL%
)

REM Create a build directory and navigate into it
mkdir build
cd build

REM Run cmake to configure the build
cmake ..
if %ERRORLEVEL% neq 0 (
    echo CMake configuration failed.
    exit /b %ERRORLEVEL%
)

REM Build with cmake and compile llama-quantize target
cmake --build . --config Debug -j10 --target llama-quantize
if %ERRORLEVEL% neq 0 (
    echo CMake build failed.
    exit /b %ERRORLEVEL%
)

echo Build completed successfully.
pause