Dev-staging is failing full integration test

biosafetylvl5 commented 1 month ago

Description

[ In 2-3 sentences briefly describe the required bug fix or feature with this issue. ]

Code to demonstrate issue

Dockerfile

FROM python:3.10-bullseye

RUN apt-get update && apt-get -y upgrade \ 
    && apt-get install -y wget git libopenblas-dev imagemagick g++ make gfortran \
    && apt-get update && apt-get install -y software-properties-common \
    && add-apt-repository -y ppa:ubuntugis/ppa \
    && apt-get install -y gdal-bin libgdal-dev \
    && pip install -U pip
ARG GEOIPS_PACKAGES_DIR=/app/geoips_packages

WORKDIR $GEOIPS_PACKAGES_DIR

ARG USER=geoips_user
# ARG GROUP=${USER}
ARG USER_ID=25000
ARG GROUP_ID=25000

ARG GEOIPS_REPO_URL=https://github.com/NRLMMD-GEOIPS/
ARG GEOIPS_OUTDIRS=/output
ARG GEOIPS_DEPENDENCIES_DIR=/data/depends
ARG GEOIPS_TESTDATA_DIR=/data/testdata/long-test-dir-name

RUN getent group ${GROUP_ID} > /dev/null || groupadd -g ${GROUP_ID} ${USER} \
    && useradd --gid ${GROUP_ID} --uid ${USER_ID} -l -m ${USER} \
    && mkdir -p $GEOIPS_OUTDIRS $GEOIPS_DEPENDENCIES_DIR $GEOIPS_TESTDATA_DIR /data \
    && chown ${USER_ID}:${GROUP_ID} $GEOIPS_OUTDIRS $GEOIPS_DEPENDENCIES_DIR $GEOIPS_TESTDATA_DIR $GEOIPS_PACKAGES_DIR

USER ${USER}

ENV PATH=${PATH}:/home/${USER}/.local/bin:${GEOIPS_DEPENDENCIES_DIR}/bin
ENV GEOIPS_REPO_URL=${GEOIPS_REPO_URL}
ENV GEOIPS_OUTDIRS=${GEOIPS_OUTDIRS}
ENV GEOIPS_PACKAGES_DIR=${GEOIPS_PACKAGES_DIR}
ENV CARTOPY_DATA_DIR=${GEOIPS_PACKAGES_DIR}
ENV GEOIPS_DEPENDENCIES_DIR=${GEOIPS_DEPENDENCIES_DIR}
ENV GEOIPS_TESTDATA_DIR=${GEOIPS_TESTDATA_DIR}

WORKDIR $GEOIPS_PACKAGES_DIR

COPY --chown=${USER_ID}:${GROUP_ID} ./environments/requirements.txt ${GEOIPS_PACKAGES_DIR}/requirements.txt

RUN python3 -m pip install rasterio \
    && python3 -m pip install numpy==1.26.4 && python3 -m pip install -r requirements.txt 
USER root
RUN git clone --single-branch -b 3-build-fails-up-to-date-setuptools https://github.com/NRLMMD-GEOIPS/akima86.git && cd akima86 && python3 setup.py install
USER ${USER}

COPY --chown=${USER_ID}:${GROUP_ID} . ${GEOIPS_PACKAGES_DIR}/geoips

WORKDIR ${GEOIPS_PACKAGES_DIR}/geoips
RUN cd ${GEOIPS_PACKAGES_DIR}/geoips \
    && pip install -e "$GEOIPS_PACKAGES_DIR/geoips[test, doc]" \
    && create_plugin_registries

Install Log

geoips_user@91614884ba05:/app/geoips_packages/geoips$ ./tests/integration_tests/full_install.sh 

Install log: /output/logs/20240927.155446_install.log

Install log: /output/logs/20240927.155446_install.log

Install log: /output/logs/20240927.155446_install.log
SUCCESS: 'git' appears to be installed successfully
    /usr/bin/git

Install log: /output/logs/20240927.155446_install.log
SUCCESS: 'python' appears to be installed successfully
    /usr/local/bin/python

Install log: /output/logs/20240927.155446_install.log
SUCCESS: 'gcc' appears to be installed successfully
    /usr/bin/gcc

Install log: /output/logs/20240927.155446_install.log
SUCCESS: 'gfortran' appears to be installed successfully
    /usr/bin/gfortran

Install log: /output/logs/20240927.155446_install.log
SUCCESS: 'g++' appears to be installed successfully
    /usr/bin/g++

Install log: /output/logs/20240927.155446_install.log
SUCCESS: 'scipy/openblas' appear to be installed successfully

Install log: /output/logs/20240927.155446_install.log
SUCCESS: cartopy_shapefiles appear to be successfully installed
         if you are having comparison errors, you can try
         deleting the contents of $CARTOPY_DATA_DIR, and
         rerunning check_system_requirements.sh cartopy_shapefiles
drwxr-xr-x 2 geoips_user geoips_user 61440 Sep 27 15:35 /app/geoips_packages/shapefiles/natural_earth/cultural
drwxr-xr-x 2 geoips_user geoips_user 40960 Sep 27 15:35 /app/geoips_packages/shapefiles/natural_earth/physical

Install log: /output/logs/20240927.155446_install.log
SUCCESS: source repo geoips appears to be installed successfully
    drwxr-xr-x 1 geoips_user geoips_user 4096 Sep 27 15:26 /app/geoips_packages/geoips

Install log: /output/logs/20240927.155446_install.log
SUCCESS: source repo data_fusion appears to be installed successfully
    drwxr-xr-x 7 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/data_fusion

Install log: /output/logs/20240927.155446_install.log
SUCCESS: source repo recenter_tc appears to be installed successfully
    drwxr-xr-x 7 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/recenter_tc

Install log: /output/logs/20240927.155446_install.log
SUCCESS: source repo geoips_clavrx appears to be installed successfully
    drwxr-xr-x 6 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/geoips_clavrx

Install log: /output/logs/20240927.155446_install.log
SUCCESS: source repo geoips_plugin_example appears to be installed successfully
    drwxr-xr-x 7 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/geoips_plugin_example

Install log: /output/logs/20240927.155446_install.log
SUCCESS: source repo template_basic_plugin appears to be installed successfully
    drwxr-xr-x 7 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/template_basic_plugin

Install log: /output/logs/20240927.155446_install.log
SUCCESS: settings repo .vscode appears to be installed successfully
    drwxr-xr-x 5 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/.vscode

Install log: /output/logs/20240927.155446_install.log
SUCCESS: tests data repo test_data_noaa_aws appears to be installed successfully
    drwx------ 3 geoips_user geoips_user 4096 Sep 27 15:35 /data/testdata/long-test-dir-name/test_data_noaa_aws

Install log: /output/logs/20240927.155446_install.log
SUCCESS: tests data repo test_data_amsr2 appears to be installed successfully
    drwxrwxr-x 5 geoips_user geoips_user 4096 May 3 01:14 /data/testdata/long-test-dir-name/test_data_amsr2

Install log: /output/logs/20240927.155446_install.log
SUCCESS: tests data repo test_data_clavrx appears to be installed successfully
    drwxrwxr-x 4 geoips_user geoips_user 4096 May 3 01:18 /data/testdata/long-test-dir-name/test_data_clavrx

Install log: /output/logs/20240927.155446_install.log
SUCCESS: tests data repo test_data_gpm appears to be installed successfully
    drwxrwxr-x 5 geoips_user geoips_user 4096 May 3 01:22 /data/testdata/long-test-dir-name/test_data_gpm

Install log: /output/logs/20240927.155446_install.log
SUCCESS: tests data repo test_data_sar appears to be installed successfully
    drwxrwxr-x 6 geoips_user geoips_user 4096 May 2 22:00 /data/testdata/long-test-dir-name/test_data_sar

Install log: /output/logs/20240927.155446_install.log
SUCCESS: tests data repo test_data_scat appears to be installed successfully
    drwxrwxr-x 6 geoips_user geoips_user 4096 Sep 25 2023 /data/testdata/long-test-dir-name/test_data_scat

Install log: /output/logs/20240927.155446_install.log
SUCCESS: tests data repo test_data_smap appears to be installed successfully
    drwxrwxr-x 5 geoips_user geoips_user 4096 May 3 12:45 /data/testdata/long-test-dir-name/test_data_smap

Install log: /output/logs/20240927.155446_install.log
SUCCESS: tests data repo test_data_viirs appears to be installed successfully
    drwxrwxr-x 4 geoips_user geoips_user 4096 May 3 12:59 /data/testdata/long-test-dir-name/test_data_viirs

Install log: /output/logs/20240927.155446_install.log
Installing tests data repo test_data_fusion .... 
  /data/testdata/long-test-dir-name/test_data_fusion/ using direct download via cira
DOWNLOADING: NextCloud Dataset test_data_fusion
Recognized test data set 
Trying to download from url https://io2.cira.colostate.edu/s/J73tEcn22smktMi/download?path=%2F&files=test_data_fusion.tgz
Downloading and extracting https://io2.cira.colostate.edu/s/J73tEcn22smktMi/download?path=%2F&files=test_data_fusion.tgz to /data/testdata/long-test-dir-name...
File is 2.7 GiB... 
Success. Files downloaded and extracted.
SUCCESS: Decompressed test_data_fusion
27_155650 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/template_basic_plugin/my_package/registered_plugins.json
27_155650 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/geoips_plugin_example/geoips_plugin_example/registered_plugins.json
27_155651 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/data_fusion/data_fusion/registered_plugins.json
27_155651 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/recenter_tc/recenter_tc/registered_plugins.json
27_155651 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/geoips_clavrx/geoips_clavrx/registered_plugins.json
27_155651 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'tc_file_naming' from 'geoips', interface_name is 'None'
27_155651 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'remss_reader' from 'geoips', interface_name is 'None'
27_155651 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'geostationary_geolocation' from 'geoips', interface_name is 'None'
27_155651 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'hrit_reader' from 'geoips', interface_name is 'None'
27_155651 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'interp_pyresample' from 'geoips', interface_name is 'None'
27_155651 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'interp_scipy' from 'geoips', interface_name is 'None'
27_155651 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'boxdefinitions' from 'geoips', interface_name is 'None'
27_155651 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/geoips/geoips/registered_plugins.json

Run Log

geoips_user@91614884ba05:/app/geoips_packages/geoips$ ./tests/integration_tests/full_test.sh 

Install log: /output/logs/20240927.155718_install.log

Install log: /output/logs/20240927.155718_install.log

Install log: /output/logs/20240927.155718_install.log
SUCCESS: 'git' appears to be installed successfully
    /usr/bin/git

Install log: /output/logs/20240927.155718_install.log
SUCCESS: 'python' appears to be installed successfully
    /usr/local/bin/python

Install log: /output/logs/20240927.155718_install.log
SUCCESS: 'gcc' appears to be installed successfully
    /usr/bin/gcc

Install log: /output/logs/20240927.155718_install.log
SUCCESS: 'gfortran' appears to be installed successfully
    /usr/bin/gfortran

Install log: /output/logs/20240927.155718_install.log
SUCCESS: 'g++' appears to be installed successfully
    /usr/bin/g++

Install log: /output/logs/20240927.155718_install.log
SUCCESS: 'scipy/openblas' appear to be installed successfully

Install log: /output/logs/20240927.155718_install.log
SUCCESS: cartopy_shapefiles appear to be successfully installed
         if you are having comparison errors, you can try
         deleting the contents of $CARTOPY_DATA_DIR, and
         rerunning check_system_requirements.sh cartopy_shapefiles
drwxr-xr-x 2 geoips_user geoips_user 61440 Sep 27 15:35 /app/geoips_packages/shapefiles/natural_earth/cultural
drwxr-xr-x 2 geoips_user geoips_user 40960 Sep 27 15:35 /app/geoips_packages/shapefiles/natural_earth/physical

Install log: /output/logs/20240927.155718_install.log
SUCCESS: source repo geoips appears to be installed successfully
    drwxr-xr-x 1 geoips_user geoips_user 4096 Sep 27 15:26 /app/geoips_packages/geoips

Install log: /output/logs/20240927.155718_install.log
SUCCESS: source repo data_fusion appears to be installed successfully
    drwxr-xr-x 7 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/data_fusion

Install log: /output/logs/20240927.155718_install.log
SUCCESS: source repo recenter_tc appears to be installed successfully
    drwxr-xr-x 7 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/recenter_tc

Install log: /output/logs/20240927.155718_install.log
SUCCESS: source repo geoips_clavrx appears to be installed successfully
    drwxr-xr-x 6 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/geoips_clavrx

Install log: /output/logs/20240927.155718_install.log
SUCCESS: source repo geoips_plugin_example appears to be installed successfully
    drwxr-xr-x 7 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/geoips_plugin_example

Install log: /output/logs/20240927.155718_install.log
SUCCESS: source repo template_basic_plugin appears to be installed successfully
    drwxr-xr-x 7 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/template_basic_plugin

Install log: /output/logs/20240927.155718_install.log
SUCCESS: settings repo .vscode appears to be installed successfully
    drwxr-xr-x 5 geoips_user geoips_user 4096 Sep 27 15:35 /app/geoips_packages/.vscode

Install log: /output/logs/20240927.155718_install.log
SUCCESS: tests data repo test_data_noaa_aws appears to be installed successfully
    drwx------ 3 geoips_user geoips_user 4096 Sep 27 15:35 /data/testdata/long-test-dir-name/test_data_noaa_aws

Install log: /output/logs/20240927.155718_install.log
SUCCESS: tests data repo test_data_amsr2 appears to be installed successfully
    drwxrwxr-x 5 geoips_user geoips_user 4096 May 3 01:14 /data/testdata/long-test-dir-name/test_data_amsr2

Install log: /output/logs/20240927.155718_install.log
SUCCESS: tests data repo test_data_clavrx appears to be installed successfully
    drwxrwxr-x 4 geoips_user geoips_user 4096 May 3 01:18 /data/testdata/long-test-dir-name/test_data_clavrx

Install log: /output/logs/20240927.155718_install.log
SUCCESS: tests data repo test_data_gpm appears to be installed successfully
    drwxrwxr-x 5 geoips_user geoips_user 4096 May 3 01:22 /data/testdata/long-test-dir-name/test_data_gpm

Install log: /output/logs/20240927.155718_install.log
SUCCESS: tests data repo test_data_sar appears to be installed successfully
    drwxrwxr-x 6 geoips_user geoips_user 4096 May 2 22:00 /data/testdata/long-test-dir-name/test_data_sar

Install log: /output/logs/20240927.155718_install.log
SUCCESS: tests data repo test_data_scat appears to be installed successfully
    drwxrwxr-x 6 geoips_user geoips_user 4096 Sep 25 2023 /data/testdata/long-test-dir-name/test_data_scat

Install log: /output/logs/20240927.155718_install.log
SUCCESS: tests data repo test_data_smap appears to be installed successfully
    drwxrwxr-x 5 geoips_user geoips_user 4096 May 3 12:45 /data/testdata/long-test-dir-name/test_data_smap

Install log: /output/logs/20240927.155718_install.log
SUCCESS: tests data repo test_data_viirs appears to be installed successfully
    drwxrwxr-x 4 geoips_user geoips_user 4096 May 3 12:59 /data/testdata/long-test-dir-name/test_data_viirs

Install log: /output/logs/20240927.155718_install.log
SUCCESS: tests data repo test_data_fusion appears to be installed successfully
    drwxr-xr-x 3 geoips_user geoips_user 4096 May 8 2023 /data/testdata/long-test-dir-name/test_data_fusion
27_155718 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/template_basic_plugin/my_package/registered_plugins.json
27_155718 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/geoips_plugin_example/geoips_plugin_example/registered_plugins.json
27_155719 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/data_fusion/data_fusion/registered_plugins.json
27_155719 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/recenter_tc/recenter_tc/registered_plugins.json
27_155719 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/geoips_clavrx/geoips_clavrx/registered_plugins.json
27_155719 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'tc_file_naming' from 'geoips', interface_name is 'None'
27_155719 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'remss_reader' from 'geoips', interface_name is 'None'
27_155719 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'geostationary_geolocation' from 'geoips', interface_name is 'None'
27_155719 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'hrit_reader' from 'geoips', interface_name is 'None'
27_155719 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'interp_pyresample' from 'geoips', interface_name is 'None'
27_155719 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'interp_scipy' from 'geoips', interface_name is 'None'
27_155719 create_plugin_registries.py:820  INTERACTIVE: Skipping module 'boxdefinitions' from 'geoips', interface_name is 'None'
27_155719 create_plugin_registries.py:318  INTERACTIVE: Writing /app/geoips_packages/geoips/geoips/registered_plugins.json

Fri Sep 27 15:57:19 UTC 2024 Running pre, log: /output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log

Fri Sep 27 15:57:19 UTC 2024  Running export
export GEOIPS_DISABLE_SHARED_CODE_CHECKS=False
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_export__GEOIPS_DISABLE_SHARED_CODE_CHECKS=False.log
        Return: 0

Fri Sep 27 15:57:19 UTC 2024  Running check_code.sh
./tests/integration_tests/../..//tests/utils/check_code.sh all ./tests/integration_tests/../../
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_check_code.sh__all_._tests_integration_tests_.._.._.log
        Return: 0

Fri Sep 27 15:58:35 UTC 2024  Running export
export GEOIPS_DISABLE_SHARED_CODE_CHECKS=True
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_export__GEOIPS_DISABLE_SHARED_CODE_CHECKS=True.log
        Return: 0

Fri Sep 27 15:58:35 UTC 2024  Running check_code.sh
./tests/integration_tests/../..//tests/utils/check_code.sh all /app/geoips_packages/recenter_tc
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_check_code.sh__all__app_geoips_packages_recenter_tc.log
        Return: 0

Fri Sep 27 15:58:35 UTC 2024  Running build_docs.sh
./tests/integration_tests/../..//docs/build_docs.sh ./tests/integration_tests/../../ geoips html_only
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_build_docs.sh__._tests_integration_tests_.._..__geoips_html_only.log
        Return: 0

Fri Sep 27 15:58:56 UTC 2024  Running build_docs.sh
./tests/integration_tests/../..//docs/build_docs.sh /app/geoips_packages/recenter_tc recenter_tc html_only
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_build_docs.sh___app_geoips_packages_recenter_tc_recenter_tc_html_only.log
        Return: 0

Fri Sep 27 15:58:59 UTC 2024  Running build_docs.sh
./tests/integration_tests/../..//docs/build_docs.sh /app/geoips_packages/data_fusion data_fusion html_only
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_build_docs.sh___app_geoips_packages_data_fusion_data_fusion_html_only.log
        Return: 0

Fri Sep 27 15:59:01 UTC 2024  Running build_docs.sh
./tests/integration_tests/../..//docs/build_docs.sh /app/geoips_packages/template_basic_plugin my_package html_only
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_build_docs.sh___app_geoips_packages_template_basic_plugin_my_package_html_only.log
        Return: 0

Fri Sep 27 15:59:02 UTC 2024  Running abi.static.Infrared.imagery_annotated.sh
./tests/integration_tests/../..//tests/scripts/abi.static.Infrared.imagery_annotated.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_abi.static.Infrared.imagery_annotated.sh.log
        Return: 0

Fri Sep 27 15:59:37 UTC 2024  Running console_script_create_sector_image.sh
./tests/integration_tests/../..//tests/scripts/console_script_create_sector_image.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_console_script_create_sector_image.sh.log
        Return: 0

Fri Sep 27 15:59:40 UTC 2024  Running console_script_list_available_plugins.sh
./tests/integration_tests/../..//tests/scripts/console_script_list_available_plugins.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_console_script_list_available_plugins.sh.log
        Return: 0

Fri Sep 27 15:59:45 UTC 2024  Running abi.static.Visible.imagery_annotated.sh
./tests/integration_tests/../..//tests/scripts/abi.static.Visible.imagery_annotated.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_abi.static.Visible.imagery_annotated.sh.log
        Return: 1

Fri Sep 27 16:00:31 UTC 2024  Running abi.config_based_output_low_memory.sh
./tests/integration_tests/../..//tests/scripts/abi.config_based_output_low_memory.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_abi.config_based_output_low_memory.sh.log
        Return: 1

Fri Sep 27 16:00:43 UTC 2024  Running abi.config_based_output.sh
./tests/integration_tests/../..//tests/scripts/abi.config_based_output.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_abi.config_based_output.sh.log
        Return: 1

Fri Sep 27 16:01:03 UTC 2024  Running ahi.tc.WV.geotiff.sh
./tests/integration_tests/../..//tests/scripts/ahi.tc.WV.geotiff.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ahi.tc.WV.geotiff.sh.log
        Return: 1

Fri Sep 27 16:01:04 UTC 2024  Running ami.static.Infrared.imagery_annotated.sh
./tests/integration_tests/../..//tests/scripts/ami.static.Infrared.imagery_annotated.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ami.static.Infrared.imagery_annotated.sh.log
        Return: 1

Fri Sep 27 16:01:05 UTC 2024  Running ami.static.Visible.imagery_annotated.sh
./tests/integration_tests/../..//tests/scripts/ami.static.Visible.imagery_annotated.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ami.static.Visible.imagery_annotated.sh.log
        Return: 1

Fri Sep 27 16:01:07 UTC 2024  Running ami.tc.WV.geotiff.sh
./tests/integration_tests/../..//tests/scripts/ami.tc.WV.geotiff.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ami.tc.WV.geotiff.sh.log
        Return: 1

Fri Sep 27 16:01:08 UTC 2024  Running ami.WV-Upper.unprojected_image.sh
./tests/integration_tests/../..//tests/scripts/ami.WV-Upper.unprojected_image.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ami.WV-Upper.unprojected_image.sh.log
        Return: 1

Fri Sep 27 16:01:09 UTC 2024  Running amsr2.tc.89H-Physical.imagery_annotated.sh
./tests/integration_tests/../..//tests/scripts/amsr2.tc.89H-Physical.imagery_annotated.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_amsr2.tc.89H-Physical.imagery_annotated.sh.log
        Return: 1

Fri Sep 27 16:01:16 UTC 2024  Running amsr2_ocean.tc.windspeed.imagery_clean.sh
./tests/integration_tests/../..//tests/scripts/amsr2_ocean.tc.windspeed.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_amsr2_ocean.tc.windspeed.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:01:19 UTC 2024  Running amsr2.config_based_no_compare.sh
./tests/integration_tests/../..//tests/scripts/amsr2.config_based_no_compare.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_amsr2.config_based_no_compare.sh.log
        Return: 1

Fri Sep 27 16:01:25 UTC 2024  Running amsr2.config_based_overlay_output.sh
./tests/integration_tests/../..//tests/scripts/amsr2.config_based_overlay_output.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_amsr2.config_based_overlay_output.sh.log
        Return: 1

Fri Sep 27 16:01:30 UTC 2024  Running amsr2.config_based_overlay_output_low_memory.sh
./tests/integration_tests/../..//tests/scripts/amsr2.config_based_overlay_output_low_memory.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_amsr2.config_based_overlay_output_low_memory.sh.log
        Return: 1

Fri Sep 27 16:01:36 UTC 2024  Running ascat_knmi.tc.windbarbs.imagery_windbarbs_clean.sh
./tests/integration_tests/../..//tests/scripts/ascat_knmi.tc.windbarbs.imagery_windbarbs_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ascat_knmi.tc.windbarbs.imagery_windbarbs_clean.sh.log
        Return: 1

Fri Sep 27 16:01:38 UTC 2024  Running ascat_low_knmi.tc.windbarbs.imagery_windbarbs.sh
./tests/integration_tests/../..//tests/scripts/ascat_low_knmi.tc.windbarbs.imagery_windbarbs.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ascat_low_knmi.tc.windbarbs.imagery_windbarbs.sh.log
        Return: 1

Fri Sep 27 16:01:40 UTC 2024  Running ascat_noaa_25km.tc.windbarbs.imagery_windbarbs.sh
./tests/integration_tests/../..//tests/scripts/ascat_noaa_25km.tc.windbarbs.imagery_windbarbs.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ascat_noaa_25km.tc.windbarbs.imagery_windbarbs.sh.log
        Return: 1

Fri Sep 27 16:01:43 UTC 2024  Running ascat_noaa_50km.tc.wind-ambiguities.imagery_windbarbs.sh
./tests/integration_tests/../..//tests/scripts/ascat_noaa_50km.tc.wind-ambiguities.imagery_windbarbs.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ascat_noaa_50km.tc.wind-ambiguities.imagery_windbarbs.sh.log
        Return: 1

Fri Sep 27 16:01:47 UTC 2024  Running ascat_uhr.tc.wind-ambiguities.imagery_windbarbs.sh
./tests/integration_tests/../..//tests/scripts/ascat_uhr.tc.wind-ambiguities.imagery_windbarbs.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ascat_uhr.tc.wind-ambiguities.imagery_windbarbs.sh.log
        Return: 1

Fri Sep 27 16:01:52 UTC 2024  Running ascat_uhr.tc.nrcs.imagery_clean.sh
./tests/integration_tests/../..//tests/scripts/ascat_uhr.tc.nrcs.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ascat_uhr.tc.nrcs.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:01:55 UTC 2024  Running ascat_uhr.tc.windbarbs.imagery_windbarbs.sh
./tests/integration_tests/../..//tests/scripts/ascat_uhr.tc.windbarbs.imagery_windbarbs.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ascat_uhr.tc.windbarbs.imagery_windbarbs.sh.log
        Return: 1

Fri Sep 27 16:01:59 UTC 2024  Running ascat_uhr.tc.windspeed.imagery_clean.sh
./tests/integration_tests/../..//tests/scripts/ascat_uhr.tc.windspeed.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ascat_uhr.tc.windspeed.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:01 UTC 2024  Running cli_dummy_script.sh
./tests/integration_tests/../..//tests/scripts/cli_dummy_script.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_cli_dummy_script.sh.log
        Return: 0

Fri Sep 27 16:02:01 UTC 2024  Running gmi.tc.89pct.imagery_clean.sh
./tests/integration_tests/../..//tests/scripts/gmi.tc.89pct.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_gmi.tc.89pct.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:04 UTC 2024  Running imerg.tc.Rain.imagery_clean.sh
./tests/integration_tests/../..//tests/scripts/imerg.tc.Rain.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_imerg.tc.Rain.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:06 UTC 2024  Running oscat_knmi.tc.windbarbs.imagery_windbarbs.sh
./tests/integration_tests/../..//tests/scripts/oscat_knmi.tc.windbarbs.imagery_windbarbs.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_oscat_knmi.tc.windbarbs.imagery_windbarbs.sh.log
        Return: 1

Fri Sep 27 16:02:08 UTC 2024  Running sar.tc.nrcs.imagery_annotated.sh
./tests/integration_tests/../..//tests/scripts/sar.tc.nrcs.imagery_annotated.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_sar.tc.nrcs.imagery_annotated.sh.log
        Return: 1

Fri Sep 27 16:02:09 UTC 2024  Running smap.unsectored.text_winds.sh
./tests/integration_tests/../..//tests/scripts/smap.unsectored.text_winds.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_smap.unsectored.text_winds.sh.log
        Return: 1

Fri Sep 27 16:02:10 UTC 2024  Running viirsday.tc.Night-Vis-IR.imagery_annotated.sh
./tests/integration_tests/../..//tests/scripts/viirsday.tc.Night-Vis-IR.imagery_annotated.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_viirsday.tc.Night-Vis-IR.imagery_annotated.sh.log
        Return: 1

Fri Sep 27 16:02:17 UTC 2024  Running viirsmoon.tc.Night-Vis-GeoIPS1.imagery_clean.sh
./tests/integration_tests/../..//tests/scripts/viirsmoon.tc.Night-Vis-GeoIPS1.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_viirsmoon.tc.Night-Vis-GeoIPS1.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:21 UTC 2024  Running viirsclearnight.Night-Vis-IR-GeoIPS1.unprojected_image.sh
./tests/integration_tests/../..//tests/scripts/viirsclearnight.Night-Vis-IR-GeoIPS1.unprojected_image.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_viirsclearnight.Night-Vis-IR-GeoIPS1.unprojected_image.sh.log
        Return: 1

Fri Sep 27 16:02:38 UTC 2024  Running abi.tc.Visible.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/abi.tc.Visible.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_abi.tc.Visible.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:39 UTC 2024  Running amsr2.tc.color37.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/amsr2.tc.color37.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_amsr2.tc.color37.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:45 UTC 2024  Running amsr2.tc.windspeed.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/amsr2.tc.windspeed.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_amsr2.tc.windspeed.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:48 UTC 2024  Running ascat_uhr.tc.nrcs.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/ascat_uhr.tc.nrcs.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ascat_uhr.tc.nrcs.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:49 UTC 2024  Running ascat_uhr.tc.windbarbs.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/ascat_uhr.tc.windbarbs.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_ascat_uhr.tc.windbarbs.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:52 UTC 2024  Running imerg.tc.Rain.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/imerg.tc.Rain.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_imerg.tc.Rain.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:54 UTC 2024  Running metopc_knmi_125.tc.windbarbs.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/metopc_knmi_125.tc.windbarbs.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_metopc_knmi_125.tc.windbarbs.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:55 UTC 2024  Running oscat.tc.windspeed.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/oscat.tc.windspeed.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_oscat.tc.windspeed.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:56 UTC 2024  Running sar.tc.nrcs.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/sar.tc.nrcs.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_sar.tc.nrcs.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:57 UTC 2024  Running smap.tc.windspeed.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/smap.tc.windspeed.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_smap.tc.windspeed.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:02:58 UTC 2024  Running viirs.tc.Infrared-Gray.imagery_clean.sh
/app/geoips_packages/recenter_tc/tests/scripts/viirs.tc.Infrared-Gray.imagery_clean.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_viirs.tc.Infrared-Gray.imagery_clean.sh.log
        Return: 1

Fri Sep 27 16:03:08 UTC 2024  Running test_all.sh
/app/geoips_packages/template_basic_plugin/tests/test_all.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_test_all.sh.log
./tests/integration_tests/../..//tests/utils/test_all_run.sh: line 24: /output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_test_all.sh.log: No space left on device
        Return: 1

Fri Sep 27 16:03:08 UTC 2024  Running test_all.sh
/app/geoips_packages/geoips_plugin_example/tests/test_all.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_test_all.sh.log
./tests/integration_tests/../..//tests/utils/test_all_run.sh: line 24: /output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_test_all.sh.log: No space left on device
        Return: 1

Fri Sep 27 16:03:08 UTC 2024  Running test_all.sh
/app/geoips_packages/geoips_clavrx/tests/test_all.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_test_all.sh.log
./tests/integration_tests/../..//tests/utils/test_all_run.sh: line 24: /output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_test_all.sh.log: No space left on device
        Return: 1

Fri Sep 27 16:03:08 UTC 2024  Running test_all.sh
/app/geoips_packages/data_fusion/tests/test_all.sh
/output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_test_all.sh.log
./tests/integration_tests/../..//tests/utils/test_all_run.sh: line 24: /output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log_test_all.sh.log: No space left on device
        Return: 1

Fri Sep 27 16:03:08 UTC 2024  Running post, final results in /output/logs/20240927/20240927.155719_geoips_full/test_all_geoips_full.log

Package: geoips_full
Total run time: 349 seconds
Number data types run: 56
Number data types failed: 44
Fri Sep 27 16:03:08 UTC 2024

jsolbrig commented 1 month ago

I just ran base_test.sh to see what that got me to start with. I get one failure from that in a YAML file diff.

27_162923    log_setup.py:50   INTERACTIVE: ********************************************************************************
27_162923    log_setup.py:55   INTERACTIVE: ** BAD Text files do NOT match exactly                                        **
27_162923    log_setup.py:55   INTERACTIVE: ** output_product:                                                            **
27_162923    log_setup.py:55   INTERACTIVE: ** /home/jsolbrig/geoips/review/outdirs/preprocessed/tcwww/tc2020/IO/IO012020/metadata/sector_information/20200518/20200518_073601_IO012020_amsr2_gcom-w1_windspeed_140kts_85p45_1p0-clean.png.yaml **
27_162923    log_setup.py:55   INTERACTIVE: ** compare_product:                                                           **
27_162923    log_setup.py:55   INTERACTIVE: ** /home/jsolbrig/geoips/review/geoips/tests/outputs/amsr2_ocean.tc.windspeed.imagery_clean/20200518_073601_IO012020_amsr2_gcom-w1_windspeed_140kts_85p45_1p0-clean.png.yaml **
27_162923    log_setup.py:55   INTERACTIVE: ** out_difftxt:                                                               **
27_162923    log_setup.py:55   INTERACTIVE: ** /home/jsolbrig/geoips/review/geoips/tests/outputs/amsr2_ocean.tc.windspeed.imagery_clean/diff_test_output_dir_jsolbrig/diff_test_output_20200518_073601_IO012020_amsr2_gcom-w1_windspeed_140kts_85p45_1p0-clean.png.yaml **
27_162923    log_setup.py:56   INTERACTIVE: ********************************************************************************
27_162923    log_setup.py:57   INTERACTIVE:

The difference is this:

25,26c25,26
< source_filename: /home/jsolbrig/geoips/review/geoips/tests/sectors/tc_bdecks/bio012020.dat
< source_sector_file: /home/jsolbrig/geoips/review/geoips/tests/sectors/tc_bdecks/bio012020.dat
---
> source_filename: $GEOIPS_PACKAGES_DIR/geoips/tests/sectors/tc_bdecks/bio012020.dat
> source_sector_file: $GEOIPS_PACKAGES_DIR/geoips/tests/sectors/tc_bdecks/bio012020.dat

Those paths point to the same thing but differ in whether the ENV is expanded. The newly produced file expands the ENV while the file used for comparison does not expand the ENV.

biosafetylvl5 commented 1 month ago

On both main and dev-staging, the PATHS variable in filenames/base_paths.py is about the same:

(geoips) [gwynu@winter geoips]$ cd geoips/filenames/
(geoips) [gwynu@winter filenames]$ python3 -c "import base_paths as b; print(b.PATHS)"
{'BASE_PATH': '/local/home/gwynu/geoips/geoips/geoips', 'GEOIPS_OUTDIRS': '/home/gwynu/geoips/outdirs', 'GEOIPS_PACKAGES_DIR': '/local/home/gwynu/geoips', 'GEOIPS_BASEDIR': '/local/home/gwynu', 'GEOIPS_DOCS_URL': 'https://nrlmmd-geoips.github.io/geoips/', 'GEOIPS_VERS': '0.0.0', 'GEOIPS_OPERATIONAL_USER': False, 'GEOIPS_COPYRIGHT': 'NRL-Monterey', 'GEOIPS_COPYRIGHT_ABBREVIATED': 'NRLMRY', 'GEOIPS_RCFILE': '', 'DEFAULT_QUEUE': None, 'BOXNAME': 'winter.cira.colostate.edu', 'GEOIPS_TESTDATA_DIR': '/home/gwynu/geoips/test_data', 'GEOIPS_DEPENDENCIES_DIR': '/local/home/gwynu/geoips_dependencies', 'PRESECTORED_DATA_PATH': '/home/gwynu/geoips/outdirs/preprocessed/sectored', 'PREREAD_DATA_PATH': '/home/gwynu/geoips/outdirs/preprocessed/unsectored', 'PREREGISTERED_DATA_PATH': '/home/gwynu/geoips/outdirs/preprocessed/registered', 'PRECALCULATED_DATA_PATH': '/home/gwynu/geoips/outdirs/preprocessed/algorithms', 'CLEAN_IMAGERY_PATH': '/home/gwynu/geoips/outdirs/preprocessed/clean_imagery', 'ANNOTATED_IMAGERY_PATH': '/home/gwynu/geoips/outdirs/preprocessed/annotated_imagery', 'FINAL_DATA_PATH': '/home/gwynu/geoips/outdirs/preprocessed/final', 'PREGENERATED_GEOLOCATION_PATH': '/home/gwynu/geoips/outdirs/preprocessed/geolocation', 'SCRATCH': '/home/gwynu/geoips/outdirs/scratch', 'LOCALSCRATCH': '/home/gwynu/geoips/outdirs/scratch', 'SHAREDSCRATCH': '/home/gwynu/geoips/outdirs/scratch', 'LOGDIR': '/home/gwynu/geoips/outdirs/logs', 'GEOIPSDATA': '/home/gwynu/geoips/outdirs/geoipsdata', 'GEOIPS_ANCILDAT_AUTOGEN': '/home/gwynu/geoips/outdirs/ancildat_autogen', 'GEOIPS_ANCILDAT': '/home/gwynu/geoips/outdirs/ancildat', 'TCWWW': '/home/gwynu/geoips/outdirs/preprocessed/tcwww', 'PUBLICWWW': '/home/gwynu/geoips/outdirs/preprocessed/publicwww', 'PRIVATEWWW': '/home/gwynu/geoips/outdirs/preprocessed/privatewww', 'TC_DECKS_DB': '/home/gwynu/geoips/outdirs/longterm_files/tc/tc_decks.db', 'TC_DECKS_DIR': '/home/gwynu/geoips/outdirs/longterm_files/tc/decks', 'TC_TEMPLATE': '/local/home/gwynu/geoips/geoips/geoips/plugins/yaml/sectors/dynamic/tc_web_template.yaml', 'HOME': '/home/gwynu', 'TCWWW_URL': '/home/gwynu/geoips/outdirs/preprocessed/tcwww', 'PUBLICWWW_URL': '/home/gwynu/geoips/outdirs/preprocessed/publicwww', 'PRIVATEWWW_URL': '/home/gwynu/geoips/outdirs/preprocessed/privatewww'}
(geoips) [gwynu@winter filenames]$ git switch main
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
(geoips) [gwynu@winter filenames]$ python3 -c "import base_paths as b; print(b.PATHS)"
{'BASE_PATH': '/local/home/gwynu/geoips/geoips/geoips/filenames/..', 'GEOIPS_DOCS_URL': 'https://nrlmmd-geoips.github.io/geoips/', 'GEOIPS_OPERATIONAL_USER': False, 'GEOIPS_OUTDIRS': '/home/gwynu/geoips/outdirs', 'GEOIPS_PACKAGES_DIR': '/home/gwynu/geoips', 'GEOIPS_BASEDIR': '/home/gwynu/geoips/..', 'GEOIPS_TESTDATA_DIR': '/home/gwynu/geoips/test_data', 'GEOIPS_DEPENDENCIES_DIR': '/home/gwynu/geoips/../geoips_dependencies', 'PRESECTORED_DATA_PATH': '/home/gwynu/geoips/outdirs/preprocessed/sectored', 'PREREAD_DATA_PATH': '/home/gwynu/geoips/outdirs/preprocessed/unsectored', 'PREREGISTERED_DATA_PATH': '/home/gwynu/geoips/outdirs/preprocessed/registered', 'PRECALCULATED_DATA_PATH': '/home/gwynu/geoips/outdirs/preprocessed/algorithms', 'CLEAN_IMAGERY_PATH': '/home/gwynu/geoips/outdirs/preprocessed/clean_imagery', 'ANNOTATED_IMAGERY_PATH': '/home/gwynu/geoips/outdirs/preprocessed/annotated_imagery', 'FINAL_DATA_PATH': '/home/gwynu/geoips/outdirs/preprocessed/final', 'PREGENERATED_GEOLOCATION_PATH': '/home/gwynu/geoips/outdirs/preprocessed/geolocation', 'GEOIPS_COPYRIGHT': 'NRL-Monterey', 'GEOIPS_COPYRIGHT_ABBREVIATED': 'NRLMRY', 'GEOIPS_RCFILE': '', 'TC_TEMPLATE': '/local/home/gwynu/geoips/geoips/geoips/filenames/../plugins/yaml/sectors/dynamic/tc_web_template.yaml', 'DEFAULT_QUEUE': None, 'BOXNAME': 'winter.cira.colostate.edu', 'HOME': '/home/gwynu', 'SCRATCH': '/home/gwynu/geoips/outdirs/scratch', 'LOCALSCRATCH': '/home/gwynu/geoips/outdirs/scratch', 'SHAREDSCRATCH': '/home/gwynu/geoips/outdirs/scratch', 'GEOIPS_ANCILDAT': '/home/gwynu/geoips/outdirs/ancildat', 'GEOIPS_ANCILDAT_AUTOGEN': '/home/gwynu/geoips/outdirs/ancildat_autogen', 'LOGDIR': '/home/gwynu/geoips/outdirs/logs', 'GEOIPSDATA': '/home/gwynu/geoips/outdirs/geoipsdata', 'GEOIPS_VERS': '0.0.0', 'TCWWW': '/home/gwynu/geoips/outdirs/preprocessed/tcwww', 'TCWWW_URL': '/home/gwynu/geoips/outdirs/preprocessed/tcwww', 'PUBLICWWW': '/home/gwynu/geoips/outdirs/preprocessed/publicwww', 'PUBLICWWW_URL': '/home/gwynu/geoips/outdirs/preprocessed/publicwww', 'PRIVATEWWW': '/home/gwynu/geoips/outdirs/preprocessed/privatewww', 'PRIVATEWWW_URL': '/home/gwynu/geoips/outdirs/preprocessed/privatewww', 'TC_DECKS_DB': '/home/gwynu/geoips/outdirs/longterm_files/tc/tc_decks.db', 'TC_DECKS_DIR': '/home/gwynu/geoips/outdirs/longterm_files/tc/decks'}
(geoips) [gwynu@winter filenames]$

and both are certainly initialized. Nor is there another method in base_paths that was edited...

biosafetylvl5 commented 1 month ago

The failure in base_test comes from amsr2_ocean.tc.windspeed.imagery_clean.sh (for me: /home/gwynu/geoips/geoips/tests/scripts/amsr2_ocean.tc.windspeed.imagery_clean.sh)

biosafetylvl5 commented 1 month ago

On main, both the compare and the generated file have unexpanded env variables.

biosafetylvl5 commented 1 month ago

Replacing the base_paths.py file with the file from main resolves the issue.

biosafetylvl5 commented 3 weeks ago

Fixed in PR

NRLMMD-GEOIPS / geoips