xRITDecompress Environment Variable Problem running SatPy Script from a crontab

[TheMetMan]

Dear SatPy Team This is not strictly an issue with the code, but it might be.

I have been using SatPy for a while now, moved over from PyTroll for Python 3

I have had it running on a Gentoo Server as Xen Guest, no problem For various reasons I am moving it onto an Ubuntu Server as Xen Guest.

I am using Conda for the Python Environment.

It runs absolutely fine when run from the command line, but it will NOT run from a crontab.

I get lots of this errors when it should be decompressing:

sh: 1: xRITDecompress: not found sh: 1: xRITDecompress: not found ......

I have even set the environment up inside the python script.

Here is the start of the output of print(os.environ) from the Python Script

environ({'SHELL': '/bin/bash', 'SCIPY_PIL_IMAGE_VIEWER': 'display', 'CONDA_EXE': '/home/francis/bin/miniconda3/bin/conda', '_CE_M': '', 'XRIT_DECOMPRESS_PATH': '/usr/local/bin/xRITDecompress', 'PWD': '/home/francis/bin/Py_MeteosatImageCreator_2/src', 'GSETTINGS_SCHEMA_DIR': '/home/francis/bin/miniconda3/envs/SatPy_Generic/share/glib-2.0/schemas', 'LOGNAME': 'francis', 'CONDA_PREFIX': '/home/francis/bin/miniconda3/envs/SatPy_Generic',

Which shows that XRIT_DECOMPRESS_PATH is set correctly as confirmed with this command:

ls /usr/local/bin/xRITDecompress /usr/local/bin/xRITDecompress

I have looked around on the web, clearly Ubuntu use vixie-cron, but my Gentoo Box uses cronie

Do you have any ideas how to overcome this?

Regards

[djhoese]

Do you get any other error messages or log messages? I suppose the "not found" is coming from the internal Popen in satpy:

https://github.com/pytroll/satpy/blob/f59da8b89f07fbaee128fc38edce8c493eebb38d/satpy/readers/hrit_base.py#L132-L137

Otherwise, I would have expected one of these error messages to show up:

https://github.com/pytroll/satpy/blob/f59da8b89f07fbaee128fc38edce8c493eebb38d/satpy/readers/hrit_base.py#L96-L108

Oh...do you have execute permissions for everyone on the xRITDecompress binary (the x permission for owner, group, and other)?

Edit: ls -l /usr/local/bin/xRITDecompress would give us something, but I could also see permissions issues with the directories above that being an issue...unlikely with /usr/local/bin though.

[TheMetMan]

@djhoese thanks for the swift reply! I think I have the answer. Overnight I thought that it must be something to do with the Conda Environment, and it turns out you can set the environment for a Conda Environment by adding a bash script into a file in the $CONDA_PREFIX/envs/YOURE_ENVIRONMENT/etc/conda/activate.d/env_vars.sh for example with the Environment Variables declared there, and they will appear when the environment is activated. This does work, but I still have the same problem. I think I am going to have to switch to the standard Python Virtual Environment to overcome this. Anyway I do NOT think this is an Issue for SatPy, but I am grateful for your input. By the way the output of ls -l /usr/local/bin/xRITDecompress is -rwxr-xr-x 1 francis francis 548864 Oct 13 10:16 /usr/local/bin/xRITDecompress Here is a link explaining the concept And another excellent one explaining my problem and how to fix it about half way down the page here. I will test it out this morning and report back Regards

[TheMetMan]

Well, @djhoese I did try! But failed miserably to get the Conda Environment to work, so I will ditch Conda and use Python venv instead, at least that works! I think you can close this now. Regards

[djhoese]

That's too bad that you had to switch. Conda does require some special care when activating things in scripts. I didn't suggest anything about that because you were setting the environment variable in your python script. That should have worked if I understood you correctly.