ctxcal not working on ISIS3

[cneish]

ISIS version(s) affected: 3.5.2

I installed this version: Example for Ubuntu 14.04 Linux x86 64-bit Intel compatible systems: rsync -azv --delete --partial isisdist.astrogeology.usgs.gov::x86-64_linux_UBUNTU/isis .

Description

I am working to calibrate a CTX image on ISIS3. I use the commands:

mroctx2isis spiceinit ctxcal

The image looks fine after the spiceinit step, but once I run ctxcal, it turns "white" (every pixel has a value of zero). I receive no errors after running ctxcal. I have never encountered this issue before using this version of ISIS! I have attached a screenshot showing an example file.

This has happened on several CTX images this morning, but one in particular is B20_017450_1542_XN_25S047W.IMG.

I would appreciate your insight on this issue.

Thank you, Catherine

[astrostu]

Sanity checks: (a) Have you successfully processed this image before? (b) Is the PDS flag set to ERROR for this image? (c) Is the brightness being output actually all HRS, or is the auto-scaling in QVIEW just wrong (I've had that happen a few times).?

[cneish]

(a) I have not processed this image before, but I have successfully processed other CTX images before. (b) I'm not sure what you mean by the PDS flag, but in the header it says "PDS_VERSION_ID = PDS3". (c) When I look at the pixel value in qview (bottom right), all pixels give me a value of zero.

[cneish]

I will also note that we processed three CTX images today, and they all had the same issue.

[astrostu]

The PDS flag can be found in the index or cumindex files on PDS. For this particular image: https://pdsimage2.wr.usgs.gov/archive/mro-m-ctx-2-edr-l0-v1.0/mrox_1102/index/index.tab . And it looks like it's set to OK, so that's not the issue.

I'm on travel but have ISIS 4.4.0 on my laptop. I can tell you that I just ran it and the image processes fine in that version of ISIS.

If you processed three images today and they all have the same issue, then maybe something changed with your computer setup in the recent past that's messing things up? If you just process the image through mroctx2isis and look at it in QVIEW, is it blank? If so, then perhaps the issue is with the download? Though I'm not sure why mroctx2isis wouldn't given an error if that were the case. If it shows data after mroctx2isis, then run spiceinit and see if it's blank.

[cneish]

It looks fine after the mroctx2isis and spiceinit steps. (See image on the right in the screenshot.) Something goes wrong during the ctxcal step.

I'm not sure what would have changed in our set-up (despite the fact it's very old!). I'm trying to update all the MRO data files right now to see if that helps.

[jessemapel]

Do you have the SPICE data local? Before 5.0.0, all of the calibration applications require local SPICE data in order to compute the solar distance. If the SPICE data is not available it instead returns a massive distance that results in all of the data being "corrupted".

[cneish]

Do you mean the kernels in the /data/mro/ directory? I'm updating that right now. But it's strange that it worked for other images in the past. Does it depend perhaps on the date of acquisition for the image?

[jessemapel]

Yes, you'll need the kernels in /data/mro/kernels. I'm surprised it's worked in the past. It could be that you have some of the kernels for older images, but not kernels for the newer images.

[cneish]

That's what I thought, but some of the images we tried were from 2010. I'm sure we had the kernels from 2010. I will try again when all the new kernels have downloaded.

Is this something you can check when you're back in the office with an older version of ISIS? I fear I am doomed to have to do an reinstall. :)

[jessemapel]

If updating your data area does not fix this, I can try and reproduce it with our install of 3.5.2 I'll also see if it's still a problem in the latest release.

[cneish]

Updating my data area did not fix the problem. I'd be curious to hear what happens on your end when you use 3.5.2. Thanks!

[astrostu]

I'll also add that B20 images are from April 2010, so unless the old kernels were replaced which the MRO team does very rarely if at all, the SPICE data wouldn't be the issue for a B20 image ... I wouldn't think, anyway.

[lwellerastro]

Hi - I just wanted to confirm where I did and did not run into problems calibrating B20_017450_1542_XN_25S047W.IMG.

The image calibrates fine when I process under isis5.0.2 or the latest version isis7.0.0RC2. The oldest version of isis I could get to is isis3.7.0 and under that version all pixels in the calibrated image have dn=0.

The isis labels have changed a tad across these versions, but one difference that stood out has to do with the Radiometry group for the images.

isis3.7.0
  Group = Radiometry
    FlatFile = $ISIS3DATA/mro/calibration/ctxFlat_0002.cub
    iof      = 0.0
  End_Group

isis5.0.2 and isis7.0.0RC2
Group = Radiometry
    FlatFile = $ISISDATA/mro/calibration/ctxFlat_0002.cub
    iof      = 2.10597029660453e-04
  End_Group

[astrostu]

@lwellerastro -- Interesting ... not sure then why @cneish would've been able to process the data before?

@cneish -- did you maybe update your SPICE data directories a few days ago (BEFORE the DN=0 issue) but hadn't in a long time before that?

[jlaura]

@lwellerastro I edited your comment to remove the full file paths and instead to use the environment variables that ISIS is expecting. That should be portable across systems and also does not expose the internal directory structures to the wider internet.

[lwellerastro]

Thanks @jlaura. Just be aware that what I shared is how it is being stored in the isis label. Maybe that is a mistake? Everything in the kernels group uses the environment variables.

[cneish]

@astrostu That's entirely possible.

Is there any way to fix this problem using ISIS3? Or will I have to upgrade to a newer version?

[jlaura]

@lwellerastro I figure it is better for our security (though not a huge concern) and, maybe more importantly, it is more portable for our users to use the env variable - even if that means an edit of what is actually in the label.

I am not sure if they label should be using the env variable. Seems it would be more portable but I defer to someone else on the team better versed in how the label are written.

@cneish I would strongly advocate updating. ISIS3.5.2 is quite out of date.

[cneish]

Is it possible to upgrade ISIS on Ubuntu 14? I originally installed that (very old) version of Ubuntu because it was the only one that was supported by ISIS. (Back in the days before Anaconda.) I would prefer not to have to update the operating system and ISIS! :) Thanks.

[jlaura]

@cneish Assuming miniconda is supported, it should work. Full disclosure - I have never tested conda on Ubuntu 14.

The conda environment abstracts away everything except for low level libraries (like clib and some graphics drivers).

[cneish]

Hi again,

I am now attempting to install ISIS 6.0 on Ubuntu 14.04, and have run into some problems.

When I first tried to open qview, I received the following error:

(isis) root@neishlab-ubuntu:~# qview: error while loading shared libraries: libavresample.so.3: cannot open shared object file: No such file or directory

I had seen a similar error when I installed ISIS 6.0 on my Mac, so I ran this command suggested by @jessemapel:

conda install -c conda-forge ffmpeg=3.4.1

Now I am encountering a new error:

(isis) root@neishlab-ubuntu:~# This application failed to start because it could not find or load the Qt platform plugin "xcb" in "".

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, xcb, vnc.

Reinstalling the application may fix this problem.

I think this is related to the program that opens up the GUI in Ubuntu, as I can still use ISIS at the command line.

However, even at the command line I am running into problems:

mroctx2isis from=D17_033814_1526_XI_27S046W.IMG to=D17_033814_1526_XI27S046W.IMG.cub ERROR No existing files found with a numerial version matching [ctxsqroot???.lut] in [$ISISDATA/mro/calibration].

I thought I had defined $ISISROOT correctly, but apparently not?

Any help you can provide would be appreciated.

Thanks, Catherine

[cneish]

Addendum: I have fixed the second problem by running this command again:

conda env config vars set ISISROOT=$CONDA_PREFIX ISISDATA=[path to data directory]

I must have made an error the first time. My apologies.

[jessemapel]

Your QT Plugin issue is most likely caused by your PATH or LD_LIBRARY_PATH having extra stuff in them. Check that you don't have extra QT installs in any of the locations pointed to by those environment variables.

[cneish]

My LD_LIBRARY_PATH is empty, but my PATH called the old version of ISIS. I deleted that, and was able to open qview. And now the CTX image looks correct! So we can officially close this thread. Thanks so much for your help.