customizing color map?

[jhennawi]

Hi All,

I have images which have a fraction of pixels which are masked. I'm trying to indicate these pixels via color-code, with each mask bit having a different color.

I was trying to do this by over plotting points using 'Points' and the canvas.add('constructedcanvas', canvas_list) functionality, kindly pointed out to me by @ejeschke.

However, for large images with significant numbers of masked points (~ 1000), this becomes so slow as to be no longer practical (ginga grinds to a halt).

The more intelligent way to achieve this goal would be modifying the color map for specific pixels. This could be done for instance by inserting these values into the image at the desired location and customizing the color map. It could also maybe be done some other way.

Do you have advice on how either a) overplay points in a way that won't be super slow, b) modify the color map or specifically change the color of certain pixels.

Many Thanks! Joe Hennawi

[pllim]

Hi, Joe. You mean like this? https://stginga.readthedocs.io/en/stable/stginga/plugins_manual/dqinspect.html

If so, it is already implemented but over at stginga. I could possibly move it to this repo if it is useful to others. The DQ flags I use are pretty HST and JWST specific but they can be customized.

[pllim]

Or if you don't want it in the plugin form but want to look at the code, it is at https://github.com/spacetelescope/stginga/blob/master/stginga/plugins/DQInspect.py . Its license is BSD. And it is no longer PY2 compatible (PY3 only). FYI.

[jhennawi]

Hi @pllim

That is beautiful and exactly what I need! I similarly have a bit mask that I would like to color code and display.

Joe

[pllim]

Great! It could use some improvement. So, I would welcome PRs if you find bugs etc. 😄 Thanks.

[jhennawi]

Happy to take it for a spin!

[jhennawi]

Hi @pllim,

I should also add that given the way I'm using ginga, I'd ideally be able to control this from the RC interface? Not sure if that is possible, but it seems that this is do-able with any plug in, i.e see my question about another plugin:

>https://github.com/ejeschke/ginga/issues/693

[pllim]

@jhennawi , unfortunately the RC interface is beyond my expertise. Hopefully @ejeschke can answer this one. In the worst case scenario where you need to re-implement it for RC, hopefully you could still re-use some code from the plugin instead of re-inventing the wheel. 🤞

[jhennawi]

Okay great.

[ejeschke]

@jhennawi, you might also have a look at the Overlays plugin as well, although I think @pllim's DQInspect is closer to what your end purpose is, it seems. (Side note @pllim, this is the second user that has asked about something like this to me, might be worth considering putting a customizable version in stock reference viewer).

Also, FWIW you could create a custom color mapper, but I think although that would be the most efficient it would also be more work.

[pllim]

Okay, I'll see what I can do... 😄

[ejeschke]

Re: invoking from RC, do you mean to just start the plugin, or to call methods on it? The former is doable using the current interface, the latter will require some small tweaks to the code, but sounds like a useful feature.

[ejeschke]

the latter will require some small tweaks to the code, but sounds like a useful feature

See #694 for an implementation of being able to call methods in a plugin from the RC interface.

[jhennawi]

This sounds great, but I'm a bit lost as to how to proceed. If I wanted to for example add @pllim's DQInspect plugin, where would I put what code? Then I presume I would start ginga with ginga ---modules=RC, MyGlobalPlugin

Or just

ginga ---modules=RC

and then

start_global_plugin()

Following what you explained in issue #693

[pllim]

DQInspect is part of stginga package (pip install stginga). If you choose to install it, then you can use --plugins=stginga.plugins.DQInspect to load it. It will have to be this way until I have a chance to move it over to Ginga.

[ejeschke]

@jhennawi, can you use the DQInspect plugin as is (assuming you could load it), or do you need to modify it to use your own flags scheme?

[jhennawi]

I'd like to modify it using my own flag names. If you move this to ginga, how do you plan to allow the user to choose their own flag names?

[pllim]

Yes, it already does allow user to customize DQ flags. As documented in https://stginga.readthedocs.io/en/stable/stginga/plugins_manual/dqinspect.html , you can set dqdict in its configuration file. It maps instrument to a particular definition file. At STScI, instrument is defined by INSTRUME keyword in FITS primary header. If you use a different keyword, you can customize your general.cfg:

https://github.com/spacetelescope/stginga/blob/196ba545b1d97166cf0a52fbe9e616d294917ab9/stginga/examples/configs/general.cfg#L73

Here is a snippet of definition file we use for ACS:

https://github.com/spacetelescope/stginga/blob/196ba545b1d97166cf0a52fbe9e616d294917ab9/stginga/data/dqflags_acs.txt#L1-L5

Here is the actual code to parse the file you give it (since yours is not package data):

https://github.com/spacetelescope/stginga/blob/master/stginga/plugins/DQInspect.py#L303-L306

(Huh, the GitHub code preview thing is not working...)

[jhennawi]

Hi @pllim,

I managed to install stginga and DQInspect, and loaded it into ginga. However, I'm not following how to give it the image which it will use as the bit mask.

To be clear, my goals are:

View an image with ginga, let's call this science.

Then highlight the pixels which have a given bit set, using another image called bitmask.

I follow your instructions for changing the bit labels, but I don't follow how to tell DQInspect to use the bitmask image.

Thanks! Joe

[pllim]

I'm not following how to give it the image which it will use as the bit mask

Oh, I think this is where it gets STScI specific. It is looking for a DQ extension in that same image with science.

https://github.com/spacetelescope/stginga/blob/196ba545b1d97166cf0a52fbe9e616d294917ab9/stginga/examples/configs/general.cfg#L71

Here is an example layout of a HST image:

>>> from astropy.io import fits
>>> with fits.open('jb5g05ubq_flt.fits') as pf:
...     pf.info()
Filename: jb5g05ubq_flt.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU     239   ()      
  1  SCI           1 ImageHDU        93   (4096, 2048)   float32   
  2  ERR           1 ImageHDU        46   (4096, 2048)   float32   
  3  DQ            1 ImageHDU        38   (4096, 2048)   int16   
  4  SCI           2 ImageHDU        93   (4096, 2048)   float32   
  5  ERR           2 ImageHDU        46   (4096, 2048)   float32   
  6  DQ            2 ImageHDU        38   (4096, 2048)   int16 

So, (SCI, 1) would have its DQ bits in (DQ, 1), (SCI, 2) in (DQ, 2), and so on...

Now that you brought this up, it might be tricky to generalize the plugin to read in DQ from some arbitrary buffer. 🤔

[jhennawi]

Hi @ejeschke,

I tried using the TVMask plugin for this, since this allows me to directly load in a mask by hand and assign it a color. However, for some reason the colors are not displaying. I loaded the image in just fine from the viewer, but not sure why nothing is happening.

[pllim]

TVMask converts the mask to boolean. If that is not a blocker for you, make sure alpha is not set to 0. In addition, if the mask only affects a handful of pixels, try zoom in to them.

https://github.com/ejeschke/ginga/blob/fa239ab8d43a9e5070663ed2b5f535e676f3c5c4/ginga/rv/plugins/TVMask.py#L271-L273

Here is what masks are supposed to look like in action:

https://ginga.readthedocs.io/en/stable/manual/plugins_local/tvmask.html

[jhennawi]

Hi @pllim,

Yeah my mask image was 1s and 0s, so I'm not sure why it is not working.

Joe

[pllim]

@jhennawi , at this point, I cannot guess further without the data. Are you able to share this one image and its mask that is not working for you?

[jhennawi]

Yeah let me play with some canned examples.

On Wed, Aug 29, 2018 at 11:27 PM P. L. Lim notifications@github.com wrote:

@jhennawi https://github.com/jhennawi , at this point, I cannot guess further without the data. Are you able to share this one image and its mask that is not working for you?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ejeschke/ginga/issues/691#issuecomment-417111730, or mute the thread https://github.com/notifications/unsubscribe-auth/AIXjoyCxF8s6x-Ngc8B-9PHDAO-dbEhWks5uVwc3gaJpZM4WPR0O .

--

Joseph F. Hennawi Associate Professor Department of Physics Broida Hall, UC Santa Barbara Santa Barbara, CA 93106-9530 Phone: 805-893-3503 Mobile: 805-450-8697 E-mail: joe@ joe@physics.ucsb.eduphysics.ucsb.edu http://web.physics.ucsb.edu/~joe/ enigma.physics.ucsb.edu

[pllim]

@jhennawi , the plugin works via traditional GUI but I have problem using the RC interface, which was what you were probably using. Nothing happened because it threw errors. See https://github.com/ejeschke/ginga/pull/694#issuecomment-417384297

[ejeschke]

See the comment I added following yours.