multi-extension fits files

[havok2063]

Hi, this looks very promising, and I would love to implement this in a site I'm developing. I understand it's a scaled down version of DS9, but I have a few questions regarding some aspects of it.

• Is it possible to view/load other extensions of a FITS file in JS9? I cannot find any documentation on how to do this. It seems to only load the first extension by default.
• Can JS9 handle IFU data as DS9 can? When loading a spectral cube, DS9 gives me a pop-up slider to control the 3rd dimension (NAXIS3) from the given extension. Is this capability available in JS9? I tried it and it doesn't appear so at first glance.

[ericmandel]

@havok2063 I'm working on multi-extension support right now. I have the low level code completed (including the public access routine JS9.DisplayExtension()) and I am currently working on the high-level plugin through which you will be able to switch between extensions. I expect to upload it into GitHub in a few days.

Next comes data cubes, which actually is a bit harder but should follow the same basic pattern.

I'll let you know when the multi-extension stuff is done, so you can evaluate it, and then we can talk about cubes. Neither of these features is used much in X-ray astronomy, so I will appreciate the help in figuring out exactly what is wanted. (If you have a nice test data file, it might be helpful to add it to the suite of test files ... Send via email or put it in a place I can access.)

JS9 did start out as a scaled down version of DS9, but it's unclear where the limitations are going to lie (aside from memory and cpu limitations). For example, it now supports image blending, which DS9 does not (http://js9.si.edu/js9/js9blend.html). It's been very interesting, and surprising, to see how far we can push this.

[havok2063]

@ericmandel That's great. I'm happy to hear this is on the agenda. Yes please do let me know when the options for display of multiple extensions is available. I'd like to test it out. I'm happy to help in the design for cube viewing. I do have a test file I can provide. I'm mainly working with optical IFU cubes.

[ericmandel]

@havok2063 I've implemented support for multi-extension FITS files, consisting of several layers:

• File-> display FITS HDUs will simply display the HDU structure of a FITS file in a text window
• the View->Extensions plugin allows you to click on an HDU extension in order to display it
• the JS9.DisplayExtension() public access routine allows you to display extensions from Javascript

It's all uploaded into GitHub and installed in http://js9.si.edu.

I've tested it on the two relatively small MEF files I have in my possession, so more extensive testing in needed, and I will appreciate your input.

Without knowing what is wanted in practice, I can only say that it seems to do the right thing. But note, for example, the following: by default, the new extension is read into the same file, replacing the old data. This is more or less what DS9 does with cubes. But the JS9.DisplayExtension() routine also allows you to specify that the new extension should be displayed as a separate file (see documentation on the Public API), making two "files" instead of one. I did not add this to the Extensions plugin, because:

• it complicates the plugin interface and internal management, perhaps needlessly
• it could lead to browser memory problems if one loads a lot of large extensions all at once

We can, of course, add this feature to the plugin, but I think this is the sort of thing we need to decide once we get more experience.

So the questions are:

• does it work?
• does it do what you want?
• what else do we need?

[havok2063]

@ericmandel Excellent. I've had a quick look at this, and it looks very promising. I like it.

Does it work? - Yes, it seems to work for me. I've tried several FITS files with ~10 extensions, the largest file being ~110MB. There was a bit of delay when loading each extension for this large FITS after clicking, but it wasn't unbearable.

A few issues with the pop-up FITS extension window:

• Resizing the window does not auto resize the text inside. The initial window shows 4 extensions, but resizing keeps the others hidden rather than exposing them.
• The spacing for headers 1-9 isn't correct, when compared to headers 10+. E.g. hdu: 2name: FLUX type: image hdu:10 name: RIMG type: image
• When I scroll down to a new extension, and select it, the extension view window jumps back to the top. Is it possible to have it remain fixed around the extension I've just selected?

Does it do what I want? - As far as viewing extensions, yes it does what I want. All the functionality that existed for the default frame seems to exist for all the other frames.

What else do we need? - It would be nice to be able to blink frames or view frames side-by-side, but I can understand your reasons for not including them in the plugin. My main use for this in my site is as a quick explore tool for playing with data. I'm imagining that if people want to do more advanced things, they can download the files themselves and use their own tools, or the full DS9.

One compromise would be to allow to view only two extensions side-by-side for comparison but not more than that, due to memory issues as you say. I can see people wanting to look at the FLUX extension at the same time while viewing a MASK extension.

That being said, I did find a workaround. I loaded the main FITS in the main window, with the first extension. Then I clicked File -> new JS9 light window, and opened up a separate pane. I opened the file again, and now was able to open up a different extension in the same file, and view them side-by-side. Is this what JS9.DisplayExtension() does?

For my implementation, I think this works well for viewing extensions. I will most likely disable the File menu, because I can't have people having direct access to the filesystem where my site is run from. But I should still have access to the extensions through the View menu, or through the API.

With these extension window fixes, I think this could potentially be done. Right now, I can't think of any other use cases for me, for the extensions. The only other thing I would want is a slider for the naxis3 data.

[ericmandel]

@havok2063 That's a good start. I'll work harder on the plugin window: all of the things you mention can be fixed ... I didn't want to spend a lot of time on HTML work at the very outset. But if you can work with the current scheme for now, I'll spend a few days working on the GUI and on issues surrounding the separate display of extensions. The nice thing about allowing such separation is that you would be able to do things like blink them or blend them as a matter of course. (And also lurking in the background is the issue of moving an image from one JS9 display to another, to make side-by-side comparisons easy.)

Would it be possible to get access to one of your 10-extension FITS files to use for testing? That would help me better understand the real-world performance limits.

[havok2063]

@ericmandel Yeah sure. I have one about ~112 MB in size. What would be the best way to deliver it?

[ericmandel]

Can you put it on a web site for 1/2 hour? ftp site? We may have an ftp site which allows anon. put, but I'd have to ask ...

[havok2063]

@ericmandel I will need to contact my collaborators first before I can share one of our IFU cubes. Once they approve I can share it with you via the Dropbox link you sent me.

[ericmandel]

If the data file is proprietary, don't bother. I'll ask around here instead. Obviously, Bill Joye has data which he used to develop the DS9 cube capabilities ...

[havok2063]

It is SDSS data that will be made public soon. I don't think they'll mind, but I just need to get confirmation.

[ericmandel]

OK, but don't use any silver bullet on this. In the meantime, I'm fixing the html problems you mentioned above (except for the header spacing issue, for which I need a file with 10+ extensions).

[ericmandel]

@havok2063 I fixed extension plugin resize, scroll, and (Chrome) spacing problems mentioned above. I'll give some thought to allowing a separate display of extensions before going on to cube processing ... but let me know if you run into problems in the meantime.

[havok2063]

@ericmandel That's great. Have they been pushed to the website for testing? I got the OK about the file and just uploaded it to your dropbox link.

[ericmandel]

@havok2063 Yes, in GitHub and installed on the js9 web site for testing. I'm working on the ability to move an image from one display to another (which turns out to be a bit tricky) as a precursor to allowing you to display an extension to a separate "file". Together, these two features will steam-line the comparison process you outlined above, which is a great idea, so thanks ... and thanks for the test file!

[havok2063]

@ericmandel Great. I've played around again with the tool, and all three fixes look to be in place. I've only tested it in Chrome so far. It looks good. I thought of another feature that would be good. I've always wanted the ability to do this in DS9 and was disappointed it was never there. Image extension arithmetic. I'd like the ability to do mathematical operations (add, subtract, etc) on different extensions, and make a new image. Or take the mean of some part of one extension and add it to another. Stuff like that. Do you think something like that is possible?

[ericmandel]

Yes, some important underlying support for image arithmetic is in place, but there are some tricky complications. Have a look at the public API routine called JS9.RawDataLayer(), which allows you to manipulate the raw data for an image. This is how, for example, the GaussianBlur function works. So in principle, if you can access an image's extensions at the same time, you can combine them arithmetically into a new raw data layer.

If you like, cut and paste your comment above into a new issue, and we can work through the details in the next few weeks.

[ericmandel]

@havok2063 I have implemented two new JS9 features, which together should open up some new analysis possibilities:

1. The Extension plugin now has a checkbox at the bottom to _Display each extension as a separate image_ When checked, newly displayed extensions act like separate images and can be blinked, blended, etc.
2. The _File->move image to_ menu option allows you to move an image to another display (including the ability to create a new lightweight window and move to it in one go). So, for example, after loading some extensions as separate images, you can display them side by side, if necessary.

The code to move the images was tricky because there are no many HTML canvases in play (one for each graphical layer). It looks like it works pretty well but ... it was tricky so ... let me know what you find.

It's been uploaded into GitHub and also installed in http://js9.si.edu.

[havok2063]

@ericmandel Great. I will have a look today, play around with it and reply with some comments. Thanks.

[ericmandel]

@havok2063 I've pushed initial 3D data cube support into GitHub, including a JS9.DisplaySlice() public routine and a DataCube plugin. It works on the 3D cube you gave me, although extracting a slice is slower than I expected. I don't see how to improve the performance, it may just be where the Javascript technology is at right now . I'm going to close this issue but feel free to start a new one.

[havok2063]

@ericmandel Sorry about not responding earlier to this. I really like what you have done. These are both really useful features. I do have some feedback about the last changes made.

1. I was wondering if we could move the 'Display each extension' checkbox to the top of the window. With it at the bottom, it's a bit hidden away when there are lots of extensions, so it's not clear to people that it exists.
2. When we move an image to a new window, via File -> move image, and then View Extensions , it can be a bit confusing if we already have the View Extensions window open for the original window. Maybe adding a Title to the new window, and any new popup windows associated with it, will help the user distinguish which pop up windows go with which image windows.
3. With multiple View Extension windows open, there is a bug where the second window will open extensions as a new image, even when the option is unchecked. I think this happens when the option is checked in the first View Extension window.

4 . This is a minor one. By default, when opening a FITS file, which extension does it open up? The first one? In my case, I noticed by default it opened the first one, the FLUX extension. When I clicked on Flux again in View Windows, it opened it up a second time with the extension index [2]. I think this is normal behavior we want, but in the case of the first extension probably always it up a second time.

[havok2063]

@ericmandel How do I use the DataCube plugin? Is this something that is usable in the browser version currently up on the web?

Edit: Nevermind, I found it. :)

[ericmandel]

@havok2063 Thanks for this good feedback. I'll make mods in the next few days ...

[havok2063]

@ericmandel I like the new DataCube feature. This works well. The speed isn't too too bad. How are you extracting the slice? Is this a server-side operation? Are you reading the entire thing into memory each time to extract a slice?

Some feedback:

A minor tweak. Could we add a label between the slice index and the slice select dropdown, that indicates the name of the extension we have open? Right now you need both this window and the View Extensions window open to really know what you're looking.

[ericmandel]

@havok2063 I'll add the extension name along with the other mods. All of these are good ideas that should be relatively easy. And you definitely found a bug, looks like I tied some mef variables to the plugin object instead of the individual plugin instance, so it got picked up globally. I'll fix all of that.

The sluggish slice access just uses standard cfitsio routines (well, cfitsio compiled to JavaScript). The data file is in memory, but 1) JavaScript cfitsio runs a bit slower than native, and 2) it does have to ungzip the data each time. Per tab browser memory is limited, so I can't simply ungzip the file once (as I believe is done by DS9).

In general, not having access to virtual memory is the major problem we face trying to do real analysis on the web. Maybe it will be the show-stopper for in-browser processing ... we'll see.

[ericmandel]

@havok2063 I made the requested mods, updated into GitHib and installed on the Web site for testing. Let me know what you find ...

Move the 'Display each extension' checkbox to the top of the window.

Done

Maybe adding a Title to the new window, and any new popup windows

Done

With multiple View Extension windows open, there is a bug where the second window will open extensions as a new image, even when the option is unchecked.

This was hard to reproduce, but I fixed a bug which resulted in HTML Extension element ids not being unique when multiple extensions were displayed as separate files. If this problem persists, I'll need more explicit instruction on how to reproduce it.

By default, when opening a FITS file, which extension does it open up? The first one? ...When I clicked on Flux again in View Windows, it opened it up a second time with the extension index [2].

When you load an image, it goes to the first interesting data it finds: primary header, followed by a search for the primary X-ray event table, followed by the first valid image. So in your case you got extension 2.

And when you click on extension 2 in separate mode, it will make a separate file. I think this is how it must be, because you cannot guarantee that original image will still have extension 2 displayed, i.e., if you first displayed another extension while not in separate mode, and the changed to seaprate mode, etc. But I agree its confusing and I am open to suggestions.

A minor tweak. Could we add a label between the slice index and the slice select dropdown, that indicates the name of the extension we have open?

Done

[havok2063]

@ericmandel This looks great thanks. All the tweaks look perfect. The bug that occurs when the multiple View Extension windows are open looks resolved. I found one additional tweak regarding this, which may be related to other issue with which loaded extension appears displayed under the File menu

• The extension number is not being updated when I click a new extension. The checkbox to display as a new image is turned off.

Here are my steps to produce this behavior.

1. Open up my cube. Open View Extensions. Check the box to open extensions as new image.
2. Click on Flux to open as new image filename[2].
3. Click File -> Move to -> new window.
4. From new window, open up View Extensions, check the box to open as a new image. Open any other extension.
5. Uncheck the box and click on Flux again. It now opens up in the existing window, replacing the other extension, but the name under File is not updated. It still displays the extension id of the old one. Plus there is already a Flux extension open.

It sounds like this is normal behavior but it's a bit confusing since 1.) that extension is already open and 2.) now the flux extension is loaded onto an extension that is labeled as something else. Could we change the display name in File from filename[extno] to filename[extname]? That might be a bit more clear. Is it possible that when opening an extension, we check if it is already opened as another image in the same window? If it is, then it just switches to that image?

• Also found a new bug now when using the View -> Datacube option. Dragging the slice slider now produces an error popup that says "ERROR from cfitsio.js: error seeking file position"

[ericmandel]

@havok2063 Thanks! I think it would be a really good idea to have the EXTNAME in the File menu display name. I have to look into the ramifications, which I'll do (while fixing bugs) later in the week.

[ericmandel]

@havok2063 I added the extname (or extnum) to the image id. This required a number of changes to various parts of JS9 to make it all consistent (including identifying and working around a mismatch between the cfitio extended filename extension number and the cfitsio library routine to get the extension number.) But in the end it makes the multi-extension processing much cleaner, and also solves a long-standing potential problem with back-end analysis. The mef plugin bug should be fixed as well.

I couldn't reproduce the data cube bug, which could have been caused by a mismatch between js9support.js and js9.js ... sometimes you just have to shift-reload the JS9 main web page. Not sure.

Lots of changes in the code, so please let me know if you find problems. (Also added histogram equalization to scaling in the interim.)

[ericmandel]

@havok2063 it turns out that the delay in accessing a new slice from your 100Mb test file is entirely due to the fact that it's in gzip'ed form. If you load the ungzip'ed file, the speed-up is striking indeed, both during the initial load and subsequent changes of slice. I'll look into this, but I suspect it's going to be one of the rough edges of Web-based analysis.

I'm close to closing this issue, unless you have more comments ...