Consider suppressing astropy's warning about FITS header card length

[profxj]

I get 100's of these when trying to view a simple VLT image:

WARNING: VerifyWarning: Keyword name 'FRAME9 _ATE' is greater than 8 characters or contains characters not allowed by the FITS standard; a HIERARCH card will be created. [astropy.io.fits.card]

See https://github.com/pypeit/PypeIt/pull/1365 for a way to fix this..

[ejeschke]

This seems reasonable to me, provided that we make it controlled by a setting. I think I would be ok with an "opt-out" strategy; i.e. the warnings would be suppressed by default, but could be re-enabled by changing a setting in the general.cfg.

What do you think, @pllim?

[ejeschke]

Also, @pllim, is there a better way to turn them off directly in astropy.fits?

[pllim]

What is the context for this request? Is this in the context of people running ginga problematic_file.fits from the command line? Or is this someone calling ginga code from their own code? Depending on the context, my recommended solution would be different.

My general philosophy is: The warning is where for a reason. The best way to get rid of it is to fix your data. But if that is not possible, depends on the context of usage.

[pllim]

p.s. 'FRAME9 _ATE' is greater than 8 char and violates the FITS standards...

[pllim]

turn them off directly in astropy.fits

No, I don't think so.

[ejeschke]

Unfortunately, there are many instruments out there that don't necessarily conform properly to the standard (especially some older ones). It might be a pain to work with those files over and over without suppressing the warnings...

[ejeschke]

How about an opt-in strategy then, @pllim?

[pllim]

Opt-in is my preference, if that is the case.

If the warnings are already captured by ginga logger, then it is simply setting the logger level. But if that is outside the logger, then I am afraid a blanket ignore by category/message, as suggested above, is the way to go (and it affects the entire session).

[ejeschke]

I do agree that seeing the errors is very useful. Would be nice if there was a way to turn them off one by one, or maybe by some signature of the FITS file. For example, this set of keywords indicates it is a VLT image--if so, don't warn me about the 'FRAME9 _ATE' keyword.

[profxj]

Yes, I love the -- ignore this if it is a VLT image -- idea.

I have never edited the general.cfg file for ginga and should prefer not to figure out how to do so.

[profxj]

I'll add this is the way VLT always writes their images, i.e. they always have an "error" filled header.

[profxj]

And I should have added that it currently takes ~30s to open+view a simple VLT image because of these warnings (should take ~2s)

[ejeschke]

@profxj, that ~30s doesn't sound right even with the warnings. Could you make a sample of this file available for testing?

[profxj]

ok, I exaggerated (but just a little):

Tue 01 Mar 2022 03:44:44 PM PST ginga FRB20211212_VLT-FORS2_g-HIGH_2022-01-28.fits Tue 01 Mar 2022 03:45:08 PM PST

said FITS file is attached. Scratch that, it won't let me upload a FITS. Grab it here:

https://drive.google.com/file/d/1l6NKonFanpZg329T-EPaIrHykrP_Njw8/view?usp=sharing

[ejeschke]

@profxj, thanks for sending that. Running on my 5 year old desktop box, I got the following result:

2022-03-01 13:52:00,627 | W | warnings.py:109 (_showwarnmsg) ... <= first warning message ... 2022-03-01 13:52:02,170 | W | warnings.py:109 (_showwarnmsg) ... <= last warning message 2022-03-01 13:52:02,695 | I | Control.py:1277 (channel_image_updated) | Channel image update: 0.0004 sec

So opening the file does take approx 2s just to read and process all the warnings. Then about 0.5s to display the image. This is stock ginga with --loglevel=20

[ejeschke]

But I see your point about the warnings--it completely swamps the log.

[profxj]

Odd.

My computer is much newer.

Let me try that log command.

On Tue, Mar 1, 2022, 15:59 ejeschke @.***> wrote:

But I see your point about the warnings--it completely swamps the log.

— Reply to this email directly, view it on GitHub https://github.com/ejeschke/ginga/issues/997#issuecomment-1055988938, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA6QDDMV3UUW4UFQOIOZR23U52VPZANCNFSM5PVHHAGA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

[ejeschke]

I'm just dragging the file to the viewer window. I usually start ginga like this:

ginga --loglevel=20 --stderr

I add --log=/path/to/some.log if I also want to record the log in a file.

[profxj]

Just executed this on my laptop, which is newer than my workstation:

ginga --loglevel=20 --stderr FRB20211127_VLT-FORS2_g-HIGH_2022-01-29.fits

Took ~12s

[ejeschke]

Just tried that, little bit less than 5s including startup time where it loads all the plugins.

[ejeschke]

I suppose it might depend on what kind of storage you have. My workstation has SSD.

[profxj]

Same (for both).

I dare say the Mac is faster! ;)

[ejeschke]

Try routing the output to a log with --log=ginga.log and then afterward look at the timestamps in the log. You should be able to see the timestamp for the first warning and then towards the end the timestamp for the last warning and then shortly after a message like Control.py:1277 (channel_image_updated), which is printed right after the image is loaded in the channel and the display updated. Should give you a good idea of where the time is going.

[ejeschke]

My box is Linux. :smile_cat:

[ejeschke]

But back to your point, it looks like it is taking some excess time to load the file due to all the warnings. I stuck in the line

warnings.simplefilter('ignore', VerifyWarning)

and the time to load the file (dragging to window, after ginga already started) went from 2.5 sec to about 1 sec.