rfrenchseti
commented
6 years ago Summary email:
OK. I'm going to defer PRODUCT_CREATION_TIME a bit though. OPUS doesn't currently have the ability to search on this kind of date field.
From: Mark Showalter Sent: Wednesday, April 18, 2018 4:19:00 PM To: Rob French Subject: Re: Audit of COISS fields in OPUS
PRODUCT_CREATION_TIME needs to be added to the General tab. The reason is that some data sets (like CIRS and NH LORRI and all HST) are distributed iteratively and revised periodically. People might want to query on the date of the latest revision. I suspect it will appear as something like "Latest revision date". For COISS, we will probably use it for the date of the latest image re-calibration. However, the PRODUCT_CREATION_TIME for the raw file is a good starting point.
If you think MISSING_LINES is useful for search, add it.
On Apr 18, 2018, at 2:44 PM, Rob French rfrench@seti.org wrote:
Do you really want PRODUCT_CREATION_TIME to be searchable? That doesn't seem like a very useful field to me.
Regarding MISSING_LINES, it probably doesn't have a non-zero value for a lot of images you care about, but it's a big deal for close-up moon images. For example, basically all of the close-up images of ENCELADUS have a non-zero value. For anyone trying to make a pretty fly-by movie or something, this is a good way to filter out bad images.
From: Mark Showalter Sent: Wednesday, April 18, 2018 2:31:29 PM To: Rob French Cc: Matthew Tiscareno; Mitchell Gordon; Michael Aye; Matthew Hedman; rchancia@uidaho.edu Subject: Re: Audit of COISS fields in OPUS
Two quick follow-ups.
In my experience, the number of MISSING_LINES is zero the vast majority of the time. That limits the value of using this term for search, and is why I recommend visible but not searchable.
You are right about SHUTTER_STATE_ID. I agree that is should be searchable.
--Mark
On Apr 18, 2018, at 1:27 PM, Rob French rfrench@seti.org wrote:
The list is just which fields are used, not where they show up in the UI. All the fields are properly under General, Cassini, or ISS as appropriate.
I like having the IMAGE_NUMBER available. It solves a problem I've had for a long time - being able to search on a range of image numbers ignoring the camera name.
The ISS camera is already listed as "Wide angle" or "Narrow angle". I can add expanded forms of the instruments, although it's going to make them fairly wordy.
The raw FILTER_NAME is already not searchable.
Breaking apart IMAGE_OBSERVATION_TYPE would require a whole new way of handling data, but I'm already sorting the individual names and converting them to title case. I can sort them in the order you suggest.
Isn't it useful to be able to search on MISSING_LINES to look for the highest-quality images?
For SHUTTER_STATE_ID I think the point is not to search for unshuttered images, but instead to search only for shuttered images. That sounds valuable to me. You can't do one without the other.
Rob
From: Mark Showalter Sent: Wednesday, April 18, 2018 12:31:49 PM To: Rob French Cc: Matthew Tiscareno; Mitchell Gordon; Michael Aye; Matthew Hedman; rchancia@uidaho.edu Subject: Re: Audit of COISS fields in OPUS
Rob,
Comments interspersed below...
On Apr 18, 2018, at 11:17 AM, Rob French rfrench@seti.org wrote:
Hi all,
Here are all of the fields in the COISS label file and how they're handled in OPUS. A given field can be searchable, not searchable but still displayed, or ignored completely. Please look over the list carefully and let me know of any changes you think we should make. Are we allowing search on things that are pretty useless? (The goal is to keep the UI fairly simple for most users). Are there other things we should allow search on? Is any of the metadata we're displaying not worth displaying? Is any of the metadata we're ignoring worth displaying?
Note I've CC'd a couple of major users of OPUS to get their opinion as well.
These fields are searchable (and also displayed on "Details" tab):
INSTRUMENT_ID (converted to NAC/WAC)
By the way, the UI should spell out "Wide-angle Camera" and "Narrow-angle Camera" because users might be unfamiliar with the abbreviations.
This raises another issue, which is that we should also have at least parenthetical names for the instruments in the UI. Examples: Cassini ISS (Imaging subsystem) Cassini UVIS (Ultraviolet spectrometer) etc. In general, we should try to avoid abbreviations that a user unfamiliar with a data set might not recognize.
FILTER_NAME (both raw and converted to normalized form)
I am very skeptical that we need the "raw" filter name in the UI. Maybe somebody thinks differently. The normalized name should be searchable and visible. "Raw" filter name (two values inside parentheses I guess?) is OK to be visible, because it does tell the user which filter is on which wheel.
START_TIME STOP_TIME EARTH_RECEIVED_START_TIME EARTH_RECEIVED_STOP_TIME
All four of the above belong on the General search tab, not the Cassini or COISS search tabs.
SPACECRAFT_CLOCK_START_COUNT SPACECRAFT_CLOCK_STOP_COUNT OBSERVATION_ID
These three above belong on the Cassini search tab.
EXPOSURE_DURATION
This goes on the image search tab. Searchable & visible. Convert the units to seconds if necessary.
DESCRIPTION FILE_SPECIFICATION_NAME
These two go to the General search tab. Searchable & visible.
IMAGE_OBSERVATION_TYPE
I see comma-separated values currently in OPUS. I think it would be better to split them up if possible. A user searching on SCIENCE should not need to click on SCIENCE,OPNAV and OPNAV,SCIENCE and SCIENCE,CALIBRATION as well.
However, I suspect this would require creating a new table to support multiple values. If this is not practical (or we can always save it for a future refinement) then we at least need to normalize the options so that, for example, "OPNAV,SUPPORT" and "SUPPORT,OPNAV" are not separate choices.
Here's the priority ordering I would propose: SCIENCE, OPNAV, CALIBRATION, SUPPORT In other words, if SCIENCE is one of the values, put it first. If not, put OPNAV first if present. If not, put CALIBRATION first, etc.
Overall ordering on the UI should be SCIENCE, SCIENCE+something else, OPNAV, OPNAV+something else, CALIBRATION, CALIBRATION+something else, SUPPORT, UNK.
In the UI, use title case and change UNK to "Unknown".
DATA_CONVERSION_TYPE GAIN_MODE_ID
Searchable & visible
INST_CMPRS_PARAM (4 parts) INST_CMPRS_RATIO INST_CMPRS_TYPE
These three should not be searchable. I don't even think they be visible, unless somebody has a reason to keep them. The user can always find this info in the labels.
EXPECTED MAXIMUM (2 parts)
Not searchable. Not visible unless somebody disagrees.
MISSING_LINES
Not searchable but yes visible.
SHUTTER_MODE_ID
Searchable & visible
SHUTTER_STATE_ID
I can't imagine a reason to search on un-shuttered images but perhaps I misunderstand. My vote is visible but not searchable.
TELEMETRY_FORMAT_ID
Visible but not searchable, unless somebody disagrees.
These fields are NOT searchable (but ARE displayed on the "Details" tab and via the API):
If the results page is too long, we could shrink this list. Many of these parameters are just relevant for calibration. The user can find them in the label if s/he is planning to do their own calibration. Maybe Matt could suggest ways to shrink this list.
ANTIBLOOMING_STATE_FLAG BIAS_STRIP_MEAN CALIBRATION_LAMP_STATE_FLAG COMMAND_SEQUENCE_NUMBER DARK_STRIP_MEAN DELAYED_READOUT_FLAG DETECTOR_TEMPERATURE ELECTRONICS_BIAS EXPECTED_PACKETS FILTER_TEMPERATURE FLIGHT_SOFTWARE_VERSION_ID INSTRUMENT_DATA_RATE INST_CMPRS_RATE (2 parts) LIGHT_FLOOD_STATE_FLAG METHOD_DESC MISSING_PACKET_FLAG OPTICS_TEMPERATURE (2 parts) ORDER_NUMBER PARALLEL_CLOCK_VOLTAGE_INDEX PREPARE_CYCLE_INDEX PRODUCT_VERSION_TYPE READOUT_CYCLE_INDEX RECEIVED_PACKETS SENSOR_HEAD_ELEC_TEMPERATURE SEQUENCE_NUMBER SEQUENCE_TITLE SOFTWARE_VERSION_ID VALID_MAXIMUM (2 parts) PRODUCT_TYPE STANDARD_DATA_PRODUCT_ID
These fields are ignored entirely:
IMAGE_NUMBER (used to created rms_obs_id but not otherwise available)
Should be both searchable and visible.
INSTRUMENT_MODE_ID (Used to compute pixel size but not otherwise available)
Should be searchable and visible.
SPACECRAFT_CLOCK_CNT_PARTITION (We assume it's always 1)
Yes it is always equal to one. Hide.
TARGET_DESC (Used to compute TARGET_NAME but not otherwise available)
Searchable and visible.
COMMAND_FILE_NAME
Hide. (It's in the label if somebody cares.)
DATA_SET_ID
Visible, because it's important from the PDS viewpoint.
IMAGE_MID_TIME IMAGE_TIME
I vote for visible but not searchable.
MISSION_PHASE_NAME
Searchable and visible, from the Cassini tab. How did we miss this one? This field should be visible and searchable for every Cassini instrument.
PRODUCT_CREATION_TIME
Searchable and visible.
PRODUCT_ID SEQUENCE_ID
Both of the above should be visible but not searchable.
I agree on everything below...
CENTRAL_BODY_DISTANCE COORDINATE_SYSTEM_NAME DECLINATION EMISSION_ANGLE INCIDENCE_ANGLE LOWER_LEFT_LATITUDE LOWER_LEFT_LONGITUDE LOWER_RIGHT_LATITUDE LOWER_RIGHT_LONGITUDE MAXIMUM_RING_RADIUS MINIMUM_RING_RADIUS NORTH_AZIMUTH_CLOCK_ANGLE TARGET_NORTH_CLOCK_ANGLE PHASE_ANGLE PIXEL_SCALE PLANET_CENTER RIGHT_ASCENSION RING_CENTER_LATITUDE RING_CENTER_LONGITUDE RING_EMISSION_ANGLE RING_INCIDENCE_ANGLE RINGS_FLAG -SC_PLANET_POSITION_VECTOR -SC_PLANET_VELOCITY_VECTOR -SC_SUN_POSITION_VECTOR -SC_SUN_VELOCITY_VECTOR -SC_TARGET_POSITION_VECTOR -SC_TARGET_VELOCITY_VECTOR SPICE_PRODUCT_ID SUB_SOLAR_LATITUDE SUB_SOLAR_LONGITUDE SUB_SPACECRAFT_LATITUDE SUB_SPACECRAFT_LONGITUDE CENTER_LATITUDE CENTER_LONGITUDE TARGET_DISTANCE TARGET_EASTERNMOST_LONGITUDE TARGET_NORTHERNMOST_LATITUDE TARGET_SOUTHERNMOST_LATITUDE TARGET_WESTERNMOST_LONGITUDE TWIST_ANGLE TARGET_LIST UPPER_LEFT_LATITUDE UPPER_LEFT_LONGITUDE UPPER_RIGHT_LATITUDE *UPPER_RIGHT_LONGITUDE DATA_SET_NAME
Everything marked with "*" is superceded by more accurate data from our own supplemental metadata. Everything marked with "-" we have already agreed not to include because it's not very accurate anyway.
Thanks, Rob
.
Currently there is metadata stored in the database that is not used for searching OR displaying. For example:
All of these should be displayed on the Details page even if they aren't searchable.