Invalid hauls in database and StoX

[elorsepp]

Some people have pointed out that StoX uses biological data from hauls that are marked as invalid. That can cause differences in calculations when compared to our conventional methodology. Could it be excluded from the analysis?

And a more general question, do we need to have invalid information in the database?

Elor

[CiaranOD]

Espen- Currently possible to filter within StoX either by using Valid/Invalid Hauls or by using specific haul numbers etc. This has to be user defined within StoX.

Hjalte- Need a descriptor (Optional comment field ?) as to why hauls is Valid or Invalid within the DB. Optional comment filed will then not be valid for filtering in StoX.

Mike- Potential to introduce a code system to describe how the trawl was undertaken and its success e.g. length of tow, some damage occurred, school missed etc.

Espen- some elements maybe valid within a haul but others may not, its important therefore to define validity codes for specific elements and let user decide.

Hassan- quality flag important

Leon- Mac trawl survey. Flags describe the data and are therefore important.

Need to find ways to describe the data with more detail, allowing others to determine if the haul data meets their specific needs. Informing the user.

[elorsepp]

During next meeting everybody who has will show examples of quality flags.

[jmjech]

I will try to get some examples from our bottom trawl survey of quality flags for the next meeting.

[SMLusseau]

In the same round can we ponder this warning in the database: "Valid hauls are supposed to have related Catch records." Although it is only a warning and data can be uploaded I think the statement is incorrect in the context of defining the term "valid haul" Example: Many of us take "blind" tows in the surface at regular intervals to confirm there are no fish of interest above the transducer and therefore unavailable to the acoustics. A zero catch is a result of interest in this instance and the haul is valid (no gear malfunction, did not miss a targeted school etc). Maybe we can rephrase the warning to "There is no Catch record associated with this haul - please check".

[HjalteParner]

@jmjech did you have some examples on this topic?

I will at least try to come up with an example of the binary flag approach for next meeting to discuss and possible take a decission by then

[jmjech]

Over the years the NOAA NEFSC has worked on a way to quantify the "validity" of a trawl haul. From my understanding, the latest in that evolution is that they have a couple of Oracle tables that provide a code (Haul Value table) and another with the data that go into how that code was determined (Tow Evaluation table). Haul Value table 1 Good tow. No gear or tow duration problem. 2 Representative, but some problem encountered due to gear or tow duration. 3 Problem tow. May or may not be representative due to gear or tow duration. 4 Not representative, due to gear or tow duration. 5 No bottom trawl 7 Aborted Haul Where the numbers 1-7 indicate how "representative" the trawl haul was in the sense of did it fall within the criteria for duration and gear damage.

The "data" table has the columns: Tow Evaluation table CRUISE6 STATION MEAN_SOG_KNOTS STDEV_SOG_KNOTS MEAN_DEPTH_METERS STDEV_DEPTH_METERS MEAN_DOOR_SPRD_METERS STDEV_DOOR_SRPD_METERS PCT_VALID_DOOR_DATA AREA_SWEPT_DOORS_MEAN_KM2 MEAN_WING_SPRD_METERS STDEV_WING_SPRD_METERS PCT_VALID_WING_DATA AREA_SWEPT_WINGS_MEAN_KM2 MEAN_HEIGHT_METERS STDEV_HEIGHT_METERS PCT_VALID_HEIGHT_DATA MEAN_BRIDLE_ANGLE_DEGREES MEAN_TRAWLDEPTH_METERS STDEV_TRAWLDEPTH_METERS PCT_VALID_TRAWLDEPTH MEAN_STBDWNCH_LINEOUT_METERS PCT_VALID_STBDWNCH_LINEOUT MEAN_PORTWNCH_LINEOUT_METERS PCT_VALID_PORTWNCH_LINEOUT MEAN_LO_DIFFERENCE_M MEAN_STBDBLK_LINEOUT_METERS PCT_VALID_STBDBLK_LINEOUT MEAN_PORTBLK_LINEOUT_METERS PCT_VALID_PORTBLK_LINEOUT MEAN_STBDBLK_TENSION_MT PCT_VALID_STBDBLK_TENSION MEAN_PORTBLK_TENSION_MT PCT_VALID_PORTBLK_TENSION MEAN_BLKTEN_DIFFERENCE_MT MEAN_STBDWNCH_TENSION_MT PCT_VALID_STBDWNCH_TENSION MEAN_PORTWNCH_TENSION_MT PCT_VALID_PORTWNCH_TENSION TYPE_CODE TYPE_COMMENT OPERATION_CODE OPERATION_COMMENT GEAR_CODE GEAR_COMMENT ACQUISITION_CODE ACQUISITION_COMMENT Where "Gear_Code" is the "haul value".

It appears that the validity/representativeness of the trawl haul is not based on catch, but on net mensuration metrics.

I'm not an expert on the bottom trawl data collected at the NOAA NEFSC, so any more in depth questions will need to have someone from the NEFSC bottom trawl group involved.

I'm not sure a binary valid/invalid quality flag is sufficient, but how may grades are needed?

[arnejohannesholmin]

The procedure to filter based on Validity in StoX is to read the file using ReadBiotic, convert to StoxBiotic, then AddToStoxBiotic and filter on the Validity afterwards, such as in the attachment:

[elorsepp]

@arnejohannesholmin please comment and describe the haul flagging system of IMR