search

aodn / nrmn-application

A web application for collation, validation, and storage of all data obtained during surveys conducted by the NRMN
GNU General Public License v3.0
4 stars 3 forks source link

Ingested Quadrat data not complying with quadrat summing to 50 rule #1162

Closed bpasquer closed 1 year ago

bpasquer commented 2 years ago

From https://github.com/aodn/nrmn-application/issues/1085#issuecomment-1302768037

For an undetermined period, quadrat data that do not comply with the "quadrat do not sum to 50" rule have been successfully ingested in the production database. The validation rule is correctly applied in the current version of the software.

nspool commented 2 years ago

Quadrats do not sum to at least 50 in transect for these ingested surveys: [NIN-S1, 2021-03-27, 5.4] [NIN-S2, 2021-03-22, 5.3] [NIN-S3, 2021-03-27, 5.3] [SBIR07, 2019-03-14, 5.3] [SBIR19, 2020-02-14, 5.2]

bpasquer commented 2 years ago

@LizziOh Since the issue is impacting 5 transects only, do you think you could fix these quadrat data with the correction tool?

bpasquer commented 2 years ago

Also, checking all the validation rule, Nathan found that survey 923400272 [TI-S4,2021-03-01,5.1] fails the "Quadrat not above 50" rule on Q2.

LizziOh commented 2 years ago

Yes, I'm happy to fix these with the corrections tool but wanted to check that the validation is working in production to avoid future errors slipping in.

nspool commented 2 years ago

On investigation, the quadrat validation was not working correctly from December last year but was fixed in the release 9.0.632 on June 17th this year. Surveys ingested during this time might not have shown either the "M3 quadrat more than 50" or "Quadrats do not sum to at least 50" blocking validations.

I reran all blocking validations on all ingested jobs and discovered the 5 surveys above fail "Quadrats do not sum to at least 50" and also [TI-S4,2021-03-01,5.1] fails "M3 quadrat more than 50". All surveys ingested after June 17th pass validation and there has been tests added to prevent this test from failing in future.