Notes for datacall 2023

[elfunesto]

• [X] add fi_lfs_code for individual metrics in templates 1-3
• [x] #223
• [x] #222
• [x] #221
• [x] #220
• [X] #219 remove all datachecks
• [x] #218
• [x] #217
• [x] #226
• [X] #227
• [X] remove columns sai_emu_nameshort in sheets of groups and individuals metrics since sai_emu_nameshort is defined at the sampling_info scale
• [X] templates release: new_data we should have columns eel_value_number and eel_value_kg, not only eel_value
• [x] #229
• [X] templates release: correct sheet name: deleted_data should not have a capital D
• [x] #281
• [ ] in some sheets the restraints are shifted (e.g. you can select an EMU in the lfs column) needs a fix
• [ ] We need to put a check on qual_id, so it cannot be integrated if no qual_id is provided

[ciaraoleary]

for the time series data do you want years when no survey was done? anticipating in a few years will we need to fill the gaps. could put a note in the comment to say survey not carried out this year.

[LBeaulaton]

in the shiny integration app, 'import dcf' button should change for 'import other sampling' as there not only DCF data see #230

[cedricbriandgithub]

@ciaraoleary You should do that, to avoid us asking data year after year, add a das_qal_id=0 missing and a comment as suggested.

[ciaraoleary]

for coastal and marine habitat where emu involves 2 ices fao areas, what are you supposed to enter?

[LBeaulaton]

new series in recruitment Rho_Y should be renamed RhoY for consistency #231

[LBeaulaton]

when integrate new individual metrics you get the message '234355 and 486270 new values inserted in the group and metric tables' --> change 'group an dmetrics' to the appropriate table

[jdpo]

For Annex 3_DE there is only one new row of data. There is also existing group metrics until 2020; hence, lines for 2021/2022 are prefilled for WarnS in new group metrics. It should give an error that values are missing, but it does not. Instead it produces several lines of modified group metrics. If the two "empty" pre-filled lines in new grouped metrics are removed it does not show modified series anymore... Screenshots and the file that causes the error sent via email to Cedric.

[ciaraoleary]

for annex 2 yellow time series; check duplicates is giving error message in red box- join columns must be present in data grser_ser_id; checking files loading in limbo #232

[ciaraoleary]

for annex 7 releases fixed eel_value_n and eel_value_kg but when i click check file get error message saying new data should have 11 columns. We have 13 columns in our dataset

[jdpo]

for Annex 5, check file produces a warning that eel area division is missing but there is no data (it is NP), so the message seems redundant

[torbjornsaterberg]

~~for Annex 2, integrating new group metrics gives the following message: Error: Failed to fetch row: ERROR: duplicate key value violates unique constraint "c_ck_uk_grser_gr" DETAIL: Key (grser_ser_id, gr_year)=(367, 2012) already exists.~~ see comment below

[rgemert]

For Annex 3, after pressing 'Check duplicate', it gives a red error box reading: Join columns must be present in data. Problem with 'grser_ser_id'. Could be related to that we want to update group metrics, but there is no grser_ser_id column present in the sheets? fixed

[ciaraoleary]

for annex 10 step 1 there is no window to save a file for deleted group metrics; tab exists in excel file.

We want to delete some of the existing data that was prefilled

[cedricbriandgithub]

OK Ignore this error

De : ciaraoleary @.> Envoyé : mardi 6 septembre 2022 15:37 À : ices-eg/wg_WGEEL @.> Cc : Cédric Briand @.>; Comment @.> Objet : Re: [ices-eg/wg_WGEEL] Notes for issue 2023 (Issue #225)

for annex 7 releases fixed eel_value_n and eel_value_kg but when i click check file get error message saying new data should have 11 columns. We have 13 columns in our dataset

— Reply to this email directly, view it on GitHubhttps://github.com/ices-eg/wg_WGEEL/issues/225#issuecomment-1238163260, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGGZJROAYESUFMTDMZYI5UTV45CIFANCNFSM6AAAAAAQBFL5HM. You are receiving this because you commented.Message ID: @.**@.>>

[cedricbriandgithub]

> for annex 7 releases fixed eel_value_n and eel_value_kg but when i click check file get error message saying new data should have 11 columns. We have 13 columns in our dataset

Isn't it 12 ?

[ciaraoleary]

for annex 7 releases fixed eel_value_n and eel_value_kg but when i click check file get error message saying new data should have 11 columns. We have 13 columns in our dataset

Isn't it 12 ?

when we add eel value kg the number of columns becomes 13

[ciaraoleary]

Ireland Annex 10 deleted data to be removed from database discussed with Hilaire

234

[ciaraoleary]

Annex 8 aquaculture I hit the check file button and it gives message 'loads biomass' and wont go any further

[cedricbriandgithub]

for coastal and marine habitat where emu involves 2 ices fao areas, what are you supposed to enter?

The unicity constraint where fao_area is not null is on columns (eel_year, eel_lfs_code, eel_emu_nameshort, eel_typ_id, eel_hty_code, eel_qal_id, eel_area_division) So you can enter two lines for the same EMU with different eel_area_division

[cedricbriandgithub]

new series in recruitment Rho_Y should be renamed RhoY for consistency

Done

[cedricbriandgithub]

when integrate new individual metrics you get the message '234355 and 486270 new values inserted in the group and metric tables' --> change 'group an dmetrics' to the appropriate table OK

[cedricbriandgithub]

For Annex 3_DE there is only one new row of data. There is also existing group metrics until 2020; hence, lines for 2021/2022 are prefilled for WarnS in new group metrics. It should give an error that values are missing, but it does not. Instead it produces several lines of modified group metrics. If the two "empty" pre-filled lines in new grouped metrics are removed it does not show modified series anymore... Screenshots and the file that causes the error sent via email to Cedric

fixed by removing empty lines

[cedricbriandgithub]

for annex 2 yellow time series; check duplicates is giving error message in red box- join columns must be present in data grser_ser_id; checking files loading in limbo

fixed

[cedricbriandgithub]

for annex 7 releases fixed eel_value_n and eel_value_kg but when i click check file get error message saying new data should have 11 columns. We have 13 columns in our dataset

Isn't it 12 ?

when we add eel value kg the number of columns becomes 13

OK fixed

[cedricbriandgithub]

for Annex 5, check file produces a warning that eel area division is missing but there is no data (it is NP), so the message seems redundant

Individual checks are mostly done column by column so we will probably not fix it.

[cedricbriandgithub]

(grser_ser_id, gr_year)=(367, 2012)

SELECT FROM datawg.t_series_ser WHERE ser_id =367 -- 367 BarY Barsebäck SELECT FROM datawg.t_groupseries_grser WHERE grser_ser_id = 367 -- nothing returned

You have two lines for the same series :-)

[cedricbriandgithub]

For Annex 3, after pressing 'Check duplicate', it gives a red error box reading: Join columns must be present in data. Problem with 'grser_ser_id'. Could be related to that we want to update group metrics, but there is no grser_ser_id column present in the sheets?

fixed

[cedricbriandgithub]

Annex 8 aquaculture I hit the check file button and it gives message 'loads biomass' and wont go any further

I think Hilaire fixed this one

[torbjornsaterberg]

(grser_ser_id, gr_year)=(367, 2012)

SELECT FROM datawg.t_series_ser WHERE ser_id =367 -- 367 BarY Barsebäck SELECT FROM datawg.t_groupseries_grser WHERE grser_ser_id = 367 -- nothing returned

You have two lines for the same series :-)

Ok. Thanks. How do we deal with this. We have two series on lengthmm based on different data?

[elfunesto]

(grser_ser_id, gr_year)=(367, 2012)

SELECT FROM datawg.t_series_ser WHERE ser_id =367 -- 367 BarY Barsebäck SELECT FROM datawg.t_groupseries_grser WHERE grser_ser_id = 367 -- nothing returned You have two lines for the same series :-)

Ok. Thanks. How do we deal with this. We have two series on lengthmm based on different data?

Hi, the idea in this table is to have a mean value at the series level (so only one value per time series and years). So theoretically, you should find the proper way to aggregate the values to have the best estimates of the overall mean.

[cedricbriandgithub]

You could provide some comments on the change of method after 2012

[jdpo]

When trying to integrate new group metrics for Annex 1, we get the following Error (PT):

Error: Failed to prepare query: ERROR: column "gr_number" is of type integer but expression is of type boolean LINE 2: (SELECT g.gr_year,g.gr_number,g.gr_comment,g.gr_dts_datasour... ^ HINT: You will need to rewrite or cast the expression

[cedricbriandgithub]

Jan dag can you open new issues for each problem ?

[jdpo]

Sure, I was wondering what would be the best option.

Von: "Cédric Briand" @.> An: "ices-eg/wg_WGEEL" @.> CC: "Jan-Dag Pohlmann" @.>, "Comment" @.> Gesendet: Mittwoch, 7. September 2022 10:38:27 Betreff: Re: [ices-eg/wg_WGEEL] Notes for issue 2023 (Issue #225)

Jan dag can you open new issues for each problem ?

— Reply to this email directly, [ https://github.com/ices-eg/wg_WGEEL/issues/225#issuecomment-1239085578 | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AGII4FX322TLUSSBAQWMBCDV5BIAHANCNFSM6AAAAAAQBFL5HM | unsubscribe ] . You are receiving this because you commented. Message ID: @.***>

[torbjornsaterberg]

Where do we upload annex 6 - Other landings. Should that go under "Catch and Landings"?