Dataset - High performance liquid chromatography (HPLC) phytoplankton pigment timeseries for the northern Salish Sea and central coast, British Columbia

hakai-it commented 2 months ago

High performance liquid chromatography (HPLC) phytoplankton pigment timeseries for the northern Salish Sea and central coast, British Columbia

https://hakaiinstitute.github.io/hakai-metadata-entry-form/#/en/hakai/7U7b8oPpeTN6gjvXlUCTGJr5pga2/-O25qzW3OpioeACGpRHl

Best Practices Checklist

In General

[x] No previous versions of this metadata record exist (eg for earlier versions of the data, if so update that record rather than creating a new one)

Data Identification

Dataset title:

[x] No version information in the title
[x] Frontloaded (with the most important information first)
[x] Include the geographical region the data apply to
[ ] Short – aim for 60 characters including spaces
[x] Does not include acronyms – put these in the keywords
[x] Does not include the word “dataset”
[x] Time series datasets should include “time series” at the end of the title

Abstract

[x] Abbreviations have been expanded upon at first mention
[x] Abstract describes how, when, what, where, why of data collection and is limited to no more than 500 words

DOI

[x] A DOI has been drafted for this record
[ ] DOI has been updated via the form after review and changes to record
[ ] DOI has been manually edited on datacite fabrica
[ ] #107

Spatial

[x] Ensure that Depth or Height Positive is correctly selected

Contact

[x] ROR and ORCID(s) are included and linked properly where applicable
[x] For datasets where DFO is a partner, ensure 'parent' ROR is added (https://ror.org/02qa1x782). DFO 'child' organizations (i.e. CHS) and their ROR are optional.
[x] Include Hakai Institute as Publisher and include data@hakai.org as email
[x] Make sure email address is provided if the role is 'Metadata Custodian' or 'Point of Contact'
[x] Add contact affiliation where known including ROR
[x] If resource is (partially) generated by Hakai researchers, include 'Tula Foundation' (with associated ROR) with 'Funder' role. Be sure to uncheck 'include in citation' for Tula Foundation.

Resources

[x] Resource links go to specific dataset download (not generic platform like waterproperties.ca)
[ ] Readme, changelog, data dictionary, protocols included in data-package (for tabular text based data)
[x] An archive folder, or other means, for older data versions is included in the data package if the version is not 1.0
[x] Links work
[ ] All files in the data package can be opened and are not corrupt
[ ] No executable files in the data package. Files should be open formats and standards (.csv, .txt for example)

timvdstap commented 2 months ago

Thanks for this submission @jdelbel - I will have a look at it on Monday. As discussed, until the data is available on the Hakai ERDDAP Server, you could indeed link to the Hakai Data Portal and indicate that your data is available upon request (unless @JessyBarrette you have any other suggestions?) Alternatively, you could create a private GitHub repository with the Hakai-dataset-template and store your dataset on there with the relevant documentation, and link to that repo.

JessyBarrette commented 2 months ago

The ERDDAP dataset is already available for this dataset and is available on the development erddap here: https://goose.hakai.org/erddap/tabledap/HakaiHPLCResearch.html

JessyBarrette commented 2 months ago

The ERDDAP dataset is ready just need to sync the metadata and push it to production erddap.

JessyBarrette commented 2 months ago

@jdelbel some comments:

[x] I would omit the HPLC acronym in the title
[x] From abstract "Data are currently not available on a repository and are available upon request." We will add an erddap for it, so I would omit that.
[x] I added hplc as keyword
[x] I added the future erddap dataset and moved it to the top since this should be main way for external users to access the data. https://catalogue.hakai.org/erddap/tabledap/HakaiHPLCResearch.html

jdelbel commented 2 months ago

It wouldn't let me click the check boxes.

I made the requested changes to the title and abstract. Your changes look good.

This link https://catalogue.hakai.org/erddap/tabledap/HakaiHPLCResearch.html is coming up as not found.

JessyBarrette commented 2 months ago

Yes I haven't pushed to production yet

You can find it for for now at

https://goose.hakai.org/erddap/tabledap/HakaiHPLCResearch.html https://catalogue.hakai.org/erddap/tabledap/HakaiHPLCResearch.html

Jessy Barrette M.Sc. Marine Data and Instrumentation Specialist Hakai Institute https://www.hakai.org/ | CIOOS Pacific https://cioospacific.ca/ | CIOOS https://cioos.ca/ | @.*** | (C) (418) 999-5745

Le lun. 22 juill. 2024 à 15:41, jdelbel @.***> a écrit :

It wouldn't let me click the check boxes.

I made the requested changes to the title and abstract. Your changes look good.

This link https://catalogue.hakai.org/erddap/tabledap/HakaiHPLCResearch.html is coming up as not found.

— Reply to this email directly, view it on GitHub https://github.com/HakaiInstitute/metadata-review/issues/106#issuecomment-2243685203, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHICYONSR6YA22MJHTX465DZNVN7HAVCNFSM6AAAAABLE74RX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGY4DKMRQGM . You are receiving this because you were mentioned.Message ID: @.***>

jdelbel commented 2 months ago

Nice, yeah I realized that after I sent the message.

Thanks Jessy.

timvdstap commented 2 months ago

It's looking good @jdelbel - few minor things:

[x] Is this record in any way related to this record, or is that simply an older version that can be deleted?
[ ] In the title I see northern Salish Sea, in the abstract Northern Salish Sea (NSS). Minor thing, but do you also want to capitalize in the title?
[ ] Is it possible to add the phytoplankton species groupings under Taxonomic Classification?
[ ] I have added Hakai Institute as Publisher (you had it as Publisher, but not selected to be included in the citation because for the same contact info you had also selected Hakai as Data Owner -- the workaround is listing Hakai Institute twice in the contacts).
[ ] I've included Drew's email because they're listed as a Point of Contact.
[ ] Now that a ERDDAP dataset is available (though not yet pushed to production), I would remove the link to the Hakai Data Portal as Primary Resource. Unless there's a reason the ERDDAP dataset won't be pushed to production for a while?

JessyBarrette commented 2 months ago

jdelbel commented 2 months ago

Thanks @timvdstap

[x] Yes, that was largely uncomplete record I had forgotten about - deleted
[x] Good catch. I actually took the capital away from northern in the abstract - I think that might be more correct, but can do the opposite if you think not.
[x] Interesting question. HPLC represents pigment concentrations and is not directly taxonomic. I basically put these data into a statistical model to estimate group level biomass. As such, I don't think the HPLC data should have taxonomic information. Thoughts?
[x] Thanks!
[x] Thanks!
[ ] I removed the portal. I don't think so, more just which data will be pushed and when as quite a bit of it is being used in research projects.

jdelbel commented 2 months ago

Why are the click boxes so glitchy - it won't let check yours and it gives me a huge sense of satisfaction to tick things off, haha.

timvdstap commented 2 months ago

[x] Good catch. I actually took the capital away from northern in the abstract - I think that might be more correct, but can do the opposite if you think not.

As far as I'm concerned it's all good, it was more the consistency that I was thinking about.

[x] Interesting question. HPLC represents pigment concentrations and is not directly taxonomic. I basically put these data into a statistical model to estimate group level biomass. As such, I don't think the HPLC data should have taxonomic information. Thoughts?

That makes sense to me!

[ ] I removed the portal. I don't think so, more just which data will be pushed and when as quite a bit of it is being used in research projects.

I still saw the portal listed as Primary Resource, but I removed it now. Where possible we try to make sure resource links go to specific datasets, rather than 'generic' platforms or portals.

Once https://catalogue.hakai.org/erddap/tabledap/HakaiHPLCResearch.html is pushed to production and 'live', please add the publication date to the record. I can then publish the record to the Hakai Catalogue and make sure that the DOI is 'Findable' as well. :)

timvdstap commented 1 month ago

Hey @jdelbel hope you had a great vacation! Just tagging you here to put it on your radar :-)

jdelbel commented 1 month ago

Thanks @timvdstap. I saw an update from Jessy on August 26th showing that it was pushed to production. Can you confirm if that is correct date to add to the record. Oddly, I no longer see the record within my account on the intake form, which looks like it has changed since my last sign on. Where can I access the form to make the change?

Also, currently no data is being pushed to erddap. I can't remember what filters Jessy had applied for deciding which data should be pushed. Can we review this and then I can make the changes so data become available?

fostermh commented 1 month ago

@jdelbel your metadata records is here https://hakaiinstitute.github.io/hakai-metadata-entry-form/#/en/hakai/7U7b8oPpeTN6gjvXlUCTGJr5pga2/-O25qzW3OpioeACGpRHl

jdelbel commented 1 month ago

@fostermh Thanks :). Weird my old link didn't go that.

timvdstap commented 1 week ago

Also, currently no data is being pushed to erddap. I can't remember what filters Jessy had applied for deciding which data should be pushed. Can we review this and then I can make the changes so data become available?

Hey @jdelbel I'm not sure what filters Jessy was applying. It seems like the most recent data is from May 28, 2024, is that accurate?

jdelbel commented 1 week ago

@timvdstap Ok, the publication date has been added.

May 28th, 2024 is the last data we received from the analysis lab. We send out 2x shipments per year - one in June and the other in December. We get results back from the second shipment in Feb of the following year. I need time to QA/QC the data requiring other phyto data-types. Thus, I think we should publish to the end of the prior year. That would mean we could now publish to the end of 2023. Also, I think the all_chl_a_flag should be AV and the quality level "Principal Investigator". I need to update these, but can do so quickly.

How does that sound?

timvdstap commented 1 week ago

I'm looping in @fostermh here, because I'm not sure how to publish data only to 2023-12-31, as opposed to it currently being 2024-05-28. I think I'm confused about the difference between 'published' and 'pushed to production': So if I understand correctly, the HakaiHPLCResearch data on ERDDAP, that includes the data till May 2024 is 'pushed to production' but that doesn't mean that it's published/openly available through our ERDDAP server? It's like an interim version (available only to Hakai?), and from this production we should select the data till 2023-12-31 to publish openly?

jdelbel commented 1 week ago

In theory, I could control what is openly available through the flagging and quality level if those are included as filters. This was the approach that Jessy was taking, but not sure how this was done on the backend.

fostermh commented 1 week ago

Currently the hplc output from out database that comprises the relevant ERDDAP dataset is limited to anything with results.

https://github.com/HakaiInstitute/hakai-erddap/blob/0ce897a5f1fcf9c2933c7042cc7f5c6e4c9baf64/views/HakaiHPLCSampleResearch.sql#L4

If you would like, I can easily change that to be anything with a Quality Level of 'Principal Investigator' which would imply that @jdelbel you had reviewed and signed off on it.

timvdstap commented 4 days ago

Admittedly I'm not fully sure what the next steps are for this record, so please correct me if where I'm wrong:

Currently HakaiHPLCResearch data till 2024-05-28 has been pushed to production, but this data is not publicly available yet through a Hakai ERDDAP Server
@jdelbel to indicate what the criteria are for the data that can be published (e.g., anything with Quality Level 'PI')
@fostermh to implement that criteria in https://github.com/HakaiInstitute/hakai-erddap/blob/0ce897a5f1fcf9c2933c7042cc7f5c6e4c9baf64/views/HakaiHPLCSampleResearch.sql#L4
@jdelbel to push the public data to ERDDAP
@jdelbel to update the link in the metadata record and do a final review
@timvdstap to finalize record in DataCite and publish to the Hakai Catalogue

jdelbel commented 4 days ago

Yes, I agree with the PI criteria threshold for publication. Good on my end to make that change.

@timvdstap "push the public data to ERDDAP" - Does this mean actually adding quality level "PI" so the data are pushed?

Otherwise, looks good to me and will work on the update and final review.

timvdstap commented 4 days ago

@timvdstap "push the public data to ERDDAP" - Does this mean actually adding quality level "PI" so the data are pushed?

That's a good question, I'm actually not sure what the process of publishing data to ERDDAP looks like at a technical level, likely @fostermh will know. My guess is indeed that if you add PI as quality level, that would satisfy the established criteria and it would automatically get pushed to the Hakai ERDDAP Server (though not sure of the frequency).

jdelbel commented 4 days ago

Ok, that will be good to know as I am unclear as well.

HakaiInstitute / metadata-review