search

aims-group / publication-site

A Django skeleton site for keeping track of publications, posters, presentations, ...
4 stars 3 forks source link

Final pre-announcement tweaks #130

Closed durack1 closed 3 years ago

durack1 commented 3 years ago

Hi @mauzey1, we've been finalizing the text for the publication page announcement email and have some minor tweaks to request

A dummy of what we hope these pages to look like (at least the layout, I didn't populate the variable field for e.g.) is below Screenshot_2021-05-21 New Publication

@taylor13 ping

durack1 commented 3 years ago

@mauzey1 in addition we wanted to tweak the text at metaForm.html to

Please click on the relevant CMIP era tabs below. Optional additional information about the models, experiments, variables, and other characteristics of data used in your study is also invited.

durack1 commented 3 years ago

Hi @mauzey1 we were hoping (belatedly) to send out this announcement email, is there a chance the tweaks above could be completed this week?

mauzey1 commented 3 years ago

@durack1 I will aim to have these changes online by the middle of Thursday.

durack1 commented 3 years ago

@mauzey1 perfect, I will aim to send this email close to that time.

mauzey1 commented 3 years ago

@durack1 So you want the realms from CMIP6 to also be used for CMIP5 when submitting a publication?

durack1 commented 3 years ago

@mauzey1 yeah that seemed the most logical thing to do, map everything into the most modern formats/names. Is there an impact on the previously registered info?

mauzey1 commented 3 years ago

@durack1 Adding realms for CMIP5 shouldn't impact the current CMIP5 entries, but I will need to update the realm for the CMIP6 entries to match the new format.

Would this new format apply to the CMIP5 variables? Do you want the CMIP6 variables to be used for CMIP5, too?

durack1 commented 3 years ago

@mauzey1 thanks for these detailed queries, I believe that the CMIP6 variables are a superset of the CMIP5 variables (with more added in CMIP6), so yes, let's use the CMIP6 entries across both.

Out of curiosity, how many of the ~1200 existing publications provided variable etc information? Many?

mauzey1 commented 3 years ago

@durack1 Do the CMIP5 variables have a name and description similar to the CMIP6 variables? The current CMIP5 variables listed look like the variable descriptions for CMIP6. Do we just mix the CMIP5 variables as they are with the CMIP6 variables?

mauzey1 commented 3 years ago

@durack1 Continuing from my previous comment... There are CMIP5 variables that match the description of several CMIP6 variables.

For example, surface upwelling shortwave flux in air from CMIP5 is also the description of several CMIP6 variables.

         'rsus - Surface Upwelling Shortwave Flux In Air',
         'rsusIs - Surface Upwelling Shortwave Flux In Air',
         'rsusLut - Surface Upwelling Shortwave Flux In Air',
         'siflswutop - Surface Upwelling Shortwave Flux In Air',
durack1 commented 3 years ago

@mauzey1 apologies telcos distracted me this afternoon.

Yes, the CMIP6 variables began from the CMIP5 variables, with some considerable long name cleanups, in addition to adding completely new variables for CMIP6, so let's go with CMIP6 as the standard variables as in most cases they include all the previous era quantities

mauzey1 commented 3 years ago

@durack1 So should we keep the original CMIP5 variables in the list alongside the CMIP6 variables, or should we replace the existing CMIP5 variables with their CMIP6 equivalent? There are numerous publications on the site that have variables already listed so we can't just get rid of the original CMIP5 variables without replacing them.

mauzey1 commented 3 years ago

Here's the list of all the CMIP6 variables that I gathered from the tables.

https://github.com/mauzey1/publication-site/blob/8ae82591a3f5cd4edb5687eda986b09509f3f9ce/scripts/variable.py#L150

mauzey1 commented 3 years ago

Here's what the submission page now looks like for CMIP6.

Screenshot_2021-05-26 New Publication

durack1 commented 3 years ago

Perfect, that looks great (on mobile) will take another peek on a bigger screen later. Did you want to merge into master? It looks right from a quick scan

mauzey1 commented 3 years ago

I think I have gotten most of the requested changes done for this issue so I will make a pull request. I do however plan to merge the CMIP5 and CMIP6 variable sets together. I also plan to completely remove the 'keyword' field from the publication database; the current changes only hide it from view.

durack1 commented 3 years ago

Perfect, thanks @mauzey1 exactly what is needed, a cleanup of the database alongside this visual tweak. Thanks again for this, looking forward to announcing it

mauzey1 commented 3 years ago

@durack1 Here's an example of the CMIP5 and CMIP6 variables combined Screenshot_2021-05-27 New Publication Screenshot_2021-05-27 New Publication(1)

This commit merges the two lists together. https://github.com/mauzey1/publication-site/commit/901aaed733af23d81afa408a203fd1274d0fbc11

Would you like me to merge this change?

durack1 commented 3 years ago

@mauzey1 apologies I mustn't have been clear. Rather than bothering with the CMIP5 variables, my suggestion was just to use the CMIP6 variables for each. Even though some of the CMIP6 variables didn't exist in CMIP5, it's more simple to use the CMIP6 variable id's (e.g. tos) and long names (e.g. Sea Surface Temperature) as they have been more thoroughly vetted to be descriptively correct

mauzey1 commented 3 years ago

@durack1 I wasn't sure how to merge the CMIP5 and CMIP6 variables until I found that the CMIP5 variable names in the form matched the standand names in the old CMIP5 tables. Thankfully, I kept the old tables for CMIP5 in the xml-cmor3-database repo. I will need to replace the CMIP5 variable names listed for publications with the new names. I will try to finish this by the end of today.

durack1 commented 3 years ago

@mauzey1 excellent thanks. If I was planning to merge, I would use the CMOR names (e.g. tos) and then use the standard names from the cmip6-cmor-tables, I believe this would provide almost complete coverage

I presume this is to migrate the existing CMIP5 database variable entries across to the new (CMIP6) keys?

durack1 commented 3 years ago

So just as an example, the CMIP5 tos variable, with long_name "Sea Surface Temperature", should map onto the CMIP6 tos variable, long_name "Sea Surface Temperature"

mauzey1 commented 3 years ago

@durack1 I was able to find most of the CMIP5 variables in the CMIP5 tables by searching for the standard names of the variables. I needed to replace spaces with underscores to match the stings. However, there were some variables that I couldn't find. Below is a list of them.

I couldn't find exact matches for these variables with any of the standard names listed in the CMIP5 tables. Do northward_derivative_of_eastward_sea_ice_velocity and eastward_derivative_of_northward_sea_ice_velocity mean eastward_sea_ice_velocity and northward_sea_ice_velocity respectively? surface_albedo did match the albsn variable in CMIP6. I also found water_sublimation_flux in the dreq.xml file of the data request used to create the CMIP6 tables.

Do I just leave these variables as they are in the publication submission form?

Is Other just a placeholder for variables that we currently don't know about? Do we still need it?

durack1 commented 3 years ago

@mauzey1 great, this is real progress, I have attempted to link CMIP5 -> CMIP6 below

Let me know if it's not clear above. @taylor13 happy for any corrections you may have noticed.

As I noted the CMIP6 data request (variable definitions) are more specific than their CMIP5 counterparts, so going with these should cover usages instances

I don't think "other" is useful, so let's remove it

mauzey1 commented 3 years ago

@durack1 I've replaced the above variables with their CMIP6 equivalent. However, I have also ran into more CMIP5 variables that I couldn't find a CMIP6-equivalent for. For these variables, I used the output name and long name from the CMIP5 tables.

However, some of these variables have multiple versions. For example, mole fraction of ozone in air has tro3 and tro3Clim. I only chose one of the choices for each variable if multiple versions were present. How would you like these variables handled?

I also had to make additional changes to the CMIP6 variables. In the previous changes, I used the standard name rather than the long name. This will be corrected in the next pull request. I also removed the variable choice Other from the menu.

durack1 commented 3 years ago

@mauzey1 thanks, unpeeling the onion indeed!

Rather than me providing matches first up, do we have any instances of existing publication database entries that list any of these variables against their registration? If yes, then I will provide a CMIP6 comparative match, if not, let's just omit them

mauzey1 commented 3 years ago

@durack1 I have found publications for all of the variables listed.

global average sea level change mole fraction of ozone in air northward ocean heat transport due to bolus advection northward ocean heat transport due to diffusion sea ice area fraction sea ice thickness sea ice transport across line sea ice x velocity sea ice y velocity sea surface temperature sea water potential density surface downward x stress surface downward y stress time water evaporation flux

Even Other has some publications listed for it. Although I got rid of that variable, the publications should remain in the database and still be accessible.

durack1 commented 3 years ago

@mauzey1 ok let's do the following CMIP5 -> CMIP6 mappings:

global average sea level change zos, Omon mole fraction of ozone in air toz, AERmon northward ocean heat transport due to bolus advection hfbasin, Omon northward ocean heat transport due to diffusion hfbasinpmdiff, Omon sea ice area fraction siconc, SImon sea ice thickness sithick, SImon sea ice transport across line sidmasstranx, SImon sea ice x velocity siu, SImon sea ice y velocity siv, SImon sea surface temperature tos, Omon sea water potential density opottempmint, Oyr surface downward x stress tauuo, Omon surface downward y stress tauvo, Omon time - this is not a valid variable, it's a coordinate water evaporation flux evs, Omon

mauzey1 commented 3 years ago

@durack1 I've mapped all of the CMIP5 variables to their CMIP6 equivalents. I think that should cover all of the changes for this issue.

mauzey1 commented 3 years ago

@durack1 The changes are now live on the site.

durack1 commented 3 years ago

@mauzey1 excellent, thanks will take a peek soon and aim to finalize and send the announcement early next week

durack1 commented 3 years ago

@mauzey1 perfect, exactly what we were chasing. Great will get back onto this announcement email and send asap

durack1 commented 3 years ago

@mauzey1 I have just taken a quick look at the submission page and it seems that the "tos" variable is not listed, see below Screenshot_2021-06-09 New Publication

Just wondering if anything else is missing?

mauzey1 commented 3 years ago

@durack1 I just applied the change to add the variable to the database. Please try looking at it again.

durack1 commented 3 years ago

@mauzey1 excellent, yep it's now there. Just wondering is this a solitary case of a missing variable or are there others?

mauzey1 commented 3 years ago

@durack1 I've discovered that there is a CMIP6 variable named prra that is listed for different modeling realms and has different long names.

The site currently lists prra - Rainfall Flux where Ice Free Ocean over Sea. Should we lists all combinations of out_name and long_name? The standard_name, rainfall_flux, is constant between all versions. Should we use standard_name instead of long_name?

taylor13 commented 3 years ago

this is disturbing. If true, that means file names won't be unique. I'll check into this further.

mauzey1 commented 3 years ago

@taylor13 I found one instance of the prra variable in different tables. If you are referring to the data set files, then they should have unique names since they combine both table and variable names.

taylor13 commented 3 years ago

Yes, that's correct, so this won't be a problem; sorry to raise an alarm. prra appears in several different tables with different definitions; not ideal, but doesn't break anything. For the purposes of the publication site, I don't think we need to distinguish between the different variants of prra.

durack1 commented 3 years ago

@taylor13 I agree, the prra - Rainfall Flux is the safest one to use - @mauzey1 can you switch that out? - having said that it's not ideal that this variable is being used across realms, it'd be great if each realm had a unique name

durack1 commented 3 years ago

@mauzey1 the IM number LLNL-WEB-823400 has just landed, so it would be great if we can update the footer and close out this issue (I think we have everything solved once that is updated, no?)