search

CFMIP / COSPv2.0

COSP - The CFMIP (Cloud Feedbacks Model Intercomparison Project) Observation Simulator Package
42 stars 38 forks source link

Updated CMOR driver for COSP2 #8

Closed dustinswales closed 6 years ago

dustinswales commented 6 years ago

@alejandrobodas @RobertPincus

I'm not sure how important this is, especially that we now have a non-CMOR dependent offline driver, but is there a desire to update the CMOR driver to use the latest version of CMOR (version3)?

Dustin

RobertPincus commented 6 years ago

Dustin, did we not have recent user complaint about trying to build the CMOR driver with a version of CMOR that’s no longer supported and not used by CMIP?

@alejandrobodashttps://github.com/alejandrobodas @RobertPincushttps://github.com/robertpincus

I'm not sure how important this is, especially that we now have a non-CMOR dependent offline driver, but is there a desire to update the CMOR driver to use the latest version of CMOR (version3)?

Dustin

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/CFMIP/COSPv2.0/issues/8, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALRriJ5UzWVOzLPxkXzpLpEJ_JxtGRRCks5td_CLgaJpZM4So_Fm.

dustinswales commented 6 years ago

Robert, Yes we did (https://groups.google.com/forum/#!topic/cosp-user/42VrqM87Y7w)

klein21 commented 6 years ago

Dustin,

If we remove the CMOR capability from the test program, do we lose anything capability/functionality?

Do climate models build their own interfaces to CMOR, such that climate models don’t directly use whatever CMOR interface is in the COSP testing program?

If so, the only people harmed by removing CMOR from the COSP testing program would be those that run COSP offline, but also want to submit the offline calculated values to CMIP/CFMIP. However, I don’t believe there are any such users as all COSP use within CMIP/CFMIP is done inline.

If so, it would be good to remove the CMOR interface, because it is not needed and because it causes trouble with those who want to test COSP but don’t need CMOR. Being decoupled from the CMOR developments is another plus in my mind.

As for the rest of the reorganization of the code, I don’t have a specific comment, other than to confirm with you that the changes do not alter (significantly) the test program’s output.

Steve

From: dustinswales notifications@github.com Reply-To: "CFMIP/COSPv2.0" reply@reply.github.com Date: Tuesday, March 13, 2018 at 10:06 AM To: "CFMIP/COSPv2.0" COSPv2.0@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [CFMIP/COSPv2.0] Updated CMOR driver for COSP2 (#8)

Robert, Yes we did (https://groups.google.com/forum/#!topic/cosp-user/42VrqM87Y7w)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/CFMIP/COSPv2.0/issues/8#issuecomment-372742197, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfO_cOdOAkir4tKm1qmT89TRuZwmhpPfks5td_xkgaJpZM4So_Fm.

RobertPincus commented 6 years ago

As for the rest of the reorganization of the code, I don’t have a specific comment, other than to confirm with you that the changes do not alter (significantly) the test program’s output.

Steve -

The merge proposed by Dustin includes no changes to source code at all beyond adding the new non-CMOR driver. Thus any changes you see should be entirely the result of using different compilers, etc. Does that seem consistent with what you’re seeing?

Alejandro was advocating for CMOR support most vocally. I’ll be interested to hear what he has to say.

klein21 commented 6 years ago

Robert,

I haven’t looked at the output. I agree with you that there shouldn’t be any significant changes because there are no changes to the source code beyond the driver. I am asking Dustin to confirm that this is case.

Yes, I am curious to know what Alejandro thinks. His opinion should carry more weight given that he has much more knowledge about the underlying code and relevant issues than do I.

Steve

From: Robert Pincus notifications@github.com Reply-To: "CFMIP/COSPv2.0" reply@reply.github.com Date: Tuesday, March 13, 2018 at 5:06 PM To: "CFMIP/COSPv2.0" COSPv2.0@noreply.github.com Cc: "Klein, Stephen A." klein21@llnl.gov, Comment comment@noreply.github.com Subject: Re: [CFMIP/COSPv2.0] Updated CMOR driver for COSP2 (#8)

As for the rest of the reorganization of the code, I don’t have a specific comment, other than to confirm with you that the changes do not alter (significantly) the test program’s output.

Steve -

The merge proposed by Dustin includes no changes to source code at all beyond adding the new non-CMOR driver. Thus any changes you see should be entirely the result of using different compilers, etc. Does that seem consistent with what you’re seeing?

Alejandro was advocating for CMOR support most vocally. I’ll be interested to hear what he has to say.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/CFMIP/COSPv2.0/issues/8#issuecomment-372860822, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfO_cLkmOxCPv-ejxzCn0jTg-M39lAr4ks5teF77gaJpZM4So_Fm.

RobertPincus commented 6 years ago

Ah - you’re asking if the answers have changed for us. They have not.

On Mar 13, 2018, at 8:11 PM, klein21 notifications@github.com wrote:

Robert,

I haven’t looked at the output. I agree with you that there shouldn’t be any significant changes because there are no changes to the source code beyond the driver. I am asking Dustin to confirm that this is case.

Yes, I am curious to know what Alejandro thinks. His opinion should carry more weight given that he has much more knowledge about the underlying code and relevant issues than do I.

Steve

klein21 commented 6 years ago

Thanks!

From: Robert Pincus notifications@github.com Reply-To: "CFMIP/COSPv2.0" reply@reply.github.com Date: Tuesday, March 13, 2018 at 5:12 PM To: "CFMIP/COSPv2.0" COSPv2.0@noreply.github.com Cc: "Klein, Stephen A." klein21@llnl.gov, Comment comment@noreply.github.com Subject: Re: [CFMIP/COSPv2.0] Updated CMOR driver for COSP2 (#8)

Ah - you’re asking if the answers have changed for us. They have not.

On Mar 13, 2018, at 8:11 PM, klein21 notifications@github.com wrote:

Robert,

I haven’t looked at the output. I agree with you that there shouldn’t be any significant changes because there are no changes to the source code beyond the driver. I am asking Dustin to confirm that this is case.

Yes, I am curious to know what Alejandro thinks. His opinion should carry more weight given that he has much more knowledge about the underlying code and relevant issues than do I.

Steve

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/CFMIP/COSPv2.0/issues/8#issuecomment-372861996, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfO_cB39qMFG8t6thFpmC59oc5ef5bZQks5teGB6gaJpZM4So_Fm.

alejandrobodas commented 6 years ago

@dustinswales @RobertPincus @klein21 Hi, I am in favour of deleting the CMOR interface. This interface was used for CMIP5, when one or two modelling centres run COSP offline. I don't think this is needed any more, and it makes the installation overly complex due to a number of additional external dependencies. Perhaps we could ask in the cosp-users group, just to be sure than no-one is opposed to this. However, I think the netcdf output should be CF-compliant so that it can still be easily readable by many data analysis software. In principle, this should not be too difficult.

RobertPincus commented 6 years ago

@alejandrobodas I’m all in favor of dropping CMOR support too, for the reasons you mention. I have been finding strict CF compliance a little difficult to achieve with RFMIP files. Given that the main role of this offline driver is to flag code changes that affect answers, how much effort do you think we should devote to this?

@dustinswaleshttps://github.com/dustinswales @RobertPincushttps://github.com/robertpincus @klein21https://github.com/klein21 Hi, I am in favour of deleting the CMOR interface. This interface was used for CMIP5, when one or two modelling centres run COSP offline. I don't think this is needed any more, and it makes the installation overly complex due to a number of additional external dependencies. Perhaps we could ask in the cosp-users group, just to be sure than no-one is opposed to this. However, I think the netcdf output should be CF-compliant so that it can still be easily readable by many data analysis software. In principle, this should not be too difficult.

alejandrobodas commented 6 years ago

@RobertPincus That's why I think doing a quick survey among the users would be good. If the offline use is very small or non-exsitent, then making the files CF-compliant would not be that important. @dustinswales Is there an output file already available in the repository? I could run it through a CF checker and see if the issues are major ones or not.

RobertPincus commented 6 years ago

@alejandrobodasThe example output file is in driver/data/outputs/UKMO/cosp2_output_um.ref.nc .

On Mar 14, 2018, at 9:15 AM, alejandrobodas notifications@github.com wrote:

@RobertPincus That's why I think doing a quick survey among the users would be good. If the offline use is very small or non-exsitent, then making the files CF-compliant would not be that important. @dustinswales Is there an output file already available in the repository? I could run it through a CF checker and see if the issues are major ones or not.

alejandrobodas commented 6 years ago

@RobertPincus thanks for this. I think the file is nearly a minimalist CF-compliant file. I have a few suggestions that I think should be easy to add:

1) add a variable for each of the dimensions. For instance, when I read the MISR histogram, I get MISR CFAD / (%) (altitude_bin_centers: 16; cloud_optical_depth_bin_centers: 7; -- : 153)

but when I read in the 3D calipso cloud fraction I get:

Calipso cloud-fraction / (%) (-- : 40; -- : 153)

The '--' indicates that there is no variable associated with that axis. Hopefully adding these is not too much work.

2) The CF checker complains of the following: latitude variable 'latitude' should define standard_name='latitude' (same for longitude)

Global attribute 'Conventions' field is not present

3) Whenever possible, I would add the standard_name attribute.

4) Some of the long names are not strictly correct, e.g. use of 'CFAD' for ISCCP and MISR histograms.

klein21 commented 6 years ago

Robert or Dustin,

I like Alejandro’s idea for sending a quick message out to the COSP-user maillist, asking if anybody would miss deleting the CMOR interface from the COSP2 testing program. You might learning something relevant.

Steve

From: alejandrobodas notifications@github.com Reply-To: "CFMIP/COSPv2.0" reply@reply.github.com Date: Wednesday, March 14, 2018 at 7:12 AM To: "CFMIP/COSPv2.0" COSPv2.0@noreply.github.com Cc: "Klein, Stephen A." klein21@llnl.gov, Mention mention@noreply.github.com Subject: Re: [CFMIP/COSPv2.0] Updated CMOR driver for COSP2 (#8)

@RobertPincushttps://github.com/robertpincus thanks for this. I think the file is nearly a minimalist CF-compliant file. I have a few suggestions that I think should be easy to add:

  1. add a variable for each of the dimensions. For instance, when I read the MISR histogram, I get MISR CFAD / (%) (altitude_bin_centers: 16; cloud_optical_depth_bin_centers: 7; -- : 153)

but when I read in the 3D calipso cloud fraction I get:

Calipso cloud-fraction / (%) (-- : 40; -- : 153)

The '--' indicates that there is no variable associated with that axis. Hopefully adding these is not too much work.

  1. The CF checker complains of the following: latitude variable 'latitude' should define standard_name='latitude' (same for longitude)

Global attribute 'Conventions' field is not present

  1. Whenever possible, I would add the standard_name attribute.
  2. Some of the long names are not strictly correct, e.g. use of 'CFAD' for ISCCP and MISR histograms.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/CFMIP/COSPv2.0/issues/8#issuecomment-373034535, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfO_cL-f5viQpGxLxNHCfwFaKxvOvVXcks5teSUogaJpZM4So_Fm.

RobertPincus commented 6 years ago

I did this already this morning. You didn’t get it?

I like Alejandro’s idea for sending a quick message out to the COSP-user maillist, asking if anybody would miss deleting the CMOR interface from the COSP2 testing program. You might learning something relevant.

Steve

From: alejandrobodas notifications@github.com Reply-To: "CFMIP/COSPv2.0" reply@reply.github.com Date: Wednesday, March 14, 2018 at 7:12 AM To: "CFMIP/COSPv2.0" COSPv2.0@noreply.github.com Cc: "Klein, Stephen A." klein21@llnl.gov, Mention mention@noreply.github.com Subject: Re: [CFMIP/COSPv2.0] Updated CMOR driver for COSP2 (#8)

@RobertPincushttps://github.com/robertpincus thanks for this. I think the file is nearly a minimalist CF-compliant file. I have a few suggestions that I think should be easy to add:

  1. add a variable for each of the dimensions. For instance, when I read the MISR histogram, I get MISR CFAD / (%) (altitude_bin_centers: 16; cloud_optical_depth_bin_centers: 7; -- : 153)

but when I read in the 3D calipso cloud fraction I get:

Calipso cloud-fraction / (%) (-- : 40; -- : 153)

The '--' indicates that there is no variable associated with that axis. Hopefully adding these is not too much work.

  1. The CF checker complains of the following: latitude variable 'latitude' should define standard_name='latitude' (same for longitude)

Global attribute 'Conventions' field is not present

  1. Whenever possible, I would add the standard_name attribute.
  2. Some of the long names are not strictly correct, e.g. use of 'CFAD' for ISCCP and MISR histograms.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/CFMIP/COSPv2.0/issues/8#issuecomment-373034535, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfO_cL-f5viQpGxLxNHCfwFaKxvOvVXcks5teSUogaJpZM4So_Fm. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

klein21 commented 6 years ago

No I didn’t, but I have noticed recently that delivery into my LLNL inbox of e-mail from Google groups has been slow…. I don’t know what that’s about…

From: Robert Pincus notifications@github.com Reply-To: "CFMIP/COSPv2.0" reply@reply.github.com Date: Wednesday, March 14, 2018 at 9:28 AM To: "CFMIP/COSPv2.0" COSPv2.0@noreply.github.com Cc: "Klein, Stephen A." klein21@llnl.gov, Mention mention@noreply.github.com Subject: Re: [CFMIP/COSPv2.0] Updated CMOR driver for COSP2 (#8)

I did this already this morning. You didn’t get it?

I like Alejandro’s idea for sending a quick message out to the COSP-user maillist, asking if anybody would miss deleting the CMOR interface from the COSP2 testing program. You might learning something relevant.

Steve

From: alejandrobodas notifications@github.com Reply-To: "CFMIP/COSPv2.0" reply@reply.github.com Date: Wednesday, March 14, 2018 at 7:12 AM To: "CFMIP/COSPv2.0" COSPv2.0@noreply.github.com Cc: "Klein, Stephen A." klein21@llnl.gov, Mention mention@noreply.github.com Subject: Re: [CFMIP/COSPv2.0] Updated CMOR driver for COSP2 (#8)

@RobertPincushttps://github.com/robertpincus thanks for this. I think the file is nearly a minimalist CF-compliant file. I have a few suggestions that I think should be easy to add:

  1. add a variable for each of the dimensions. For instance, when I read the MISR histogram, I get MISR CFAD / (%) (altitude_bin_centers: 16; cloud_optical_depth_bin_centers: 7; -- : 153)

but when I read in the 3D calipso cloud fraction I get:

Calipso cloud-fraction / (%) (-- : 40; -- : 153)

The '--' indicates that there is no variable associated with that axis. Hopefully adding these is not too much work.

  1. The CF checker complains of the following: latitude variable 'latitude' should define standard_name='latitude' (same for longitude)

Global attribute 'Conventions' field is not present

  1. Whenever possible, I would add the standard_name attribute.
  2. Some of the long names are not strictly correct, e.g. use of 'CFAD' for ISCCP and MISR histograms.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/CFMIP/COSPv2.0/issues/8#issuecomment-373034535, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfO_cL-f5viQpGxLxNHCfwFaKxvOvVXcks5teSUogaJpZM4So_Fm. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/CFMIP/COSPv2.0/issues/8#issuecomment-373085651, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfO_cDBWM5QTq-wV8yykYFLuDvpcga0Sks5teUT0gaJpZM4So_Fm.

klein21 commented 6 years ago

Oh, I think the problem may be with the e-mail address I associate with Google. Anyways, I see your message Robert to the COSP-user group….

From: "Klein, Stephen A." klein21@llnl.gov Date: Wednesday, March 14, 2018 at 9:30 AM To: "CFMIP/COSPv2.0" reply@reply.github.com, "CFMIP/COSPv2.0" COSPv2.0@noreply.github.com Cc: Mention mention@noreply.github.com Subject: Re: [CFMIP/COSPv2.0] Updated CMOR driver for COSP2 (#8)

No I didn’t, but I have noticed recently that delivery into my LLNL inbox of e-mail from Google groups has been slow…. I don’t know what that’s about…

From: Robert Pincus notifications@github.com Reply-To: "CFMIP/COSPv2.0" reply@reply.github.com Date: Wednesday, March 14, 2018 at 9:28 AM To: "CFMIP/COSPv2.0" COSPv2.0@noreply.github.com Cc: "Klein, Stephen A." klein21@llnl.gov, Mention mention@noreply.github.com Subject: Re: [CFMIP/COSPv2.0] Updated CMOR driver for COSP2 (#8)

I did this already this morning. You didn’t get it?

I like Alejandro’s idea for sending a quick message out to the COSP-user maillist, asking if anybody would miss deleting the CMOR interface from the COSP2 testing program. You might learning something relevant.

Steve

From: alejandrobodas notifications@github.com Reply-To: "CFMIP/COSPv2.0" reply@reply.github.com Date: Wednesday, March 14, 2018 at 7:12 AM To: "CFMIP/COSPv2.0" COSPv2.0@noreply.github.com Cc: "Klein, Stephen A." klein21@llnl.gov, Mention mention@noreply.github.com Subject: Re: [CFMIP/COSPv2.0] Updated CMOR driver for COSP2 (#8)

@RobertPincushttps://github.com/robertpincus thanks for this. I think the file is nearly a minimalist CF-compliant file. I have a few suggestions that I think should be easy to add:

  1. add a variable for each of the dimensions. For instance, when I read the MISR histogram, I get MISR CFAD / (%) (altitude_bin_centers: 16; cloud_optical_depth_bin_centers: 7; -- : 153)

but when I read in the 3D calipso cloud fraction I get:

Calipso cloud-fraction / (%) (-- : 40; -- : 153)

The '--' indicates that there is no variable associated with that axis. Hopefully adding these is not too much work.

  1. The CF checker complains of the following: latitude variable 'latitude' should define standard_name='latitude' (same for longitude)

Global attribute 'Conventions' field is not present

  1. Whenever possible, I would add the standard_name attribute.
  2. Some of the long names are not strictly correct, e.g. use of 'CFAD' for ISCCP and MISR histograms.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/CFMIP/COSPv2.0/issues/8#issuecomment-373034535, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfO_cL-f5viQpGxLxNHCfwFaKxvOvVXcks5teSUogaJpZM4So_Fm. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/CFMIP/COSPv2.0/issues/8#issuecomment-373085651, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfO_cDBWM5QTq-wV8yykYFLuDvpcga0Sks5teUT0gaJpZM4So_Fm.

dustinswales commented 6 years ago

CMOR capability has been removed from COSP2 offline driver.