geoschem / geos-chem-cloud

Run GEOS-Chem easily on AWS cloud
http://cloud.geos-chem.org
MIT License
39 stars 9 forks source link

Use the same AMI for GC-classic and GCHP? #18

Closed JiaweiZhuang closed 5 years ago

JiaweiZhuang commented 5 years ago

Thanks to the fixes in geoschem/gchp#6, now GCHP can run correctly in the same software environment as GC-classic (Ubuntu 18.04, gcc 7.3.0, netCDF 4.6.0. The only addition is OpenMPI3). I see no reason of maintaining two separate AMIs, where the software libraries largely duplicate. This also avoids duplicating ~100 GB of minimum input data.

Users can test GCHP immediately after playing with GC-classic on the cloud. Seems a great education opportunity for users.

My only concern is that GCHP might lag behind GC-classic's version. For example, when the new version of MAPL is implemented, it is not clear how much work it would take to make it run properly on the cloud.

If there're no objections I will use the same AMI for both. @yantosca @lizziel @msulprizio

yantosca commented 5 years ago

I think it’s ok to use the same AMI for both GCC and GCHP for now.

KThe new MAPL might not be ready for a while. We might have to build a new AMI once the new MAPL is ready, due to other software requirements.

If we can get people using GCC and GCHP in the cloud ASAP, then that’s huge!

Users can start with an AMI and git pull newer updates (e.g patches) as the model evolves.

Bob Y.

Get Outlook for iOShttps://aka.ms/o0ukef


From: Jiawei Zhuang notifications@github.com Sent: Friday, December 14, 2018 6:05 PM To: geoschem/geos-chem-cloud Cc: Yantosca, Robert M.; Mention Subject: [geoschem/geos-chem-cloud] Use the same AMI for GC-classic and GCHP? (#18)

Thanks to the fixes in geoschem/gchp#6https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_geoschem_gchp_issues_6&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=r4L-GKpPwNdkKJVsvHMxhwfWSKyk_G4CfjPjWOHcJEo&e=, now GCHP can run correctly in the same software environment as GC-classic (Ubuntu 18.04, gcc 7.3.0, netCDF 4.6.0. The only addition is OpenMPI3). I see no reason of maintaining two separate AMIs, where there the software libraries largely duplicate. This also avoids duplicating ~100 GB of minimum input data.

Users can test GCHP immediately after playing with GC-classic on the cloud. Seems a great education opportunity for users.

My only concern is that GCHP might lag behind GC-classic's version. For example, when the new version of MAPL is implemented, it is not clear how much work it would take to make it run properly on the cloud.

If there're no objections I will use the same AMI for both. @yantoscahttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_yantosca&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=fDSAR0q31Z-LriikkV5KgtXLA3bPBd7o-_8Q0q0ufHU&e= @lizzielhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lizziel&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=F5-LU6TVUOmDy-_ZbEpw2v1ImtUdnzqoFBgWtIRfq5U&e= @msulpriziohttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_msulprizio&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=vqctgX_sOp-435FdE0D8JNlg27RLRLNkcqtWLZ-qGhQ&e=

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_geoschem_geos-2Dchem-2Dcloud_issues_18&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=2ZVY4rNvTDMCpXI7lLWaTkfPdZ6onSYXNXUwDvGxgc8&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AFm56Boed4sbchOUSh0fzNHFW46xkS4Vks5u5C7OgaJpZM4ZUZtp&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=UZuVQFCnu15jYTGssSl3z3MMETCt5m-y6RytUPwqDvk&e=.

lizziel commented 5 years ago

Using the same AMI for both GC classic and GCHP would be ideal. We may need to update to gcc 8.2 for the new MAPL, but we can deal with that when it happens.


From: Bob Yantosca notifications@github.com Sent: Friday, December 14, 2018 6:52:03 PM To: geoschem/geos-chem-cloud Cc: Lundgren, Elizabeth W; Mention Subject: Re: [geoschem/geos-chem-cloud] Use the same AMI for GC-classic and GCHP? (#18)

I think it’s ok to use the same AMI for both GCC and GCHP for now.

KThe new MAPL might not be ready for a while. We might have to build a new AMI once the new MAPL is ready, due to other software requirements.

If we can get people using GCC and GCHP in the cloud ASAP, then that’s huge!

Users can start with an AMI and git pull newer updates (e.g patches) as the model evolves.

Bob Y.

Get Outlook for iOShttps://aka.ms/o0ukef


From: Jiawei Zhuang notifications@github.com Sent: Friday, December 14, 2018 6:05 PM To: geoschem/geos-chem-cloud Cc: Yantosca, Robert M.; Mention Subject: [geoschem/geos-chem-cloud] Use the same AMI for GC-classic and GCHP? (#18)

Thanks to the fixes in geoschem/gchp#6https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_geoschem_gchp_issues_6&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=r4L-GKpPwNdkKJVsvHMxhwfWSKyk_G4CfjPjWOHcJEo&e=, now GCHP can run correctly in the same software environment as GC-classic (Ubuntu 18.04, gcc 7.3.0, netCDF 4.6.0. The only addition is OpenMPI3). I see no reason of maintaining two separate AMIs, where there the software libraries largely duplicate. This also avoids duplicating ~100 GB of minimum input data.

Users can test GCHP immediately after playing with GC-classic on the cloud. Seems a great education opportunity for users.

My only concern is that GCHP might lag behind GC-classic's version. For example, when the new version of MAPL is implemented, it is not clear how much work it would take to make it run properly on the cloud.

If there're no objections I will use the same AMI for both. @yantoscahttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_yantosca&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=fDSAR0q31Z-LriikkV5KgtXLA3bPBd7o-_8Q0q0ufHU&e= @lizzielhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lizziel&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=F5-LU6TVUOmDy-_ZbEpw2v1ImtUdnzqoFBgWtIRfq5U&e= @msulpriziohttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_msulprizio&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=vqctgX_sOp-435FdE0D8JNlg27RLRLNkcqtWLZ-qGhQ&e=

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_geoschem_geos-2Dchem-2Dcloud_issues_18&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=2ZVY4rNvTDMCpXI7lLWaTkfPdZ6onSYXNXUwDvGxgc8&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AFm56Boed4sbchOUSh0fzNHFW46xkS4Vks5u5C7OgaJpZM4ZUZtp&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=XTJBopAzwSsAgEljIk6g1TdJEZxb_bVLEjVNpBxfBIw&m=jXOOdp5GNdqUQNRM4eN9nhJ2GBeh5xXFrMjNlj6Msh4&s=UZuVQFCnu15jYTGssSl3z3MMETCt5m-y6RytUPwqDvk&e=.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_geoschem_geos-2Dchem-2Dcloud_issues_18-23issuecomment-2D447512803&d=DwMFaQ&c=WO-RGvefibhHBZq3fL85hQ&r=xyVOGV-4mAPz62S8RZON4khwZesGKcGg2_BHL4y5NjQ&m=ojC--PZp3rAzVIwE4XicXlye4IydtRrdGrni7vc3uvg&s=h3mkwI6HDu1FDyQeO9k8u5xVXPr7cT8xTHplwr-nazo&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAnyq1bBxF8uoRBW-5Fg6422E0PmNKCGR2ks5u5DmjgaJpZM4ZUZtp&d=DwMFaQ&c=WO-RGvefibhHBZq3fL85hQ&r=xyVOGV-4mAPz62S8RZON4khwZesGKcGg2_BHL4y5NjQ&m=ojC--PZp3rAzVIwE4XicXlye4IydtRrdGrni7vc3uvg&s=H8zsyCgUuvEzqxsRP_h5sWjLX5-vPx7G97pubu6X4qE&e=.

JiaweiZhuang commented 5 years ago

Given that GCHP+gfortran is kind of buggy currently (#geoschem/gchp#15), maybe take GCHP down after the bug is fixed / after we actually use gfortran for benchmark? Or, I can just put it here purely for educational purpose but not scientific analysis.