search

eguil / Density_bining

Density bining code
2 stars 5 forks source link

fix 19 - Code tweaks to get running on cm5 historical #22

Closed durack1 closed 10 years ago

durack1 commented 10 years ago

@eguil this code, based off your branch we were working on today successfully runs across the 5 model test case (ACCESS1-0, BNU-ESM, EC-EARTH, IPSL-CM5A-LR and MIROC4h). I'm a little uncertain about the results from MIROC4h for example, the masking may still be an issue with this (and potentially other) models..

Let me know if this looks good and you can merge my changes, make any final tweaks you want and then let me know when you pull your changes back across and get this running..

I've left all the commented code tidyup to you, as it might be insightful as to how I was trying to test this to get it running..

durack1 commented 10 years ago

@eguil add this in and you should be ok: https://github.com/durack1/Density_bining/blob/master/binDensity.py#L453-461

You might also want to compare my master and yours to make sure you're not missing any other bits..

durack1 commented 10 years ago

@eguil the code trims MIROC4h down to 12 months, as more than this will top out the memory..

eguil commented 10 years ago

@durack1 the code was in there allright. It ran ok for timeint= 1,12. I 'll try again for 1,24...

On 2/11/14 00:36, Paul J. Durack wrote:

@eguil https://github.com/eguil the code trims MIROC4h down to 12 months, as more than this will top out the memory..

— Reply to this email directly or view it on GitHub https://github.com/eguil/Density_bining/pull/22#issuecomment-61387898.

Eric Guilyardi IPSL/LOCEAN - Dir. Rech. CNRS Tour 45, 4eme, piece 406 UPMC, case 100 4 place Jussieu, F-75252 Paris Tel: +33 (0)1 44 27 70 76 Prof. Eric Guilyardi NCAS Climate Meteorology Department University of Reading Reading RG6 6BB - UK Tel: +44 (0)118 378 8315

             http://ncas-climate.nerc.ac.uk/~ericg
eguil commented 10 years ago

@durack1 how do I compare codes on github ? I could not easily find how... On 2/11/14 00:35, Paul J. Durack wrote:

@eguil https://github.com/eguil add this in and you should be ok: https://github.com/durack1/Density_bining/blob/master/binDensity.py#L453-461

You might also want to compare my master and yours to make sure you're not missing any other bits..

— Reply to this email directly or view it on GitHub https://github.com/eguil/Density_bining/pull/22#issuecomment-61387869.

Eric Guilyardi IPSL/LOCEAN - Dir. Rech. CNRS Tour 45, 4eme, piece 406 UPMC, case 100 4 place Jussieu, F-75252 Paris Tel: +33 (0)1 44 27 70 76 Prof. Eric Guilyardi NCAS Climate Meteorology Department University of Reading Reading RG6 6BB - UK Tel: +44 (0)118 378 8315

             http://ncas-climate.nerc.ac.uk/~ericg
durack1 commented 10 years ago

@eguil try this https://github.com/durack1/Density_bining/compare/eguil:master...master

You can also do the same with my repo as the base..

durack1 commented 10 years ago

@eguil the reason the 12 months was set for MIROC4h was due to the size of the grid, 24 months will likely reintroduce your memory errors.. Unless of course you leave my code in there and it'll step through the 24 months in two timesteps?

eguil commented 10 years ago

@durack1 the test for 12 months chunks for MIROC was there all the time. If I ask timeint=1,12 it works fine and the results are ok (but for the fact that for year 1, the basins zonal mean are all equal to the global zonal mean - same bug as for the other models). If I ask timeint=1,24, the I get a memory allocation bug. I don't know if it fails in the first chunk or in the second as my stdout is only limited to the last 50 lines...

On 2/11/14 18:05, Paul J. Durack wrote:

@eguil https://github.com/eguil the reason the 12 months was set for MIROC4h was due to the size of the grid, 24 months will likely reintroduce your memory errors.. Unless of course you leave my code in there and it'll step through the 24 months in two timesteps?

— Reply to this email directly or view it on GitHub https://github.com/eguil/Density_bining/pull/22#issuecomment-61414246.

Eric Guilyardi IPSL/LOCEAN - Dir. Rech. CNRS Tour 45, 4eme, piece 406 UPMC, case 100 4 place Jussieu, F-75252 Paris Tel: +33 (0)1 44 27 70 76 Prof. Eric Guilyardi NCAS Climate Meteorology Department University of Reading Reading RG6 6BB - UK Tel: +44 (0)118 378 8315

             http://ncas-climate.nerc.ac.uk/~ericg
eguil commented 10 years ago

@durack1 ok , now I even got a memory crash for timeint=1,12. Keeping an eye, the memory reqs. went up to 97 GB ! (90% of the machine !). I suggest we go ahead without MIROC4h. All the other checks are ok.

On 2/11/14 18:05, Paul J. Durack wrote:

@eguil https://github.com/eguil the reason the 12 months was set for MIROC4h was due to the size of the grid, 24 months will likely reintroduce your memory errors.. Unless of course you leave my code in there and it'll step through the 24 months in two timesteps?

— Reply to this email directly or view it on GitHub https://github.com/eguil/Density_bining/pull/22#issuecomment-61414246.

Eric Guilyardi IPSL/LOCEAN - Dir. Rech. CNRS Tour 45, 4eme, piece 406 UPMC, case 100 4 place Jussieu, F-75252 Paris Tel: +33 (0)1 44 27 70 76 Prof. Eric Guilyardi NCAS Climate Meteorology Department University of Reading Reading RG6 6BB - UK Tel: +44 (0)118 378 8315

             http://ncas-climate.nerc.ac.uk/~ericg