search

chrisroadmap / ar6

Figure and data generation for Chapter 7 of the IPCC's Sixth Assessment Report, Working Group 1 (plus assorted other contributions)
MIT License
34 stars 27 forks source link

Two layer speed #8

Closed znicholls closed 3 years ago

znicholls commented 4 years ago

@chrisroadmap have a look at the updated notebook. On my laptop I can do 5 000 samples for 8 scenarios in about 120 seconds, 10 scenarios in about 3:40 (I only have 8 cores :( ). Hopefully you get something similar. I went up to 20 000 but didn't have space to save the output file...

The way I've set it up, it is only parallel on the scenarios so you can only use as many cores as there are scenarios (so anything more than 10 cores is a waste of time at the moment). It's possible to parallelise on configuration and scenarios if you want to start running on 50 core machines but I figured one step at a time for now.

p.s. Given I'm away next week, adding lewisjared here might not be a bad idea.

chrisroadmap commented 3 years ago

I think I'll close this as we now address this elsewhere (e.g. openscm-twolayermodel).