search

imagepbl / effort-sharing

Analysis and plotting scripts for effort sharing, associated with the Carbon Budget Explorer
MIT License
0 stars 0 forks source link

Performance of datareader method #26

Closed MarkMDekker closed 9 months ago

MarkMDekker commented 9 months ago

This is about determine_global_co2_trajectories(). It's too slow, I'm checking it out today.

MarkMDekker commented 9 months ago

It was much faster before with my old environment. Not sure what changed.

chantalwuer commented 9 months ago

Does it still take 15 mins for you with the new environment?

MarkMDekker commented 9 months ago

The datareader class, yes. But it used to take me 1.5-2 min. However, I'm still working on a pip environment. My conda issue is still not fixed. But nevertheless, there are many for loops in this method than can go :-). I do have a few other priorities for today, though. So will focus on those first.

chantalwuer commented 9 months ago

Okay, I have another call with the factory now to figure out my performance issues. I replicated your old environment but the datareader class took me 30-40 minutes regardless. We suspect it has to do with the Windows defender. Hopefully I can also get down to 15 mins soon..

MarkMDekker commented 9 months ago

It seems that @martijnvandermarel suggests that we all should migrate our environments to either the C or K drive. Ideally also the code / repo clone should be on C or K. Will try this out. @chantalwuer could you, if you tried that out, confirm if this helped your issue? If so, feel free to close this issue here. afbeelding

MarkMDekker commented 9 months ago

Chantal and I both implemented the enviroment + scripts on K, and it indeed was much faster than on X:. So this can be closed.