search

architecture-building-systems / CityEnergyAnalyst

The City Energy Analyst (CEA)
https://www.cityenergyanalyst.com/
MIT License
196 stars 66 forks source link

Sensitivity analysis and supercomputer for cea #374

Closed jimenofonseca closed 7 years ago

jimenofonseca commented 8 years ago

So we are facing the problem of limited computing power with CEA. Simulations just take a lot to run. Especially while doing sensitivity analysis.While we thought will be a problem later on with the optimization of energy systems is a problem now.

I talked to the super computer cluster at FCL which has 100 cores available for us to use. The infrastructure is based on a linux system and uses Docker to create a repository and handle any job. It seems relatively easy to use and install. We basically install everything in a folder in Docker and send any job from there.

From our IT at FCL they say that the ETHZ new supercomputer will be based in linux and soon or later we will need to make our simulations in such systems. This brings structural problems to be made to the CEA and its dependencies.

This brings the next questions:

@daren-thomas could you advise and help us in the process of connecting to these supercomputers?

daren-thomas commented 8 years ago

BTW: The brutus (and it's sister euler) computers are linux based as well!

From simulations I have done for Arno in the past on the brutus cluster, what happens is you divide the simulation task into independent tasks that can be run. You create a runner script - a batch file - that orchestrates running the whole process. Since each node on the cluster can only be used for a specific amount of time, it is a good idea to have one simulation run per node/job and submit new jobs after that.

I would gladly take on this task.

jimenofonseca commented 8 years ago

Great, let me know how we can get started.

On 19 Oct 2016, at 4:33 pm, Daren Thomas notifications@github.com<mailto:notifications@github.com> wrote:

BTW: The brutus (and it's sister euler) computers are linux based as well!

From simulations I have done for Arno in the past on the brutus cluster, what happens is you divide the simulation task into independent tasks that can be run. You create a runner script - a batch file - that orchestrates running the whole process. Since each node on the cluster can only be used for a specific amount of time, it is a good idea to have one simulation run per node/job and submit new jobs after that.

I would gladly take on this task.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/architecture-building-systems/CEAforArcGIS/issues/374#issuecomment-254748944, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AIjrgnwTRtIpKQRwheXlpK7kfPtyVhp2ks5q1dXVgaJpZM4KakVN.

daren-thomas commented 8 years ago

A first step could be to create the pip install mechanism you were talking about in #355 - that would make setting up a linux environment a bit easier. I will look into running cea on euler today and tomorrow and report back to you!

jimenofonseca commented 8 years ago

thanks!

jimenofonseca commented 8 years ago

@daren-thomas i completed the sensitivity analysis and it works good without crashing for both sobol and morris methods. is there a way i could connect to euler and run simulations form here?

daren-thomas commented 8 years ago

check out PR #394 - there is a README.md in the euler subfolder that explains how to run things.