search

ecmwf / ecflow

ECMWF's workflow manager
Apache License 2.0
41 stars 15 forks source link

Min/recommended hardware requirements #15

Closed CptOrange16 closed 2 years ago

CptOrange16 commented 2 years ago

Hello

Do you have any minimum/recommended hardware specs defined?

Asking because i couldn't find any on the wiki.

iainrussell commented 2 years ago

Hello @CptOrange16,

The requirements are that your machine runs either macOS or Linux. As for memory requirements, it really depends on the size of the suites that you will be running - they can be anything from a single node to hundreds of thousands of nodes. In the latter case, you could expect memory usage to be several hundred MB, maybe one or two GB in more extreme cases. This is true for both the server and the UI. The CPU demands for the UI are not heavy, but again for the server, it will depend on how many nodes are in the suites, and also how may clients are connecting to the server. So we don't set minimum requirements because it depends so much on the specifics of your particular setup.

Best regards, Iain

CptOrange16 commented 2 years ago

@iainrussell thank you so much for the reply. Yes it makes perfect sense now, on why you didn't provide a set of minimum requirements.

As for memory requirements, it really depends on the size of the suites that you will be running - they can be anything from a single node to hundreds of thousands of nodes. In the latter case, you could expect memory usage to be several hundred MB, maybe one or two GB in more extreme cases. This is true for both the server and the UI.

The CPU demands for the UI are not heavy, but again for the server, it will depend on how many nodes are in the suites, and also how may clients are connecting to the server. So we don't set minimum requirements because it depends so much on the specifics of your particular setup.

Please correct me if I am wrong, but an ecflow node is each one of the nodes that appear in the tree structure in the ecflow_ui For example, the structure below, this would be 13 nodes

A/
├─ B/
│  ├─ B1/
│  │  ├─ 0000/
│  │  │  ├─ Task1/
│  │  │  ├─ Task2/
│  │  ├─ 0010/
│  │  │  ├─ Task1/
│  │  │  ├─ Task2/
│  ├─ B2/
│  │  ├─ 0000/
│  │  │  ├─ Task1/
│  │  │  ├─ Task2/

If my definition of node is correct, i am planning on 2 systems, one with 1450 nodes and another with 14400 nodes. Would 0.14 MB and 14.4 MB be good estimates?

As for the cpu, there are supposed to be few clients, max 2.

iainrussell commented 2 years ago

Hi @CptOrange16, yes that's right. I really can't comment on the real-world memory usage for your cases, but they do not seem to be heavy, so I'd expect pretty much any hardware to be able to handle it.

CptOrange16 commented 2 years ago

I forgot to close this issue, sorry. Thanks for the help :)