Requirements and dependencies

[backeb]

List of requirements and dependencies

Please update this list as needed (by clicking Edit in the top right of this comment)

Python libaries in the Docker container

• Python tools: MOTU client for CMEMS download (https://github.com/clstoulouse/motu-client-python) cdsapi for ECMWF download (https://github.com/ecmwf/cdsapi) coastserv for pre-processing (https://github.com/openearth/coast-serv-back-end) dfm_tools for Post-processing (https://github.com/openearth/dfm_tools)
• Others: python==3.7 os gc glob datetime numpy shutil netCDF4 scipy.interpolate

Requirements for the provider (need to be provided)

Data requirements

• CMEMS GLOBAL_ANALYSISFORECAST* (~5 days hindcast + ~2 days forecast) [few Megabytes]
• ECMWF ERA5 hindcast (~5 days hindcast) [few Megabytes]
• GFS forecast (~2 days forecast) [few Megabytes]
• FES2012 tidal component data (https://www.aviso.altimetry.fr/es/data/products/auxiliary-products/global-tide-fes/description-fes2012.html) [~4GB but downloaded only once, not operationally]

Cloud requirements

• Linux OS
• SSH Access with root privileges
• Docker Support
• Managed Kubernetes Cluster Support
• Minimum Hardware Requirements for VMs
• 32GB RAM
• min 4 (optimally 12) CPU Cores @ 3.5 GHz
• 250+ GB SSD Working Directory, 5+ TB Long Term Storage
• Auto Scaling for 1 to 10 Nodes

HPC (for model simulations)

• [ ] ...
• [ ]

[lorincmeszaros]

I am checking the requirements. It will be filled soon.

[lorincmeszaros]

Edited wih basic requirements

[backeb]

@kkoumantaros please advise if the above requirements can be met

[yan0s]

Concerning the cloud requirements at GRNET cloud we can support 1) Linux OS 2) SSH Access with root privileges 3) Docker Support 4) Managed Kubernetes Cluster Support 5) Minimum Hardware Requirements for VMs 6) 32GB RAM 7) 5+ TB Long Term Storage

We can only offer MAX CPU frequency at 3.0GHz (Xeon(R) CPU E5-2650 v3) the number of cores can be fulfilled. 8) min 4 (optimally 12) CPU Cores @ 3.5 GHz

and we need to investigate further if the following can be supported: 9) 250+ GB SSD Working Directory 10) Auto Scaling for 1 to 10 Nodes

Would it be possible to run the use case in our cloud without SSDs and Kubernetes autoscaling?

[backeb]

Would it be possible to run the use case in our cloud without SSDs and Kubernetes autoscaling?

@avgils could you comment on this point?

[backeb]

@jkonarski can you comment on the above requirements and flag any potential problems delivering those?

[jkonarski]

@backeb I have checke the requrements and I have only comment on

4. Managed Kubernetes Cluster Support This is partial at the moment, as we are working on Magnum to be ready at first quarter of 2022. Currently we propose ready to use custom software based on terraform scripts that you can find on our website: https://creodias.eu/faq-other/-/asset_publisher/SIs09LQL6Gct/content/how-to-configure-kubernetes

The rest is compliant.

[backeb]

@lorincmeszaros @avgils please specify what the providers need to provide and what will be in the Docker container.

Actions:

1. Set up VM on GRNET
2. Install what we need from the user side
3. Create a snapshot of the VM for future provisioning (contact point: @yan0s)

Need a managed Kubernetes cluster. Providers (GRNET and CloudFerro) have recipes to deploy Kubernetes. CloudFerro working on providing managed Kubernetes cluster - @jkonarski please investigate when / how this can be achieved.

https://creodias.eu/faq-other/-/asset_publisher/SIs09LQL6Gct/content/how-to-configure-kubernetes @avgils can you review and test this doc

[enolfc]

From EGI we also have tooling for setting up k8s automatically. See https://docs.egi.eu/users/cloud-container-compute/k8s/ and also via web dashboard here: https://appsgrycap.i3m.upv.es:31443/im-dashboard/login

[backeb]

@avgils @lorincmeszaros could you please have a look at the documentation referenced above:

1. https://github.com/c-scale-community/use-case-hisea/issues/3#issuecomment-886671042
2. https://github.com/c-scale-community/use-case-hisea/issues/3#issuecomment-886765178

[lorincmeszaros]

@lorincmeszaros @avgils please specify what the providers need to provide and what will be in the Docker container.

I have separated what is in the Docker container and what needs to be provided.

[lorincmeszaros]

From EGI we also have tooling for setting up k8s automatically. See https://docs.egi.eu/users/cloud-container-compute/k8s/ and also via web dashboard here: https://appsgrycap.i3m.upv.es:31443/im-dashboard/login

it seems that we need permission to access the dashboard:

[lorincmeszaros]

@avgils @lorincmeszaros could you please have a look at the documentation referenced above:

1. Requirements and dependencies #3 (comment)
2. Requirements and dependencies #3 (comment)

@avgils I believe it is better if you check whether these documentations are sufficient for us