CIROH-UA / NGIAB-CloudInfra

NextGen In A Box: NextGen Generation Water Modeling Framework for Community Release (Docker version)
https://docs.ciroh.org/docs/products/Community%20Hydrologic%20Modeling%20Framework/nextgeninaboxDocker/
12 stars 19 forks source link

Amazon EC2 request - Stevens Institute of Technology #31

Closed blackhawk70 closed 8 months ago

blackhawk70 commented 1 year ago

1. Requester Information: Dr. Marouane Temimi Associate Professor Department of Civil, Environmental, and Ocean Engineering (CEOE) Tel:+12012165303 Mail to: mtemimi@stevens.edu

2. Project Information: The project aims to introduce an innovative, automated deep learning-based approach for near real-time satellite monitoring of river ice conditions within the northern watersheds of the United States and Canada. This technique harnesses high-resolution imagery from the VIIRS bands onboard the NOAA-20 and NPP satellites and utilizes the U-Net deep learning algorithm for the semantic segmentation of images, even under challenging conditions such as varying cloud cover and land surface variations.

3. Project Description: The project involves the development of software and scripts in multiple programming languages, including Bash, MATLAB, Python, and the Google Earth Engine.

4. Resource Requirements: CPUs: 30 Memory: 64 Gb Network Bandwidth: at least 1Gbps EBS bandwidth: at least 1Gbps Disk size: at least 300Gb OS: Linux (Ubuntu) Root access required

Options:

  1. Cloud Provider: AWS

  2. Required Services in the Cloud:

    List of AWS Services

    • EC2
    • Amazon CloudWatch or Amazon Managed Grafana

    List of Azure Services

    • Virtual Machines
    • Azure File Storage
    • Azure Machine Learning

5. Timeline: The project requires resources as soon as possible because the system is operational for Alaska, and it is currently winter, which is a critical period for river ice monitoring. The project is expected to run for a minimum of one year.

6. Security and Compliance Requirements: No security and compliance requirements

7. Estimation: Monthly: 624.15/Month for 1 year or 475.96/Month for 3 years

8. Approval: Contact Dr. Marouane Temimi (contact info above)

benlee0423 commented 1 year ago

@blackhawk70

Thank you for submitting the issue. For CPUs requirement, is this referring to vCPUs? https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/cpu-options-supported-instances-values.html

benlee0423 commented 1 year ago

The estimate is based on the 1 year or 3 year savings plan, and it needs to add 49.49 USD/monthly for EBS volume cost of 300GB as well.

blackhawk70 commented 1 year ago

@benlee0423 for the CPUs, correct I meant the vCPUs

arpita0911patel commented 1 year ago

Dear Dr. Marouane, I'm wondering if you would be open to considering on-premises VMware for this project?

jameshalgren commented 1 year ago

We have recommended the following: 1) that we wait 2 weeks to see if the GCP resource can become available. 2) in the meantime, we create the AWS sub-account in case that option is the only one available so that the experiment can start in time for the Northeast freeze season. 3) we schedule a meeting (in January, perhaps) to discuss how this can be integrated with other routing/FIM and hydroinformatics research on the CIROH Research to Operations Hybrid Cloud.

arpita0911patel commented 1 year ago

Created Stevens subaccount in AWS CIROH Account and provided Dr. Marouane Temimi mtemimi@stevens.edu admin access to that account.

MAbdelkader94 commented 1 year ago

Updated request:

1. Requester Information: Dr. Marouane Temimi Associate Professor Department of Civil, Environmental, and Ocean Engineering (CEOE) Tel:+12012165303 Mail to: mtemimi@stevens.edu

2. Project Information: Our project, titled "Advancing Research in Cold Regions Hydrology to Support the Modeling and Mapping of Ice-Induced Flood Inundation," focuses on developing an enhanced system for river and lake ice mapping and monitoring across the U.S. It employs an automated deep learning-based approach using U-Net algorithms for semantic segmentation of high-resolution VIIRS imagery from NOAA-20 satellite. This initiative overcomes monitoring challenges posed by varying cloud cover and land surface variations in northern watersheds. Aligned with NOAA's mission, the outcomes of this project will be shared publicly through a Google Earth Engine-based system and integrated into NOAA NWC and NWS RFCs reports, enhancing streamflow forecasts and flood inundation mapping. Additionally, the project serves as a model for CIROH members and partners to advance the NextGen Modeling Framework in cold regions, integrating critical ice information from satellite imagery, and thus contributes significantly to the shared infrastructure.

3. Project Description: The project involves the development of software and scripts in multiple programming languages, including Bash, MATLAB, and Python. The river ice maps generated using the automated system are pushed to Google Bucket and then to an image collection on Google Earth Engine Platform.

4. Resource Requirements: CPUs: 30 Memory: 64 Gb Network Bandwidth: at least 1Gbps EBS bandwidth: at least 1Gbps Disk size: at least 300Gb OS: Linux (Ubuntu) Root access required

Options:

  1. Cloud Provider: AWS or GCP

  2. Required Services in the Cloud:

List of AWS Services

List of GCP Services

5. Timeline: The project is shovel ready. The system is ready for automated deployment as soon as the resources become available. The project is expected to run for a minimum of one year.

6. Security and Compliance Requirements: No security and compliance requirements

7. Estimation: Monthly: 624.15/Month for 1 year or 475.96/Month for 3 years

8. Approval: Contact Dr. Marouane Temimi (contact info above)

arpita0911patel commented 1 year ago

Based on our discussion, the following actions have been taken:

A Stevens subaccount has been established, and users have been added to that account. Please ensure that you manage instance starts and stops to control costs. Additionally, let's work together to ensure we do not exceed a monthly cost of $500.

We will provide you with a Google account for Google-related services in the near future once we have CIROH google account established.

Thank you

arpita0911patel commented 8 months ago

Closing this request as access is provided to Stevens subaccount for this project.