Temporary storage for data transfer

[chaopengshen]

1. Requester Information: This should include the name and contact information of the person making the request. Chaopeng Shen The Pennsylvania State University Email: cshen@engr.psu.edu Office: 814-863-5844

2. Project Information: Provide a brief description of the project and its goals. This can help the infrastructure team understand the context and purpose of the requested resources. Please highlight how this project will be benefit from and/or provide benefit to other resources on the shared infrastructure. This request is for projects at the Cooperative Institute for Research to Operations in Hydrology (CIROH), founded by NOAA and USGS. The projects' goals are to develop large-scale differentiable models for hydrological modeling. The developed models can be the potential next-generation national water model, and the generated products can also benefit other related research in this field.

3. Project Description: If your project involves developing software or scripts, briefly describe the software you plan to develop. This project aims to develop PyTorch-supported differentiable models

4. Resource Requirements: Specify the compute, storage, and network resources needed for the project. Be as specific as possible about the number of resources required, and any specific configurations or capabilities needed. This information will help the infrastructure team determine the appropriate resources to allocate.

We request at least 1.2 TB temporary storage on S3 with RW permission? This is to enable transfer of data from Penn state system to Univ of Alabama compute system.

Options:

1. Cloud Provider: AWS/Azure/GCP

2. Required Services in the Cloud:

   List of AWS Services

   • S3

5. Timeline: Indicate the expected timeline for the project and when the resources will be needed. This information can help the infrastructure team plan and allocate resources accordingly.

These projects start in 2023, thus we need these resources as soon as possible.

6. Security and Compliance Requirements: If there are any specific security or compliance requirements for the project, these should be clearly stated in the request. This will help ensure that the necessary security measures are in place for the project.

7. Estimation: Include any cost estimation or requirements for the project. This will help the infrastructure team select the most cost-effective solutions for the project.

AWS Cost Calculator: https://calculator.aws/#/

Google Cloud Pricing Calculator: https://cloud.google.com/products/calculator

8. Approval: Indicate the necessary approval processes or sign-offs required for the request.

[arpita0911patel]

"HPC_Data_Transfer" subaccount has been created for this request in AWS account.

[arpitapatel09]

@benlee0423 will let you know when S3 bucket is created and provide access keys.

[benlee0423]

I have sent 3s bucket name and access key through email.

[arpitapatel09]

[like] Arpita Patel reacted to your message:

---

From: Benjamin Lee @.> Sent: Tuesday, February 27, 2024 5:02:45 PM To: CIROH-UA/NGIAB-CloudInfra @.> Cc: Arpita Patel @.>; Comment @.> Subject: [EXTERNAL] Re: [CIROH-UA/NGIAB-CloudInfra] Temporary storage for data transfer (Issue #116)

You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification

I have sent 3s bucket name and access key through email.

— Reply to this email directly, view it on GitHubhttps://github.com/CIROH-UA/NGIAB-CloudInfra/issues/116#issuecomment-1967150407, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2RZVHSKTBETJXE7OCOK42LYVYGTLAVCNFSM6AAAAABD23Q2VWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRXGE2TANBQG4. You are receiving this because you commented.Message ID: @.***>