search

geoschem / geos-chem-cloud

Run GEOS-Chem easily on AWS cloud
http://cloud.geos-chem.org
MIT License
39 stars 9 forks source link

[QUESTION] Launch instances via Command Line Interface on Windows #26

Closed FeiYao-Edinburgh closed 1 year ago

FeiYao-Edinburgh commented 4 years ago

I thought this issue previously but just realised that it might be worthy of an effort. From this link we know it is quite convenient to launch an EC2 instance using aws ec2 run-instances. However, the solution the link provides only targets Linux/Mac users. For Windows users, scripts modifications are needed. Using the GitBash recommended by the Quick Start Guide, we won't have aws software by default. Installing aws on Windows can be achieved with Anoconda or Miniconda. And a line of code, e.g. export PATH=.../miniconda3/Scripts:$PATH, needs to be added to tell the GitBash where to find the aws. Besides this, I also found that --block-device-mapping DeviceName="/dev/sda1",Ebs={VolumeSize=$EBS_SIZE} does not work properly if launching from Windows but don't know how to resolve it.

yantosca commented 4 years ago

Thanks for bringing this up. We use a shared AWS account here so it is problematic for us to use CLI due to security issues. So I haven't been able to test this.

JiaweiZhuang commented 4 years ago

Installing aws on Windows can be achieved with Anoconda or Miniconda. And a line of code, e.g. export PATH=.../miniconda3/Scripts:$PATH, needs to be added to tell the GitBash where to find the aws.

Seems like Windows starts to have more native Linux support: https://devblogs.microsoft.com/commandline/announcing-wsl-2/. Would this work for you? In general using Python packages on Windows is annoying...

I also found that --block-device-mapping DeviceName="/dev/sda1",Ebs={VolumeSize=$EBS_SIZE} does not work properly if launching from Windows but don't know how to resolve it.

What error did you get? You might need to change /dev/sda1 to something else due to EC2/EBS updates. Try /dev/nvme0n1p1 (shown by df)

We use a shared AWS account here so it is problematic for us to use CLI due to security issues.

You can still use CLI by making a new IAM user. See "Step 1: Create an AWS IAM User" at https://aws.amazon.com/getting-started/tutorials/backup-to-s3-cli/

FeiYao-Edinburgh commented 4 years ago

What error did you get? You might need to change /dev/sda1 to something else due to EC2/EBS updates. Try /dev/nvme0n1p1 (shown by df)

Thanks for your reply. I tried again but still got the following similar errors. I personally think it's a problem of different filesystems. I bring this because there might be some potential Windows GitBash users who want to launch an EC2 instance conveniently with flexible EBS. It is totally fine without specifying the EBS, in which case we can only get a default EBS volume.

An error occurred (InvalidBlockDeviceMapping) when calling the RunInstances operation: Invalid device name C:/Users/s1855106/AppData/Local/Programs/Git/dev/sda1
An error occurred (InvalidBlockDeviceMapping) when calling the RunInstances operation: Invalid device name C:/Users/s1855106/AppData/Local/Programs/Git/dev/nvme0n1p1

Frankly, this issue doesn't bother me too much for the moment. I bring this up supposing maybe you have some interest.

Cheers, Fei

JiaweiZhuang commented 4 years ago

I tried again but still got the following similar errors.

Does /dev/xvda work?

FeiYao-Edinburgh commented 4 years ago

Does /dev/xvda work?

Unfortunately No. Similar error, An error occurred (InvalidBlockDeviceMapping) when calling the RunInstances operation: Invalid device name C:/Users/s1855106/AppData/Local/Programs/Git/dev/xvda.

JiaweiZhuang commented 4 years ago

Hmm why does your device name have a C:/Users/s1855106/AppData/Local/Programs/Git/ prefix... Would you be able to remove it?

FeiYao-Edinburgh commented 4 years ago

That was the information output from GitBash after running aws ec2 run-instances. The device name within my script remains as --block-Connection reset by 129.215.8.197 port 22VolumeSize=$EBS_SIZE}. I guess this issue is mainly due to Windows machine do not have a proper device for AWS Linux machines to refer to?

yantosca commented 1 year ago

Closing this issue.

ennui93NN commented 1 year ago

For posterity, if you are in this situation: try prepending MSYS_NO_PATHCONV=1 to the beginning of your command-line. See Git for Windows: Known Issues.

kirang270 commented 1 year ago

@yantosca @JiaweiZhuang @FeiYao-Edinburgh I am getting the same error that @FeiYao-Edinburgh got , I am pasting the error statement here that I got while running the 'instance_create.sh' script on my git bash .kindly help me out if you already know the solution for this . "An error occurred (InvalidBlockDeviceMapping) when calling the RunInstances operation: Invalid device name C:/Program Files/Git/dev/sda1 "