【GEOSChem-on-cloud user testing and survey】(Phase one)

JiaweiZhuang commented 5 years ago

Dear GEOS-Chem users,

We are excited to invite you to attend the first user testing and survey for the GEOSChem-on-cloud project! Our major purposes are:

Collecting user statistics for our paper aiming at BAMS. The abstract has already been accepted.
Identifying problems with the current documentation, and make it better.

This testing should take you at most an hour and cost less than $0.1.

Why would you bother attending if you already have GEOS-Chem running smoothly on your local computer? Here are some good reasons:

You will be able to get immediate access to the latest version of the model and the full input dataset. (See what's new in the recently-released 12.0.0/12.0.1 in the Newsletter and version history)
You can learn many widely-applicable skills to solve exciting problems in Earth science.
Even with powerful local computers, the cloud is still very useful for quick testing or meeting peak computing needs.
We don't want to lock you in the cloud -- you can easily move the software environment between cloud and local computers using containers.

How to attend the testing & survey:

Part 1. Sign-up for an GitHub account

The survey will be done transparently on GitHub, so the first thing is signing up for an GitHub account if you don't have one already. GitHub is also the currently recommended way to submit your code updates, so it is nice to have an account, anyway.

Please add minimal information to your GitHub profile, such as a picture, one-sentence personal introduction, or a link to personal website / Google scholar site. So we can know who you are and how to contact you. Also consider giving a star to the repositories under the GEOS-Chem team. Just click on the "star" button on the upper-right corner of each repository page. This "shows appreciation to the repository maintainer for their work".

GEOS-Chem has a reasonably large amount of users, but it is sad that its GitHub repo receives so little attention because most scientists don't use GitHub. Here is actually a much better place for discussion compared to private emails, because all discussions are public and can be easily found by anyone with similar problems.

Part 2. Fill-in basic form

You will reply to this survey by simply posting GitHub comments within this GitHub issue (i.e. the current page you are looking at). See the "comment" cell at the bottom of this page, with a big, green "comment" button.

The first part is general info:

Background: University, department, which role.
GEOS-Chem experience: X years

The second part is just a place holder right now. It will be the time you spend on a minimal GEOS-Chem demo on the cloud, as explained in the next section:

Step 1: X minutes
Step 2: X minutes
Step 3: X minutes
Step 4: X minutes
General feedback: For example, is any step confusing?

Part 3. Go through the "Quick start guide" and record the time

We have a very comprehensive documentation, but the most important part for a new user is the Quick start guide. That guide has 5 major steps. We sincerely ask you to go through them and record the time you spend on each step. The timing starts when you begin to follow the instruction for a particular step, and ends according the following rules. Please read through the rules to get a general idea before actually starting the timing.

Step 1. Sign up an AWS account: This step is considered finished when you can log into AWS console with your account. Note that a credit card is needed, although the cost of this short demo is negligible (~$0.1). If you don't have a credit card but still wants to try the cloud, please contact us individually. For this test, I think it is very useful to see how long it takes to get the account. (If you have already signed-up for an account, simply put a rough number of how long that took last time. Additional operations such as subscribing to the educational credit is not included in the timing.)
Step 2: Launching a server with GEOS-Chem pre-installed: This step is considered finished when you can see the running server (also called "EC2 instance") with a green "running" icon in your AWS console.
Step 3: Log into the server and run GEOS-Chem: This step is considered finished when you can execute the simulation by the command ./geos.mp. No need to wait for the simulation to finish. (For Windows users, do not include the time on installing terminal software like Git-BASH).
Step 4: Analyze output data with Python and Jupyter: This step is considered finished when you can see the Jupyter notebook interface in your browser. You will find this step easier if you take a look at our interactive Python tutorial first. However, you don't have to know Python in order to follow this step. You can simply copy and paste the commands shown in the guide. I just want to see if people can successfully connect to the Jupyter notebook program on the cloud. Learning Python can be a separate topic.
Step 5: Shut down the server: This step is trivial and there is no need to report the time on it. But do remember to shut down the server, otherwise you may be charged by much more than $0.1.

Final comments

This thread (i.e. GitHub issue) is only for replying to the survey. All replies/comments should stick to the format specified above. If a comment doesn't conform to the format, I will edit it (without changing the actual content). To discuss the strategy of the survey, please use #14.
Please be honest about the timing. This survey is not to test you, but to test us... If you spend much longer than others, it is probably because our documentation or AWS's UI is confusing. We could have made the survey private (e.g. using Google Forms), but I really want to make all "raw data" transparently visible. The problem with posting results here is that people might be affected by other's results. So, please do post the actual time you spend.
If you want to dive deeper after going through the quick start guide, please see all the Beginner tutorials and then go through a much more complete workflow.

Thanks very much for your time and help! Hope you find this new cloud computing capability useful!

JiaweiZhuang commented 5 years ago

Here's a sample reply:

Background: XX University, XX department, Graduate Student GEOS-Chem experience: X years Step 1: X minutes Step 2: X minutes Step 3: X minutes Step 4: X minutes General feedback: None

lizziel commented 5 years ago

Background: Harvard University, GEOS-Chem Support Team, Scientific Programmer GEOS-Chem experience: 4 years Step 1: 6 minutes Step 2: 6 minutes Step 3: 5 minutes (not including wait time for GEOS-Chem simulation to run) Step 4: 9 minutes General feedback:

I created a new account different from my existing account for this test. I found that there was latency in Amazon verification of my account that caused me to have to wait to launch my first instance. I recommend doing account set up and then coming back a bit later to complete steps 2+.
In Step 4 I recommend telling users how to exit python and how to log out of AWS from the command line. Putting this at the point where users have ipython open but are then directed to re-log in to use Jupiter Notebook would be a good place for this.
It is important to note that when you log back in to use Jupyter Notebook you must source activate geo prior to running the Jupiter command. This is missing from the tutorial.
When I went to the Jupiter page on my browser I was not able to navigate to parent directories. This means users should always launch it from their home directory in the instance to ensure access to all files in all directories. A better way, however, is to append --notebook-dir ~ to the Jupyter command such that the home directory is the default. I recommend adding this to the tutorial.
I think the tutorial should up front tell users about stopping and terminating instances to avoid unexpected charges. Move this to end of Step 2 but still repeat in step 5..
Finally, the time it took me to complete this would probably be a lower bound for new users since I have done this before.

ayshaw commented 5 years ago

Background: Harvard University, Environmental Sciences and Engineering, Graduate Student, GEOS-Chem experience: 1 year Step 1: 10 minutes Step 2: 1.5 minutes Step 3: 3 minutes Step 4: 13 minutes General Feedback

I had created an AWS account in my CS205 class. I had used Jiawei's tutorial instead of the class tutorial then. It took around 10 minutes to set up the account.
This step was well documented and easy to follow.
same as above
I think the tutorial was straightforward but I took longer because I wasn't used to using port forwarding to use jupyter notebook. I use XQuartz and ssh -X (which is also a type of port forwarding) to access jupyter notebook in the remote server I usually use.

msulprizio commented 5 years ago

Background: Harvard University, GEOS-Chem Support Team, Senior Scientific Programmer GEOS-Chem experience: 7 years

Step 1: 5 minutes Step 2: 6 minutes, wait 1 hour for account verification, <1 min to finish Step 3: 7 minutes Step 4: 34 minutes

General feedback:

In Step 2, the text 'In the EC2 console, click on “AMI” (Amazon Machine Image)...' should say "AMIs"
In Step 2, also put red circle around "Launch Instances" in creating key pair figure
After clicking "Launch Instances" the first time, I got "Launch status: Launch Failed. Your account is currently being verified. Verification normally takes less than 2 hours. Until your account is verified, you may not be able to launch additional instances or create additional volumes. If you are still receiving this message after more than 2 hours, please let us know by writing to aws-verification@amazon.com. We appreciate your patience."
It might be worth asking participants what OS they use.
I used a PC and MobaXterm, so step 4 took me a while to figure out the port forwarding step. I repeated Step 4 again using Git-BASH as suggested and it took 4 minutes.

yantosca commented 5 years ago

Harvard University, GEOS-Chem Support Team, Senior Software Engineer GEOS-Chem experience: 20+ years Step 1: 5 minutes Step 2: 5 minutes Step 3: 5 minutes Step 4: 16 minutes General feedback:

A couple of times I had to re-login because the AWS instance display gave the hostname as root@... but you needed to login as ubuntu@... That is annoying but I'm not sure what you can do about it.
I installed Git bash (install time not included above). But in step 4 I often had trouble cutting & pasting into Git bash. Perhaps that is because I am more familiar with e.g. MobaXterm, but it was an annoyance.
At this command: "Visit http://localhost:8899" etc. That opens Jupyter notebook in general, but I got hung up trying to get the jupyter notebook to change folders. If you follow the "Reactivate the python environment and run jupyter notebook..., that will open in the geosfp_4x5_standard folder. You might want to tell people to change to the ~/tutorial/python_example folder before having the open the notebook. Then it should be easier to find the notebook file.

Otherwise I found everything pretty straightforward. Great job!

febfly commented 5 years ago

Harvard University, SEAS, Postdoc GEOS-Chem experience: 5 years Step 1: 4 minutes Step 2: 5 minutes, wait ~ half hour for verification Step 3: 7 minutes Step 4: 8 minutes, a little bit confusion in activate python environment

General feedback: Quite straightforward following the instructions. Thanks!

yanglibj commented 5 years ago

Harvard University, SEAS, Postdoc GEOS-Chem experience: 9 months Step 1: 10 minutes Step 2: 7 minutes Step 3: 3 minutes Step 4: 2 minutes

General feedback: Very user friendly. The tutorial is very clear and easy to follow. Thanks!

mkelp commented 5 years ago

Harvard University, EPS, Grad Student GEOS-Chem experience: Ran for undergrad thesis Step 1: 5 minutes Step 2: 6 minutes Step 3: 4 minutes Step 4: 6 minutes

General feedback: Easy to follow this guide and felt like I will use again in the future.

viral211 commented 5 years ago

Harvard University, SEAS Postdoc GEOS-Chem experience: 6 years Step 1: 10 minutes Step 2: 30 minutes (had to wait for account verification) Step 3: 5 minutes Step 4: 15 minutes Genreal comments: The quick start guide is well organized and straightforward. After being able to actually run the model and see the output in about an hour, I feel much more confident about using GEOSChem-on-cloud.

ngc2244 commented 5 years ago

Background: Harvard University, Postdoc GEOS-Chem experience: 7 years Step 1: 7 minutes Step 2: 5 minutes (takes ~ 1hour for account verification) Step 3: 3 minutes Step 4: 8 minutes General feedback:

I did my test on a Ubuntu-PC.
The Quick start guide is very helpful and straightforward. Good job and thanks!

tianjialiu commented 5 years ago

Harvard University, EPS Grad Student GEOS-Chem experience: None Step 1: 5 minutes Step 2: 6 minutes Step 3: 4 minutes Step 4: 6 minutes

General comments: Easy to follow the guide! Very clear instructions and good intro to GEOS-Chem for non-users. Specific comments:

Link the "Create an AWS account" button image to "https://portal.aws.amazon.com/billing/signup#/start" rather than the image itself; the "http://aws.amazon.com" website says "Sign up" rather than "Create an AWS account" for the button
More detailed explanation of "preferably $HOME/.ssh" i.e. what is the home folder generally? users will probably cd to the users/xxx/Downloads folder on a mac rather than the home folder; explain how users can move the .pem key with the mv command (step 3)
Maybe write out full command (ssh -i "xx.pem" ubuntu@xxx.com and chmod 400 xx.pem), replacing the xx's and adding it as a code block so users can easily copy the commands for this specific tutorial (step 3)
Explicitly show how to change "root" to "ubuntu" by showing how to edit command, i.e. ssh -i "xx.pem" ubuntu@xxx.com instead of ssh -i "xx.pem" root@xxx.com (step 3)

zsx-GitHub commented 5 years ago

Background: Harvard University, visiting graduate student GEOS-Chem experience: 0.5 years

Step 1: 5 minutes Step 2: 5 minutes (account verification takes 15 min) Step 3: 3 minutes (takes 15 min to install Git-BASH, because I use windows 10 system.) Step 4: 3 minutes

General feedback: It will be faster for me to use GEOS-Chem on AWS if I have Git-BASH already installed or if I use Mac. Thanks! Running GEOS-Chem will be much more convenient!

rsilvern commented 5 years ago

Background: Harvard University, Graduate Student GEOS-Chem experience: 4 years

Step 1: 8 minutes Step 2: 9 minutes Step 3: 7 minutes Step 4: 6 minutes

General feedback:

Overall excellent tutorial!
Even though I had an AWS account, it had been dormant for a while (> 1 year) so I had to re-activate it.
At end of Step 2 (Launch Instance), had to wait for verification but it quickly worked after hitting "Retry"
Beginning of Step 3 is slightly confusing, maybe show your example above the "Connect to your Instance" screenshot and/or explicitly state that your specific xxx.com is given in your window.

pengfeiliu371 commented 5 years ago

Background: Harvard University, SEAS, Postdoc GEOS-Chem experience: 1 year Step 1: 8 minutes Step 2: 5 minutes (not including 10 mins waiting time for AWS verification) Step 3: 10 minutes Step 4: 7 minutes General feedback:

The verification of AWS takes some time
The jupyter command could be save to a file to save time to launch

tscarpelli commented 5 years ago

Background: Harvard University, Graduate Student GEOS-Chem experience: 0 years

Step 1: 8 minutes Step 2: 13 minutes Step 3: 14 minutes Step 4: 20 minutes (lots of time wasting involved though)

General feedback: AWS account took some time to set up. I agree with above user: 'Overall excellent tutorial! At end of Step 2 (Launch Instance), had to wait for verification but it quickly worked after hitting "Retry"'

Was slightly confused by order of directions for beginning of step 3.
I'm also not super familiar with using terminal so some of the directions could be made more detailed for people not familiar with terminal (assume someone doesn't know how to move directories, etc.).
Maybe having clear separations from steps would also help

Chuanyang12 commented 5 years ago

Background: Peking University, Graduate Student GEOS-Chem experience: 0 years

Step 1: 9 minutes Step 2: 18 minutes Step 3: 50 minutes Step 4: 15 minutes

General feedback: 1.It took some time for the verification of the AWS account. 2.In the Step 3 to log into the server, most of time wasted on my carelessness to change the IP to my own IP. Don't copy the example line directly! 3.For people with 0 experience in Geoschem and nearly no experience in modelling like me, I run the GeosChem within 2 hours (1 hour time wasted on unnecessary things). So it's really a great tutorial. Although it will take aditional time to become fully familiar with the GeosChem in AWS (like the output data analysis and download), it's still a great inspiration if you can start it quickly.

Maybe you can add more detailed instructions on how to run our self-designed GeosChem model

lyralan commented 5 years ago

Background: MIT, Graduate Student GEOS-Chem experience: 0 years

Step 1: 5 minutes Step 2: 5 minutes (waited for another 20 min to verify AWS) Step 3: 15 minutes Step 4: 20 minutes

General feedback: First time to use AWS but found it extremely user-friendly with Jiawei's tutorial. Everything went well under instruction. I am not a python/jupyter wizard but with this pleasant trial I would like to explore more into it.

lhenneman commented 5 years ago

Background: Harvard University, Postdoc, School of Public Health, Postdoc GEOS-Chem experience: 0 years Step 1: 8 minutes Step 2: 12 minutes (15 min. for AWS verification) Step 3: 4 minutes Step 4: 7 minutes General feedback: None. Thank you for putting this together.

ZhongjingJiang commented 5 years ago

Background: Peking University, Graduate Student GEOS-Chem experience: 0 years Step 1: 120 minutes (a lot of time waste for phone verification) Step 2: 12 minutes (another 10 minutes for AWS verification) Step 3: 18 minutes (another 10 minutes for simulation) Step 4: 12 minutes (just analyzed with python, not include Jupyter notebook) General feedback:

Step1：I cannot received the phone call for phone verification, the system may have some problem for Chinese users and it took me quite a lot time to ask the support center to do the verification.
Step3：When login used ubuntu, there showed something wrong with authentication and made me really confused. And it took me some time to figure out. Just choose "yes" when they ask you if you still want to establish the connection.
In general, It's a good tutorial. I can finish the test according to these steps. However, for user without GEOS-Chem experience and people not familiar with Linux. It will take longer time to finish.

jimmielin commented 5 years ago

Background: Peking University, Undergraduate Student GEOS-Chem experience: 1 year

Step 1: 10 minutes Step 2: 5 minutes Step 3: 15 minutes (there was considerable SSH lag from China.) Step 4: 8 minutes (I used the pre-provided output file)

Step 2 took shorter time for me because I already had an AWS account, so I was already verified. From my memory signing up for AWS did require some verification time, around ~10 minutes I believe for Step 1 & 2.

Overall it is a great tutorial! Thanks for putting it together.

lshen2009 commented 5 years ago

Background: Harvard University, Post-doc GEOS-Chem experience: 7 years

Step 1: 10 min Step 2: 3 min Step 3: 9 min Step 4: 3 min

Great tutorial!

giscc commented 5 years ago

Background: University of Lille, LOA/CNRS, Postdoc

GEOS-Chem experience: 2 years Step 1: 20 minutes (lots of time wasting for account verification and activation) Step 2: 10 minutes Step 3: 10 minutes Step 4: 15 minutes

General feedback: Great tutorial, easy to follow.

FeiYao-Edinburgh commented 5 years ago

Background: The University of Edinburgh, School of GeoSciences, Graduate Student GEOS-Chem experience: 0 years Step 1: 8 minutes Step 2: 7 minutes plus 13 minutes for acount verification Step 3: 5 minutes plus 8 min to complete simulation Step 4: 9 minutes plus 3 minutes to open jupyter notebook in Linux environment. General feedback: In step 4, everything was ok until I tried open the jupyter url using Google Chrome in Windows environment. I tried for several times and it's still not ok now. Therefore, I tried connect to jupyter notebook in Linux environment and succeeded in less than 3 minutes. I still do not know why I cannot open the url in Windows environment as the Git Bash showed no error. It's quite inspiring to get the GEOS-Chem model run for a novice like me. I would be grateful if you could write more self-customized GEOS-Chem running tutorials based on AWS!

yantosca commented 4 years ago

Closing out this issue now.

geoschem / geos-chem-cloud