Create account and explore

[zajichek]

Make an account on AWS and explore various options for subscribing to free resources. Some questions:

• What server options are there?
• What is the difference between them?
• How do I setup R Shiny web apps to run?

[zajichek]

Following the steps detailed here: https://www.charlesbordet.com/en/guide-shiny-aws/

Making the server

1. Create a Shiny app

A very basic application has been developed and committed to this repository: https://github.com/zajichek/shinydemo-aws/commit/2c5c70b9873ec97bd3fc27566ae4c7778a0161d5

It just takes samples from a Normal distribution and plots them.

2. Create an AWS account

• Created an account using my personal email address
• Had to input a credit card, but you can use free tier services for demo purposes
• We are led to a AWS management console
• It seems to take a while to verify the account, which must be done to start creating a server (took about 30-60 minutes)

Launching an instance

• Searched for "EC2" and opened the console
• On the LHS, clicked "Instances", then "Launch Instances"
• Created a tag
• For the Amazon Machine Images (AMI), selected "Ubuntu"
• Selected the "t2.micro" instance, as it's sufficient for demo purposes and free tier eligible
• Created a key-pair, which is basically just an access file I'll need to access my server instance (.pem file). Named it "aws_test", then created it, and saved it to my local machine.
• Left the default Network Settings as-is
• Left the Configure Storage as default (8 GB of storage)
• Then I launched the instance

I get 1 year of free (limited) usage. The t2.micro costs ~$.01 dollars per hour after that, assuming that hours are compute time, not just the existence of the server. But maybe I'm wrong. I also activated AWS Free Tier email alerts, so it'll keep track when I'll actually be charged for something, if that were to happen.

3. Connecting to the server

Clicked the button to connect to the instance

On this page, just left as-is. Then pressed "Connect". It took me to a shell console where I was on the server.

If I go back to the main instances page in my AWS console, I see my instance, then I click "Connect" at the top, it'll take me back to that above panel.

Then in order to connect from my local machine (Mac OS), I go to the SSH client tab, and view the address

Then I just did what the directions said, once I opened Terminal and navigated to the located of my .pem file:

chmod 400 "aws_test.pem"
ssh -i "aws_test.pem" ubuntu@ec2-3-145-84-43.us-east-2.compute.amazonaws.com

And I was in

My user name is "ubuntu", and we can say there is nothing in here. There is stuff at lower levels:

[zajichek]

Starting from the previous comment...

Using this as reference: https://www.charlesbordet.com/en/guide-shiny-aws/#how-to-install-shiny-server

Installing R and Shiny Server

R is not installed, but it gives you a way to do it.

In the tutorial, it says to run these steps

$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
$ sudo add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/'
$ sudo apt update
$ sudo apt install r-base r-base-dev
$ sudo R
> install.packages("shiny")

However, on the CRAN page (https://cloud.r-project.org/index.html), there are some slightly different, albeit similar steps.

# update indices
sudo apt update -qq
# install two helper packages we need
sudo apt install --no-install-recommends software-properties-common dirmngr
# add the signing key (by Michael Rutter) for these repos
# To verify key, run gpg --show-keys /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc 
# Fingerprint: E298A3A825C0D65DFD57CBB651716619E084DAB9
wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | sudo tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
# add the R 4.0 repo from CRAN -- adjust 'focal' to 'groovy' or 'bionic' as needed
sudo add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/"
sudo apt install --no-install-recommends r-base

I tried following the CRAN page first,

ubuntu@ip-172-31-5-38:~$ sudo apt update -qq
26 packages can be upgraded. Run 'apt list --upgradable' to see them.
ubuntu@ip-172-31-5-38:~$ sudo apt install --no-install-recommends software-properties-common dirmngr
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
software-properties-common is already the newest version (0.99.48).
software-properties-common set to manually installed.
dirmngr is already the newest version (2.4.4-2ubuntu17).
dirmngr set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 26 not upgraded.

It just updated some stuff, and the dirmngr was already up to date. Now if we check the key as described (from CRAN page):

ubuntu@ip-172-31-5-38:~$ gpg --show-keys /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc 
gpg: directory '/home/ubuntu/.gnupg' created
gpg: keybox '/home/ubuntu/.gnupg/pubring.kbx' created
gpg: can't open '/etc/apt/trusted.gpg.d/cran_ubuntu_key.asc': No such file or directory

The file doesn't exist, which I believe is why in the tutorial he has this line sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9 (note the Fingerprint matches in the tutorial what was on the CRAN page). So we ran that, and it appears to have done something, but the this command is deprecated. So instead we ran wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | sudo tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc as described on the CRAN page, and now when we run gpg --show-keys /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc it shows something

I'm assuming this is the intended outcome.

Then I ran the next line from the CRAN page, which is the same as in the tutorial except it will work on different flavors on ubuntu: sudo add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/", and we get success:

Now the update part seemed to update things related to CRAN:

Finally, we will install R (going with the CRAN page command): sudo apt install --no-install-recommends r-base

This led to many errors and warnings when trying to install Shiny in the next step, so I'm trying again: sudo apt install r-base r-base-dev

That appeared to install a lot more stuff and led to a successful installation of shiny with:

$ sudo R
> install.packages("shiny")

Although there were many messages/warnings with all of the C code running, so hopefully that doesn't cause issues.

Then I went to this page (https://posit.co/download/shiny-server/) as directed and copied the commands under Ubuntu to install:

sudo apt-get install gdebi-core
wget https://download3.rstudio.org/ubuntu-18.04/x86_64/shiny-server-1.5.22.1017-amd64.deb
sudo gdebi shiny-server-1.5.22.1017-amd64.deb

I now see this on the server

Deploy the app

If we go back to the /etc directory, we can see the port that Shiny runs from by default

Now we have to "Open the ports on the firewall". The AWS UI is different from the tutorial, so we did this...

• Click the instance name from the main page

• Click the Security tab at the bottom, and click the Security Group name:

• Click the "Edit Inbound Rules" button

• Click "Add a rule"

• Add the custom TCP rule according to the guide

And then Save the changes.

Now if we go to the address: http://3.145.84.43:3838/ we see the default Shiny Server app running.

As the Welcome page suggests, I installed rmarkdown on the server as well.

Which we now see the Shiny document running:

The index.html file is what is running at the root of the server address when access on the web, so you would just edit that to create a landing page.

[zajichek]

Loading this app on the server

We'll do this by directly importing this git repository into /srv/shiny-server/. I tried cloning the repo directly from within, but we get an permissions error:

So, if we instead do it as the tutorial suggests (by cloning it home and then linking it to the server, that should work: From /home/ubuntu',git clone https://github.com/zajichek/shinydemo-aws.git`

Then we create a symbolic link to the server. I believe this is useful because the source code actually lives on in the home folder, where git can access it, then the symbolic link from the shiny server allows access to it.

cd /srv/shiny-server/
sudo ln -s ~/shinydemo-aws .

So we go into the server, and then create a link mapped to the location in the home directory.

When we go to the address: http://3.145.84.43:3838/shinydemo-aws/ we get an error:

If we look at the configuration file:

If desired, we can have the webpage NOT show the index of apps by changing directory_index on to directory_index off

The tutorial also suggests adding preserve_logs true; to the .conf file, to keep all logs. So I did. (note I had to use control+X to save on Mac).

Then I restarted the shiny server sudo systemctl reload shiny-server

I tried changing file permissions

sudo chown shiny:shiny shinydemo-aws
sudo chown -R shiny:shiny shinydemo-aws

But that didn't work.

The sample apps do work though: http://3.145.84.43:3838/sample-apps/hello/

I tried removing the README and .Rproj files but that didn't work.

Also tried removing the preserve_logs thing in the .conf file, but that didn't work.

I tried installing packages as a super user

sudo su
R
> install.packages("shiny")

But that didn't work either. Also tried doing in .libPaths()[3], that didn't work.

Tried this:

sudo -i
R -e "install.packages('PACKAGENAME', lib='/usr/lib/R/library')"

Didn't work.

It has to be something with the file permissions, as I've created an app explicitly in /srv/shiny-server/ with the same exact contents as this one, and it works: http://3.145.84.43:3838/temp_app/

UPDATE:

Currently the app was symbolically linked where its source was /home/ubuntu. Instead of that, I just directly copy-pasted it into /srv/shiny-server/ (after re-cloning it to have all contents)

From /home/ubuntu (after removing the other ones):

git clone https://github.com/zajichek/shinydemo-aws.git
sudo cp -r shinydemo-aws /srv/shiny-server/
sudo rm -r shinydemo-aws

That works!

[zajichek]

Setting custom domain

In my Netlify domain management for zajichekstats.com I created a new DNS record that was type A, specified the host name as "apps" and entered the IP address. This successfully points the following address to the Shiny server index: http://apps.zajichekstats.com:3838/

And I can access this app that we've deployed here: http://apps.zajichekstats.com:3838/shinydemo-aws/

Removing the port requirement

If we don't want to specify the port number, we can optionally go into /etc/shiny-server/shiny-server.conf and change listen 3838 to listen 80. I did that here:

Then I restarted the server: sudo systemctl restart shiny-server

Then I added a new Inbound rule for the new port:

Success. http://apps.zajichekstats.com/ takes us to the index, and http://apps.zajichekstats.com/shinydemo-aws/ takes us to this app.

/the reason this works is because 80 is the default port for any web address. So when you type a web address, it by default goes to port 80, this that's why it works.

However, as the tutorial notes, this is not optimal, because you may want other stuff on your server. The better way is to use nginx to route ports for you.

Using nginx

[zajichek]

Uploading a rendered Quarto document to the app:

1. Connect to the server and clone repo

# Connect
ssh -i "aws_test.pem" ubuntu@ec2-3-145-84-43.us-east-2.compute.amazonaws.com

# Restart server (there were updates)
sudo systemctl restart shiny-server 

# Import repo
git clone https://github.com/centralstatz/ExampleApps.git

2. Copy/paste the rendered directory to the shiny server

sudo cp -r ExampleApps/GoogleAnalyticsDashboard /srv/shiny-server/

Now when I go here: http://apps.zajichekstats.com/GoogleAnalyticsDashboard/ I get an error

cd /var/log/shiny-server
sudo cat GoogleAnalyticsDashboard-shiny-20240929-130526-34159.log

In the logs I get this message:

So it's trying to run the quarto document, not just render the HTML page.

Then when I tried to access the html file directly (http://apps.zajichekstats.com/GoogleAnalyticsDashboard/GoogleAnalyticsDashboard.html) I was getting a "Not Found" error.

But, I then deleted the .qmd file from the server directory entirely, then when I went to the app:

http://apps.zajichekstats.com/GoogleAnalyticsDashboard/

I was given an index of the files:

Then clicking on the html page (http://apps.zajichekstats.com/GoogleAnalyticsDashboard/GoogleAnalyticsDashboard.html) it actually does render:

So if we wanted to do something this way, you just need to store only the rendered documents on the server, not the code.

I believe that if you just name the HTML file as index.html, maybe with or without the quarto doc, it should actually render at the root instead of the specific page name.

Yes this worked

ubuntu@ip-172-31-5-38:/srv/shiny-server/GoogleAnalyticsDashboard$ sudo cp GoogleAnalyticsDashboard.html index.html
ubuntu@ip-172-31-5-38:/srv/shiny-server/GoogleAnalyticsDashboard$ ls
GoogleAnalyticsDashboard.html  GoogleAnalyticsDashboard_files  custom.scss  images  index.html

Now the root link opens the index file. http://apps.zajichekstats.com/GoogleAnalyticsDashboard/