DEPLOYMENT ISSUE: Fails when using a Github Action with a Github Hosted Runner

[MichaelAkridge-NOAA]

What is the name of the product?

NMFS Color Palettes Explorer

Link to the product's code, if available (e.g., to the GitHub Source Code)?

• https://github.com/MichaelAkridge-NOAA/NOAA-NMFS-Brand-Resources/tree/main/app/NMFS_Color_Palettes_Explorer

Which type of product were you trying to deploy (e.g., Quarto Doc, Shiny App, etc.)?

• streamlit python app

How were you trying to deploy it (e.g., One click from the Rstudio IDE, Git-backed workflow)?

Via github action workflow - github hosted runner

• https://github.com/MichaelAkridge-NOAA/NOAA-NMFS-Brand-Resources/blob/main/.github/workflows/deploy_app_github_runner.yml

What did you experience?

When deploying via a GitHub action workflow(GitHub hosted runner), the process will fail. Unable to reach server. Error below:

Error: Exception trying to connect to https://test-connect.fisheries.noaa.gov/ - [Errno 110] Connection timed out
Error: Process completed with exit code 1.

and via another method this error:

Error: Error: connect ETIMEDOUT 137.75.93.30:443

What do you hope to experience?

I would hope to see it successfully deploy via a GitHub runner to streamline publishing, devops and eliminate the need for a self hosted runner.

To help with troubleshooting, I've been successful at deploying the app with the same github action and code via a self hosted runner.

So I imagine there could be some posit server config that doesn't allow specific domain traffic? But any and all help is greatly appreciated.

• https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/about-githubs-ip-addresses
• https://docs.github.com/en/get-started/using-github/allowing-access-to-githubs-services-from-a-restricted-network

Edit 20240703: Some additional information: I also tried using the rsudio connect-publish action but got the same error. Even after generating a manifest.json file.

• https://github.com/rstudio/actions/tree/main/connect-publish
• https://github.com/sol-eng/python-examples/blob/main/.github/workflows/deploy.yml

[k-doering-NOAA]

Thanks for reporting @MichaelAkridge-NOAA ! I think the cause of this is because the test/dev instances are only accessible at a science center/on VPN.

Would the Gitbacked workflow work for you? It would allow the app to be automatically updated when changes to a branch are made, which I think is what you are looking for?

[MichaelAkridge-NOAA]

Thanks @k-doering-NOAA ! Yeah that's what I was thinking was the cause as well. Although it also fails when deploying to the prod instance.

Is it possible to have someone look into adding 'github.com' as whitelisted traffic to the instances? Not sure how the server is setup, but possibly by updating the DNS filter, iptables, acls, or etc.?

The gitbacked workflow is great, but its not as programmatic as a true continuous integration and continues deployment (CI/CD) pipeline. Here is the Posit link with more information:

• https://solutions.posit.co/operations/deploy-methods/ci-cd/github-actions/

[k-doering-NOAA]

@MichaelAkridge-NOAA interesting, I've successfully deployed by calling this reusable github action before, but that was about 9 months ago and I haven't tested it since. I will make a note to test it again to see if I am experiencing the same issue that you are running into, then we can loop in our sys admin.

Could you explain the steps you are include in the CI/CD pipeline? I'm wondering if there are other ways to accomplish it, or if a single gha workflow makes the most sense.

[k-doering-NOAA]

@MichaelAkridge-NOAA , I had some issues getting my older workflow to work, so I think there may have been some updates.

I opened a jira issue for our sys admin to track this (I think you should be able to see it, but let me know if not): https://apps-st.fisheries.noaa.gov/jira/browse/NPCG-18

[MichaelAkridge-NOAA]

Hey @k-doering-NOAA , oh that is interesting. I just checked about deploying to prod because I was curious, and the workflow seems to work now! But only after generating & using a new API key.

So that led me to check posit connect, and I did notice some strangeness. Maybe due to server upgrades, but it appears my original prod API key I generated a while back was populated on all three(prod, dev, test) servers. All with the same name, create date, and last four key digits. So that might be why it didn't work. Not sure how that happened.

[k-doering-NOAA]

Aha, thanks for reporting this, Michael - I think this might have been due to how we migrated over to the new OS.

Do you still want to investigate deploying to test, if possible?

I'll flag the key issue for our Sys admin.

[MichaelAkridge-NOAA]

Hey @k-doering-NOAA , just tried same workflow with test URL and a new test api key, but same connection issues sadly. So makes sense GitHub traffic just isn't able to access the dev/test servers and only accessible at a science center/on VPN.

If that's an easy update for Sys admin to clear Github traffic that would be awesome. If its super difficult for some reason, then no biggie. I can always run self hosted runners at my science center. I just know that's not option for everyone.

As for the CI/CD pipeline, I'm still developing it. Was trying to get the basic connect and deploy completed first. But would eventually want to fully integrate more into the GitHub ecosystem now that NMFS has GitHub Enterprise.

After fully developed I would like to use CI/CD for things like:

• update, test, and deploy multiple apps at same time using the gha workflows
• utilize conditional deployment. So if a deployment to dev/test fails for a quick example it would halt further deployment to the prod server and possibly rollback changes/commits
• consistent deployment processes across all environments (dev, test, and prod)
• detailed deployment logs for troubleshooting (instead of manually checking them on posit connect or saving terminal output)
• use Github for cloud development (codespace) and automated testing/deployment( gha workflows)
• integration with other workflows of course to leverage things like
• • code quailty checks for python/r code
• • security scans
• • etc.

[k-doering-NOAA]

Thanks for sharing! You were just added as a watcher on the Jira ticket so we can discuss the test traffic issues with the sys admin here: https://apps-st.fisheries.noaa.gov/jira/browse/NPCG-18

Edit: Great to hear about some of your CI/CD pipeline ideas!

[MichaelAkridge-NOAA]

Thanks @k-doering-NOAA !

[k-doering-NOAA]

Unfortunately, @MichaelAkridge-NOAA discovered github runners do not have static IPs, so setting something up for the test instance isn't something we could pursue right now.

However, using a larger github hosted runner would allow for a static IP and could be done in the future