Continuous site deployment

[BenediktOrnHjaltason]

Find a way for updating the work-in-progress site for each commit.

[jorlar]

Find a way for updating the work-in-progress site for each commit.

I belive that with this you mean some kind of "live edition" of the page while it's a work in progress?

[BenediktOrnHjaltason]

Yes. I have a free tier VM in AWS at the moment where I have been hosting a live version of the site, but something has broken since I turned the instance off from AWS console. I will look into it today.

Up until now I have just been publishing the app on my PC and copy/pasting it into the folder in the VM where IIS references it.

I remember when I was playing around with Heroku some years ago, the hosting was directly connected to the repo and every new commit would automatically update the live site. I am hoping we can get something similar going here.

EDIT: The VM can be remoted into with Remote Desktop:

Computer: ec2-13-48-27-71.eu-north-1.compute.amazonaws.com

User: Administrator

Password: Xt-=7W?CwOh.nP(phefk(3Zzjene$2pe

The site was running on port 81 and was previously reachable with ec2-13-48-27-71.eu-north-1.compute.amazonaws.com:81

EDIT: I suspect something went wrong with the MySql database when I shut down the VM instance. If I try to stop db server in Workbench the program crashes.

EDIT: I figured out that stopping and starting the VM triggered assigning of a new IP-address for VM, which in turn broke the connection string.

[jorlar]

Thats fully possible, at least with Azure. Not that familiar with connection AWS to version-controll such as github. In Azure DevOps you have the same functionality as you had in Herouku.

I guess this would be some kind of a cost-issue...

Edit: Found this tho... https://aws.amazon.com/blogs/devops/integrating-with-github-actions-ci-cd-pipeline-to-deploy-a-web-app-to-amazon-ec2/

[BenediktOrnHjaltason]

That sounds great.

Talking to Atle Magnussen the other day, he is also more familiar with Azure.

I originally was trying to get a VM going in Azure, but it was crap, so tried AWS. But it sounds like a VM is not the way to go anyway for hosting it ;D

I was also previously researching streaming endpoints, and it sounds like Azure's Media Services is cheaper than AWS.

So it sounds like we maybe should go with Azure for hosting + streaming endpoints when the time comes.

Sounds like a free trial of Azure with its 200$ trial period could get us far.

[atlemagnussen]

I would like to pose a suggestion on hosting in Azure.

At least regarding the Test Environment of the application while in development. We don't need to decide the "Prod" version now necessarily - but it would of course be better if they are the same model in the end :)

The "obvious choice" would be Azure App Service. Very little hazzle, proven and stable. But, it comes with a monthly steady cost, not based on consumption. Not ideal for a Test site which would be idle most of the time. it does not auto scale, only manually - like dragging a slider to give it more power. But it might be OK for production when/if traffic is steady anyways.

I have been waiting for Azure to provide a simple hosting model based on consumption and it seemed it arrived last year, something called Azure Container Apps. So it would use docker, containers and whatever under the hood, but we don't have to think too much about that once CI CD is up :) They have other models but they seem to be more complex, personally I don't want to micro manage Kubernetes stuff :)

This would be something like Heroku, since you mentioned it. Also similar too Google Cloud Run, which I have tested and is a very good model.

This model will scale down to zero containers and cost nothing when idling and pricing would be per request, cpu, memory and so on when active. It will auto scale when traffic increases - which makes it suitable for production as well, probably. Azure Container Apps Pricing

[BenediktOrnHjaltason]

@atlemagnussen Azure Container Apps sounds extremely interesting!

Sounds like we could use it for hosting a work-in-progress testing site with negligible cost. Also it seems like the most modern way of doing it. I would be comfortable setting this up on my personal Azure instance to start with.

Which CI tools do you guys have most experience with? Circle CI?

[atlemagnussen]

@BenediktOrnHjaltason I have used Azure Pipelines mostly and yes it works well with Azure hosting, but not terribly impressed either.

It seems to me that Microsoft is focusing more on Github Actions

I have tested it briefly and it's kind of the same concept as most CI I have seen, set up config file, usually yaml, hazzle around until it's right and then it's good. But the idea of having all the "code and build" in one place sounds nice, to me at least :)

[daghendrik]

Hi guys 👋 Jumping into this conversation to offer my two cents:

I'm not very familiar with Azure, or the .NET stack for that matter, but I have some experience with other platforms that might suit this web app and organization.

Fly.io has a generous (forever) free plan that works nicely for low traffic sites. As long as you can make a docker image of your app, you can deploy it. I've used it for some small projects, such as a genome research database built on PostgreSQL, Dropwizard & Angular. I found a guide on how to publish an ASP.NET Core App in Fly.io., that might be of use. From my own experience, Fly.io integrates nicely with GitHub actions too.

If (re)writing the app in another language is an option, and separating the API and the front-end is a part of that, then either using Github Pages or Cloudflare Pages for (free) hosting of the frontend would be my recommendation. I would then pair that with an auto-scaling "serverless" platform (pay for what you use), such as AWS Lambda (either directly or via serveless.com) or Cloudflare Workers.

Since the database would most likely be read-heavy, and few if any concurrent writes, I'd look at SQlite as an option for the DB. Cloudflare D1, currently in alpha, is a managed SQLite service that looks very interesting (low cost, ensures ACID, distributed). LiteFS (from fly.io) is another interesting SQLite service.

As for streaming providers, the cheapest one that I've found is Cloudflare Stream. I have built services on top of their on-demand streaming product, and it has worked nicely

[BenediktOrnHjaltason]

@daghendrik Thank you for the input!

I will be taking a good look at Fly.io.

Cloudflare is coming up a lot recently as a good option for handling streaming.

I think since there is a good amount of experience with ASP.NET within the group, we should maybe stick with that.

But it is possible to have an ASP.NET project + React, or ASP.NET + Angular. There are project templates for that in Visual Studio. Would it be possible to make a more "sexy" site with that?

Please excuse my lack of knowledge about front-ends. How does it work hosting Backend and Frontend separately?

[atlemagnussen]

Ideally if we go the frontend-path you want to keep it out of Visual Studio, since its not a good editor for html, js and css :) usually it could be just in another folder, or repo.

ASP.net can be both server side rendering of web pages and API, so we can keep it and configure it to serve as a web api. Still using the same setup you have going with database and all that.

The web app can be hosted as static HTML, JS and CSS files on one subdomain (typically www.hessdalen.org) and the server in another subdomain (i.e. server.hessdalen.org), that is a clean separation Also possible to deploy them together so that the api would be www.hessdalen.org/api i.e.

Anyways, then the web app client calls the server via REST when it needs to get or store data

[daghendrik]

Piggybacking on @atlemagnussen's answer; with a separate frontend and backend on different domains you'll possibly run into issues with CORS and cookie settings ("samsite" for JWT auth cookies, for example), but all that's straightforward to handle. Keeping things on subdomains of the same root domain removes some of those challenges, like @atlemagnussen mentions.

Using server-side rendering has its benefits, but for my own projects I tend to host the api and the js/html on separate servers and domains. I feel that it's a more flexible approach and can be useful if you later want to create other clients for the api than just the web app (native mobile app for example). Not that you should engineer for a need that isn't there yet, but it's worth mentioning.

For front-end I've got the most experience with hosting on S3 + Cloudfront (CDN), but I've also used Github pages and cloudflare pages.

ASP.NET Core and C# sound good to me. Don't have any problem with that stack and I fully agree that you should go with what we/you're most familiar/productive with.

[BenediktOrnHjaltason]

It should be said that it is in the plans to have automatic detection of phenomenon events in Hessdalen, and present these events on the website with video clips and other sensor data. So there is a future need for a REST API to send that data to the server.

Hakan Kayal, who is in the Slack and is a PhD in aerospace engineering at a german university, is doing similar research with detection of UFO-phenomenon with machine learning algorithms to exclude mundane flying objects.

[atlemagnussen]

it would be such a kick to get like a push notification when there is a new phenomena detected, wow :)

I would agree with the flexibility of web api/app separation. SSR only will always be more kind of limited and sutiable for web pages that don't have too much fancy interactivity, even though it is possible to mix things up - that just creates more work. Sounds like we are heading towards more "app like" use cases here and modern front end frameworks are really good.

[BenediktOrnHjaltason]

Absolutely, that would be awesome. People would love that. 😀

That's actually a great idea. Members could get push or email notifications, and access to recorded events asap. And then publicly visible on website after a delay.