search

covidsimteam / prox-e

Simple REST APIs and dashboard to showcase CST's core functionalities
https://www.covidsim.team
GNU General Public License v3.0
2 stars 5 forks source link

Azure deployment and pipeline for Dashboard #14

Closed sookah closed 4 years ago

sookah commented 4 years ago

Will be deploying for demo purposes.

the branch at deployments/ui-dash

The build:

The Release:

Deployment:

Sites: https://proxedashstorage.z29.web.core.windows.net/

sookah commented 4 years ago

Azure DevOps setup

The setup / project is now named prox-e

sookah commented 4 years ago

Link repos

For test purposes and initial deployment the repo has been cloned as new repo in azure ops, once pipeline looks good we will make a yaml on the present github repo, this also mean i will not create a branch here.

sookah commented 4 years ago

CI triggers

For now set to master

sookah commented 4 years ago

pipelines for dependencies, build/test

Following pipelines (for dashboard) and relative to the prox-e upper directory has been setup:

prox-e-dashboard CI > dashboard-CI >

They are contained in their own scripts for more modularity

sookah commented 4 years ago

Issues:

Fixes

fixed with @angular/cli@9.1.12 image

add angular-devkit npm install --save-dev @angular-devkit/build-angular

no ngcc yet image

sookah commented 4 years ago

packages status

image

Where as the pacakge json has:

image

there is a mismatch here for compiler on force for test image

sookah commented 4 years ago

ok at this point this definitely also feels like .lock is not correct and in sync with package.json

the angular compiler is 9.1.12 where as package.json has 9.0.4 (caret)

sookah commented 4 years ago

image

fixed with: npm install --save opencollective-postinstall

sookah commented 4 years ago

my global vs dashboard local modules

image

sookah commented 4 years ago

feels like issue on my setup, with angular cli throwing the same error on a newly created project for the sake of finishing i will use the ops servers

sookah commented 4 years ago

failing on mixin for nb-auth image

sookah commented 4 years ago

build passing, next to deploy

image

sookah commented 4 years ago

dist folder not being outputted

also not on ../out-tsc/spec

tested with clean build on gitpod and on azure image

sookah commented 4 years ago

pipelines are complete for deployment, but the build seems to work incorrectly. further investigation is needed. We can also use deployment options from ng for azure

risavkarna commented 4 years ago

Just ran the older solution again and seems to work. Please check latest yml file on master for reference.

done-2

risavkarna commented 4 years ago

The dist folder is set to ui directory on the root in angular.json. This probably needs to be created before the build artifacts are outputted there.

sookah commented 4 years ago

indeed, it must be this, the output path is ok to be in the dist folder, but if we want at build time to be at a different location, example for the ci or top level builds, than we can change with something like --output-path=dist/dashboard | ../../ui

also verified angular.json, tsconfig*

sookah commented 4 years ago

build/stage yaml so far

pool:
  name: Azure Pipelines
steps:
- task: NodeTool@0
  displayName: 'Use Node 14.x'
  inputs:
    versionSpec: 14.x

- script: 'npm install'
  workingDirectory: hi/dashboard
  displayName: 'Install Deps'

- script: 'echo No tests to run for now'
  workingDirectory: hi/dashboard
  displayName: Test

- script: 'npm run build:prod'
  workingDirectory: hi/dashboard
  displayName: Build

- script: |
   find . -maxdepth 2 -type d -exec ls -ld "{}" \;

  workingDirectory: hi/dashboard
  displayName: 'list directory 2 level deep'

- task: CopyFiles@2
  displayName: 'Copy Dist'
  inputs:
    SourceFolder: hi/dashboard/dist
    TargetFolder: '{Build.ArtifactStagingDirectory}/dist'

- task: ArchiveFiles@2
  displayName: 'Archive $(Build.ArtifactStagingDirectory)'
  inputs:
    rootFolderOrFile: '$(Build.ArtifactStagingDirectory)'

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: dashboard'
  inputs:
    ArtifactName: dashboard
sookah commented 4 years ago

passing builds now image

sookah commented 4 years ago

Release phase image

risavkarna commented 4 years ago

Do you have the url or IP? Need to install couch in 5984 on that vm. And nginx conf.

risavkarna commented 4 years ago

Also 'Move pipelines to present repo once verified(github)Currently in Karnali test mode' can be checked off soon I guess.

sookah commented 4 years ago

Artifact looks empty

for the url part this has to go to azure blobs, will be setting that up now. I have noticed another issue will try to solve that first:

image the artifact looks empty. Ill fix the final archive parts issues if any

Artifact builds are now resolved ✅

image

for the 'move pipelines..'

sookah commented 4 years ago

since the deployment is a blob, for couch we may want to host another place? that keeps it separate to the frontend deployment.

we can spin up a VM or an app (or two for couch and the deployment).

simpler item is just a VM with both couch and nginx serving the dashboard assets

sookah commented 4 years ago

Recommendation, IDeation

sookah commented 4 years ago

Deployment

This is for creating and monitoring the deployment as an app service image

Create Azure App [Done] ✅

image

Connect to release pipeline

sookah commented 4 years ago

Artifact builds are now resolved ✅

image

sookah commented 4 years ago

Deployment pipeline now linked to App:

image

Issue

page not updated on deployment

ssh-ing into the box shows that the folders are there image

sookah commented 4 years ago

Fails on SCM

image

sookah commented 4 years ago

Will be creating an ASP.Net container to see if the deployments are better there, there processes are bound to change, but for now we will need to see the application running to figure more pipelines (eg couch)

sookah commented 4 years ago

Move pipelines to present repo once verified(github)

Starting to create the pipeline now

Issue

image

sookah commented 4 years ago

Azure blob for page hosting

https://proxedashstorage.z29.web.core.windows.net/

Blob is now operational so is the app, they both throw same errors on production to the guess here is some routes are breaking. Error message highlights missing es2015 packages, tsconfig can be a starting place

image

sookah commented 4 years ago

Move pipelines to present repo once verified(github)

Pipeline is now dependent on the master of this repo. 🎲

Final builds

image

image

sookah commented 4 years ago

Updating the output path to dist

i have removed ../../ui and the need for creating a directory from angular.json

the reason is that the apps(dashboard) will be modular, azure pipeline seems to have context and directories based tasks. this allows for a complete (and not having to create dirs on top level since the app is not aware of it) atomic builds on each. For now only dashboard has been done in this manner. Also pipelines like directory trigger(for example changes on only hi/dashboard can trigger a new pipeline and the pipeline will remain inactive if others for eg. hi/prox-e is updated

For a mono repo and a high level build interface a seperate pipeline can be maintained.

@risavkarna bro what is your idea on this ? new issue? I will close this one for now since all items are complete.

risavkarna commented 4 years ago

@sookah, we can revisit this when we have prox-e in the pipeline.

risavkarna commented 4 years ago

The code for these changes do not seem to have been merged to the repo yet. Also, which azure VM is this hosted on? Is the azure dev pipeline setup created outside of the official Covid HPC Consortium subscription?

risavkarna commented 4 years ago

covid.cosys.work and covid.nep.work domains are allowed. Now we need to add URL namespaces to the dashboard and the couch central master DBMS and serve them with SSL/TLS

sookah commented 4 years ago

@risavkarna