M1-12 Maintenance and DevOps

[elisabettai]

(This is a task, not a milestone) Continuously maintain o²S²PARC platform, harden, and increase stability and scalability to accommodate increased use.

For example:

• enable scaling of selected core services through recently introduced modularization and plugin concept
• file-system performance boosting, especially for large files

Deliverable: monthly update slides as part of the NIH SIM-Core update meeting Deadline: Month 1–12 (continuous task)

DETAILS IN https://github.com/orgs/ITISFoundation/projects/9/views/34

### Watermelon
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/172
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/173
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/176

### Sundae
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/176
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/200
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/190
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/215
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/248
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/250
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/258
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/262
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/274
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/276
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/278
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/282
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/287

### Baklava (ops)
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/pull/310
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/pull/312
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/pull/313
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/pull/314
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/319
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/pull/322
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/323
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/pull/329
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/332
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/issues/333
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/pull/335
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/pull/338
- [ ] https://github.com/ITISFoundation/osparc-ops-environments/pull/340

[elisabettai]

This is a continuation of ITISFoundation/osparc-simcore#428, fresh start into Y6. Could also be used to write updates on DevOps side.

[pcrespov]

Update on sprint Vaporwave

Done

• https://github.com/ITISFoundation/osparc-simcore/issues/3398
• https://github.com/ITISFoundation/osparc-simcore/issues/3404
• https://github.com/ITISFoundation/osparc-simcore/issues/3408
• https://github.com/ITISFoundation/osparc-simcore/pull/3403
• https://github.com/ITISFoundation/osparc-simcore/pull/3401
• ITISFoundation/osparc-simcore#3379
• ITISFoundation/osparc-simcore#3367
• ITISFoundation/osparc-simcore#3365
• ITISFoundation/osparc-simcore#3363
• Add: Seventh machine on master
• Prometheus: Update, Crash, Reasonings
• Expose PostgresDB: Enable inter-deployment copying script
• oSparc Infra changes: SSH-Hardening
• ITISFoundation/osparc-simcore#3358
• Create Mattermost Channel Alert for graylog
• Bugfix: Docker machine label changes
• E-Mail Script: Notify logged in users during maintenance
• Update logspout-gelf service to latest version and move to itisfoundation github & dockerhub
• Update grafana major version
• Upgrade of docker registry
• New ITISFoundation/osparc-simcore#734
• https://github.com/ITISFoundation/osparc-issues/issues/558
• https://github.com/ITISFoundation/osparc-issues/issues/619
• https://github.com/ITISFoundation/osparc-simcore/pull/3112

Ongoing

• https://github.com/ITISFoundation/osparc-issues/issues/638
• ITISFoundation/osparc-issues#617

Open

• https://github.com/ITISFoundation/osparc-issues/issues/666
• https://github.com/ITISFoundation/osparc-issues/issues/735
• https://github.com/ITISFoundation/osparc-simcore/issues/2008
• https://github.com/ITISFoundation/osparc-simcore/issues/2212
• https://github.com/ITISFoundation/osparc-simcore/issues/2260
• https://github.com/ITISFoundation/osparc-simcore/issues/2354
• https://github.com/ITISFoundation/osparc-simcore/issues/2410
• https://github.com/ITISFoundation/osparc-simcore/issues/2886
• https://github.com/ITISFoundation/osparc-simcore/issues/3073
• https://github.com/ITISFoundation/osparc-simcore/issues/3145
• https://github.com/ITISFoundation/osparc-simcore/issues/3151
• https://github.com/ITISFoundation/osparc-simcore/pull/2817
• https://github.com/ITISFoundation/osparc-simcore/issues/2156
• https://github.com/ITISFoundation/osparc-simcore/issues/2715
• https://github.com/ITISFoundation/osparc-simcore/issues/2769
• https://github.com/ITISFoundation/osparc-simcore/issues/2771
• https://github.com/ITISFoundation/osparc-simcore/issues/2980
• https://github.com/ITISFoundation/osparc-simcore/issues/2994
• https://github.com/ITISFoundation/osparc-simcore/issues/2991
• https://github.com/ITISFoundation/osparc-simcore/issues/3033
• https://github.com/ITISFoundation/osparc-simcore/issues/3078
• https://github.com/ITISFoundation/osparc-simcore/issues/3033
• https://github.com/ITISFoundation/osparc-simcore/issues/3256
• https://github.com/ITISFoundation/osparc-simcore/issues/3265
• ITISFoundation/osparc-simcore#3387
• ITISFoundation/osparc-issues#750
• ITISFoundation/osparc-simcore#3375
• ITISFoundation/osparc-simcore#3374
• ITISFoundation/osparc-simcore#3372
• ITISFoundation/osparc-simcore#3373
• ITISFoundation/osparc-simcore#3371
• ITISFoundation/osparc-simcore#3360

[sanderegg]

Goal for sprint Katherine Switzer

• https://github.com/ITISFoundation/osparc-issues/issues/644
• https://github.com/ITISFoundation/osparc-simcore/issues/2410

[mrnicegyu11]

Update on sprint Katherine Switzer

User support

• https://github.com/ITISFoundation/osparc-issues/issues/754
• https://github.com/ITISFoundation/osparc-issues/issues/644
  • https://github.com/ITISFoundation/osparc-simcore/pull/3463
  • https://github.com/ITISFoundation/osparc-simcore/pull/3464
  • https://github.com/ITISFoundation/dockerfiles/pull/72
  • https://github.com/ITISFoundation/osparc-simcore/pull/3473
  • https://github.com/ITISFoundation/osparc-simcore/pull/3509
• https://github.com/ITISFoundation/osparc-issues/issues/594
  • https://github.com/ITISFoundation/osparc-simcore/issues/3447
  • https://github.com/ITISFoundation/osparc-simcore/pull/3501

Security

• https://github.com/ITISFoundation/osparc-issues/issues/739
  • https://github.com/ITISFoundation/osparc-simcore/pull/3510

Bugfixes / Technical debt

• https://github.com/ITISFoundation/osparc-simcore/pull/3498
• https://github.com/ITISFoundation/osparc-simcore/pull/3490
• https://github.com/ITISFoundation/osparc-simcore/pull/3477
• https://github.com/ITISFoundation/osparc-simcore/issues/3453
• https://github.com/ITISFoundation/osparc-simcore/pull/3454
• https://github.com/ITISFoundation/osparc-simcore/issues/3372
• https://github.com/ITISFoundation/osparc-simcore/pull/3448
• https://github.com/ITISFoundation/osparc-simcore/pull/3440
• https://github.com/ITISFoundation/osparc-simcore/pull/3431

CI/CD

• https://github.com/ITISFoundation/osparc-simcore/issues/2410
• https://github.com/ITISFoundation/osparc-simcore/pull/3480
• https://github.com/ITISFoundation/osparc-simcore/pull/3369
• https://github.com/ITISFoundation/osparc-simcore/pull/3429
• https://github.com/ITISFoundation/osparc-simcore/issues/3435
• https://github.com/ITISFoundation/osparc-simcore/pull/3489
• https://github.com/ITISFoundation/osparc-simcore/pull/3481
• https://github.com/ITISFoundation/osparc-simcore/pull/3434

e2e testing

• https://github.com/ITISFoundation/osparc-simcore/pull/3432
• https://github.com/ITISFoundation/osparc-simcore/pull/3495

Pre/releases during this sprint

• https://github.com/ITISFoundation/osparc-issues/issues/750
• https://github.com/ITISFoundation/osparc-issues/issues/734
• https://github.com/ITISFoundation/osparc-issues/issues/744
• https://github.com/ITISFoundation/osparc-issues/issues/753
• https://github.com/ITISFoundation/osparc-issues/issues/759

Dependencies

• https://github.com/ITISFoundation/osparc-simcore/pull/3500
• https://github.com/ITISFoundation/osparc-simcore/pull/3492
• https://github.com/ITISFoundation/dockerfiles/pull/73
• https://github.com/ITISFoundation/osparc-simcore/pull/3442
• https://github.com/ITISFoundation/osparc-simcore/pull/3444
• https://github.com/ITISFoundation/osparc-simcore/pull/3451
• https://github.com/ITISFoundation/osparc-simcore/pull/3496
• https://github.com/ITISFoundation/osparc-simcore/pull/3497

DevOps

• Traefik Dashboard /api handling, which leads to not showing weird errors if a jupyter-service container dies (thx @elisabettai )
• Add log-rotation to docker daemon
• Dont run docker daemon on loglevel DEBUG
• Harmonize /docker mounts
• Added 2 grafana visualisations
• Add e2e test for oSparc products,APIs, and OpenAPI Swagger Doc Pages
• Allow for CPU/RAM limits in simcore & ops services, mitigating https://github.com/docker/compose/issues/7771
• Add pre-commit hooks and linting to opsarc-ops repo (thx @pcrespov)
• Add graylog alert for dy-volume-removal finding zombies

[pcrespov]

Goal for sprint Athena

• https://github.com/ITISFoundation/osparc-simcore/issues/2410
• https://github.com/ITISFoundation/osparc-simcore/issues/2393
• https://github.com/ITISFoundation/osparc-issues/issues/594
• https://github.com/ITISFoundation/osparc-issues/issues/739
• https://github.com/ITISFoundation/osparc-issues/issues/759
• https://github.com/ITISFoundation/osparc-simcore/issues/3421
• https://github.com/ITISFoundation/osparc-issues/issues/638
• https://github.com/ITISFoundation/osparc-simcore/issues/2625
• https://github.com/ITISFoundation/osparc-simcore/pull/3502
• https://github.com/ITISFoundation/osparc-issues/issues/735
• https://github.com/ITISFoundation/osparc-simcore/issues/3447
• https://github.com/ITISFoundation/osparc-simcore/pull/3480
• https://github.com/ITISFoundation/osparc-simcore/pull/3553
• https://github.com/ITISFoundation/osparc-issues/issues/771
• https://github.com/ITISFoundation/osparc-issues/issues/775

[sanderegg]

Update on sprint Athena

User support

Security

Bugfixes / Technical debt

• https://github.com/ITISFoundation/osparc-simcore/issues/2625, https://github.com/ITISFoundation/osparc-simcore/issues/3564
• https://github.com/ITISFoundation/osparc-simcore/pull/3553
• https://github.com/ITISFoundation/osparc-simcore/issues/3421
• https://github.com/ITISFoundation/osparc-issues/issues/638
• https://github.com/ITISFoundation/osparc-simcore/pull/3502
• https://github.com/ITISFoundation/osparc-simcore/issues/3598
• https://github.com/ITISFoundation/osparc-simcore/issues/3593
• https://github.com/ITISFoundation/osparc-simcore/issues/3599
• https://github.com/ITISFoundation/osparc-simcore/issues/3594
• https://github.com/ITISFoundation/osparc-simcore/issues/3552
• https://github.com/ITISFoundation/osparc-simcore/issues/3560
• https://github.com/ITISFoundation/osparc-simcore/issues/3517
• https://github.com/ITISFoundation/osparc-simcore/pull/3525
• https://github.com/ITISFoundation/osparc-simcore/pull/3513
• https://github.com/ITISFoundation/osparc-simcore/pull/3571
• https://github.com/ITISFoundation/osparc-simcore/pull/3580

  CI/CD

• https://github.com/ITISFoundation/osparc-simcore/pull/3480
• https://github.com/ITISFoundation/osparc-simcore/issues/3588

  e2e testing

• https://github.com/ITISFoundation/osparc-simcore/issues/3575
• https://github.com/ITISFoundation/osparc-simcore/issues/3539

  Pre/releases during this sprint

• https://github.com/ITISFoundation/osparc-issues/issues/759
• https://github.com/ITISFoundation/osparc-issues/issues/771
• https://github.com/ITISFoundation/osparc-issues/issues/775

  Dependencies

• https://github.com/ITISFoundation/osparc-issues/issues/735
• https://github.com/ITISFoundation/osparc-simcore/issues/3605

  DevOps

• Bugfix release for deployment-agent: v.0.10.4 : https://github.com/ITISFoundation/osparc-deployment-agent/issues/121
• Remove burstable aws instances
• PG Migration v10 --> v14
• Refactor ops-repo
• Licensing server adjustments
• Add pyenv to all machines
• Add secure sudo password to all machines
• Increase machine memory on osparc.io managers
• Terraform: Prototype for EC2, VPC, Subnets, Elastic IPs, SSH Keys, LoadBalancer
• Admin-Panel: Add docker socket access, script by @GitHK

[sanderegg]

Goal for sprint Zefram Cochrane

• Staging releases
• Production releases
• Remove openapi-core library from osparc-simcore, https://github.com/ITISFoundation/osparc-simcore/issues/3515
• https://github.com/ITISFoundation/osparc-simcore/issues/3638
• https://github.com/ITISFoundation/osparc-simcore/issues/3421
• https://github.com/ITISFoundation/osparc-issues/issues/638
• https://github.com/ITISFoundation/osparc-simcore/issues/2410
• https://github.com/ITISFoundation/osparc-simcore/issues/3541
• https://github.com/ITISFoundation/osparc-simcore/issues/3387

[sanderegg]

Update on sprint Zefram Cochrane

User support

• https://github.com/ITISFoundation/osparc-issues/issues/724
• https://github.com/ITISFoundation/osparc-issues/issues/797
• https://github.com/ITISFoundation/osparc-issues/issues/799
• https://github.com/ITISFoundation/osparc-issues/issues/803
• https://github.com/ITISFoundation/osparc-issues/issues/809
• https://github.com/ITISFoundation/osparc-issues/issues/810
• https://github.com/ITISFoundation/osparc-issues/issues/811

  Security

  Bugfixes / Technical debt

• https://github.com/ITISFoundation/osparc-simcore/issues/3582
• https://github.com/ITISFoundation/osparc-simcore/issues/3642
• https://github.com/ITISFoundation/osparc-simcore/issues/3648
• https://github.com/ITISFoundation/osparc-simcore/issues/3669
• https://github.com/ITISFoundation/osparc-simcore/issues/3674
• https://github.com/ITISFoundation/osparc-simcore/issues/3675
• https://github.com/ITISFoundation/osparc-simcore/issues/3677
• https://github.com/ITISFoundation/osparc-simcore/issues/3678
• https://github.com/ITISFoundation/osparc-simcore/issues/3686
• https://github.com/ITISFoundation/osparc-simcore/issues/3689
• https://github.com/ITISFoundation/osparc-simcore/issues/3691
• https://github.com/ITISFoundation/osparc-simcore/issues/3695
• https://github.com/ITISFoundation/osparc-simcore/issues/3711
• https://github.com/ITISFoundation/osparc-simcore/issues/3712
• https://github.com/ITISFoundation/osparc-simcore/issues/3716
• https://github.com/ITISFoundation/osparc-simcore/issues/3717
• https://github.com/ITISFoundation/osparc-simcore/issues/3719
• https://github.com/ITISFoundation/osparc-simcore/issues/3720
• https://github.com/ITISFoundation/osparc-simcore/issues/3740

CI/CD

• https://github.com/ITISFoundation/osparc-simcore/issues/3680
• https://github.com/ITISFoundation/osparc-simcore/pull/3681
• https://github.com/ITISFoundation/osparc-simcore/issues/3681

  e2e testing

• https://github.com/ITISFoundation/osparc-simcore/issues/3650
• https://github.com/ITISFoundation/osparc-simcore/issues/3707

  Pre/releases during this sprint

• https://github.com/ITISFoundation/osparc-issues/issues/795
• https://github.com/ITISFoundation/osparc-issues/issues/800
• https://github.com/ITISFoundation/osparc-issues/issues/805
• https://github.com/ITISFoundation/osparc-issues/issues/813
• https://github.com/ITISFoundation/osparc-issues/issues/820

Dependencies

• https://github.com/ITISFoundation/osparc-simcore/issues/3682
• https://github.com/ITISFoundation/osparc-simcore/issues/3735

  DevOps

• Re-Establish Postgres database backups
• Create AWS Machine Image (AMI) with S4L-lite already on it
• Enable dynamic services in local deployment
• Switch staging.osparc.io to new github repository
• Proof of Concept for limiting maximum filesize in a docker container
• Add s4l-lite domains and products everywhere
• Enable E2E tests when maintenance page is still up
• Fix: graylog dashboard provisioning
• Service deprecation on osparc.io (Date: 30th of Jan 2023)

[sanderegg]

Update on sprint Resistance is Futile

User support

• https://github.com/ITISFoundation/osparc-issues/issues/823
• https://github.com/ITISFoundation/osparc-issues/issues/843
• https://github.com/ITISFoundation/osparc-issues/issues/846
• https://github.com/ITISFoundation/osparc-issues/issues/864

  Security

  Bugfixes / Technical debt

• https://github.com/ITISFoundation/osparc-simcore/pull/3748
• https://github.com/ITISFoundation/osparc-simcore/pull/3750
• https://github.com/ITISFoundation/osparc-simcore/pull/3755
• https://github.com/ITISFoundation/osparc-simcore/pull/3757
• https://github.com/ITISFoundation/osparc-simcore/pull/3758
• https://github.com/ITISFoundation/osparc-simcore/pull/3759
• https://github.com/ITISFoundation/osparc-simcore/pull/3760
• https://github.com/ITISFoundation/osparc-simcore/pull/3761
• https://github.com/ITISFoundation/osparc-simcore/pull/3767
• https://github.com/ITISFoundation/osparc-simcore/pull/3768
• https://github.com/ITISFoundation/osparc-simcore/pull/3772
• https://github.com/ITISFoundation/osparc-simcore/pull/3775
• https://github.com/ITISFoundation/osparc-simcore/pull/3782
• https://github.com/ITISFoundation/osparc-simcore/pull/3785
• https://github.com/ITISFoundation/osparc-simcore/pull/3792
• https://github.com/ITISFoundation/osparc-simcore/pull/3793
• https://github.com/ITISFoundation/osparc-simcore/pull/3794
• https://github.com/ITISFoundation/osparc-simcore/pull/3803
• https://github.com/ITISFoundation/osparc-simcore/pull/3806
• https://github.com/ITISFoundation/osparc-simcore/pull/3807
• https://github.com/ITISFoundation/osparc-simcore/pull/3809
• https://github.com/ITISFoundation/osparc-simcore/pull/3811
• https://github.com/ITISFoundation/osparc-simcore/pull/3814
• https://github.com/ITISFoundation/osparc-simcore/pull/3815
• https://github.com/ITISFoundation/osparc-simcore/pull/3817
• https://github.com/ITISFoundation/osparc-simcore/pull/3821
• https://github.com/ITISFoundation/osparc-simcore/pull/3824
• https://github.com/ITISFoundation/osparc-simcore/pull/3826
• https://github.com/ITISFoundation/osparc-simcore/pull/3828
• https://github.com/ITISFoundation/osparc-simcore/pull/3831
• https://github.com/ITISFoundation/osparc-simcore/pull/3834
• https://github.com/ITISFoundation/osparc-simcore/pull/3835
• https://github.com/ITISFoundation/osparc-simcore/pull/3838
• https://github.com/ITISFoundation/osparc-simcore/pull/3839
• https://github.com/ITISFoundation/osparc-simcore/pull/3843
• https://github.com/ITISFoundation/osparc-simcore/pull/3845
• https://github.com/ITISFoundation/osparc-simcore/pull/3846
• https://github.com/ITISFoundation/osparc-simcore/pull/3850
• https://github.com/ITISFoundation/osparc-simcore/pull/3861
• https://github.com/ITISFoundation/osparc-simcore/pull/3868
• https://github.com/ITISFoundation/osparc-simcore/pull/3870
• https://github.com/ITISFoundation/osparc-simcore/pull/3871
• https://github.com/ITISFoundation/osparc-simcore/pull/3872
• https://github.com/ITISFoundation/osparc-simcore/pull/3873
• https://github.com/ITISFoundation/osparc-simcore/pull/3874
• https://github.com/ITISFoundation/osparc-simcore/pull/3879
• https://github.com/ITISFoundation/osparc-simcore/pull/3881
• https://github.com/ITISFoundation/osparc-simcore/pull/3882
• https://github.com/ITISFoundation/osparc-simcore/pull/3886

CI/CD

• https://github.com/ITISFoundation/osparc-simcore/pull/3751

  e2e testing

• https://github.com/ITISFoundation/osparc-simcore/pull/3762

  Pre/releases during this sprint

• https://github.com/ITISFoundation/osparc-issues/issues/822

• https://github.com/ITISFoundation/osparc-issues/issues/824

• https://github.com/ITISFoundation/osparc-issues/issues/828

• https://github.com/ITISFoundation/osparc-issues/issues/832

• https://github.com/ITISFoundation/osparc-issues/issues/838

• https://github.com/ITISFoundation/osparc-issues/issues/840

• https://github.com/ITISFoundation/osparc-issues/issues/841

• https://github.com/ITISFoundation/osparc-issues/issues/847

• https://github.com/ITISFoundation/osparc-issues/issues/848

• https://github.com/ITISFoundation/osparc-issues/issues/849

• https://github.com/ITISFoundation/osparc-issues/issues/851

• https://github.com/ITISFoundation/osparc-issues/issues/852

• https://github.com/ITISFoundation/osparc-issues/issues/854

• https://github.com/ITISFoundation/osparc-issues/issues/855

• https://github.com/ITISFoundation/osparc-issues/issues/857

• https://github.com/ITISFoundation/osparc-issues/issues/858

• https://github.com/ITISFoundation/osparc-issues/issues/860

• https://github.com/ITISFoundation/osparc-issues/issues/861

• https://github.com/ITISFoundation/osparc-issues/issues/865

• https://github.com/ITISFoundation/osparc-issues/issues/866

• https://github.com/ITISFoundation/osparc-issues/issues/867

• https://github.com/ITISFoundation/osparc-issues/issues/868

  DevOps

• 🚨 Ad-hoc: Enable 10 "hot" machines for the s4l-lite release https://git.speag.com/oSparc/osparc-ops-environments/-/issues/431

• 🚨 Z43 electricity surge aftermath: https://git.speag.com/oSparc/osparc-ops-environments/-/issues/399 , https://git.speag.com/oSparc/osparc-ops-environments/-/issues/425

• 🚨 Credential exposure alert aftermath: https://git.speag.com/oSparc/osparc-ops-environments/-/issues/410 https://git.speag.com/oSparc/osparc-ops-environments/-/issues/403

• 🔊 Add Graylog alert for director-v2 malfunction, simcore agent malfunction https://git.speag.com/oSparc/osparc-ops-environments/-/issues/439 , https://git.speag.com/oSparc/osparc-ops-environments/-/issues/428 , https://github.com/ITISFoundation/osparc-ops-environments/pull/104

• 🔊 Remove graylog mail alerts on all deployments in favor of mattermost alerts https://git.speag.com/oSparc/osparc-ops-environments/-/issues/422

• ✨ Feature: Different maintenance pages per deployment https://github.com/ITISFoundation/osparc-ops-environments/pull/93

• ✨ Create Statuspage for S4L-Lite https://git.speag.com/oSparc/osparc-ops-environments/-/issues/399

• ✨ Add e2e tests for s4l-lite alias domains https://git.speag.com/oSparc/osparc-ops-environments/-/issues/411

• ♻️✨ Enable testing. routes for osparc https://git.speag.com/oSparc/osparc-ops-environments/-/issues/429 / https://github.com/ITISFoundation/osparc-ops-environments/pull/101 https://github.com/ITISFoundation/osparc-ops-environments/pull/101

• ♻️ Database backup not working after restart https://git.speag.com/oSparc/osparc-ops-environments/-/issues/443 , https://github.com/ITISFoundation/osparc-ops-environments/pull/105

• ♻️ Improved logging/log-level on aws-prod https://git.speag.com/oSparc/osparc-ops-environments/-/issues/435

• ♻️ Change AWS s4l-lite machine instance type https://git.speag.com/oSparc/osparc-ops-environments/-/issues/437

• ♻️ Mailserver fixes & config for s4l-lite https://git.speag.com/oSparc/osparc-ops-environments/-/issues/412 , https://git.speag.com/oSparc/osparc-ops-environments/-/issues/409 , https://github.com/ITISFoundation/osparc-ops-environments/pull/103

• ♻️ Remove CEPH Prototype machines for repurposing https://git.speag.com/oSparc/osparc-ops-environments/-/issues/376

• 🐛 Fix git config on all deployments https://git.speag.com/oSparc/osparc-ops-environments/-/issues/423

• 🐛 Fix: Missing variable https://git.speag.com/oSparc/osparc-ops-environments/-/issues/407

• 🐛 Fix grafana dashboard bug https://git.speag.com/oSparc/osparc-ops-environments/-/issues/427

• 🐛 Fix: Wrong Product CORS https://github.com/ITISFoundation/osparc-ops-environments/pull/89

• 🐛 Fix: Bucket CORS script https://github.com/ITISFoundation/osparc-ops-environments/pull/91

• 🐛 Fix: wrong call_make https://github.com/ITISFoundation/osparc-ops-environments/pull/92

• 🐛 Fix: All docker node labels need to be lowercase https://github.com/ITISFoundation/osparc-ops-environments/pull/94

[sanderegg]

Goal for sprint Mithril

Waiting to decide which issues of the Maintenance panel get here.

• https://github.com/ITISFoundation/osparc-simcore/issues/3943
• https://github.com/ITISFoundation/osparc-issues/issues/877
• https://github.com/ITISFoundation/osparc-simcore/issues/3421

--

• https://github.com/ITISFoundation/osparc-simcore/issues/3891
• https://github.com/ITISFoundation/osparc-simcore/issues/3531
• https://github.com/ITISFoundation/osparc-simcore/issues/3832
• Remove openapi-core library from osparc-simcore, https://github.com/ITISFoundation/osparc-simcore/issues/3515

[mrnicegyu11]

Goal for sprint Mithril

Taken from https://git.speag.com/oSparc/osparc-ops-environments/-/issues?milestone_title=Mithril

• Certificates expiration : test in all stack
• PoC: connect a computer using separate ITIS account to osparc.io
• Increase necessary limits for AppTeam Std Simulations on S4L/AWS
• Find a way to know how many instances for a given type of instance is available on a subnet in aws
• s4l-lite monitoring dashboard
• Not able to connect to AWS monitoring
• Set SIMCORE_VCS_RELEASE_TAG and SIMCORE_VCS_RELEASE_DATE in agent_create_stack_script
• Rational about making aws prod more redudant and stable
• Create a test to check that the maintenance-page for TIP is up
• upgrade graylog to version 5?
• upgrade portainer
• Grafana Dashboard Display in the osparc office
• Mail Server - test aws solution
• Set latest Autoscaling service ENVs
• Credentials to access AWS costs
• Create admin user in all deploys
• Deployment agent: Massive Refactoring, bugFix, Tag Sync
• REGISTRY_SSL is hard-coded and does not fetch its value from the configuration for staging-aws
• branch used for staging AWS is not the latest name of the staging branch
• Add graylog alert for IP range exhaustion
• Compile S3 egress costs
• Update PULL_REQUEST_TEMPLATE message to improve communication with devops

[sanderegg]

Update on sprint Mithril

see https://github.com/orgs/ITISFoundation/projects/9/views/34

Releases

• https://github.com/ITISFoundation/osparc-simcore/issues/4023
• https://github.com/ITISFoundation/osparc-issues/issues/906
• https://github.com/ITISFoundation/osparc-issues/issues/896
• https://github.com/ITISFoundation/osparc-simcore/issues/4081
• https://github.com/ITISFoundation/osparc-simcore/issues/4002
• https://github.com/ITISFoundation/osparc-simcore/issues/4009
• https://github.com/ITISFoundation/osparc-simcore/issues/4020

  Bugfixes/Technical Debt

• https://github.com/ITISFoundation/osparc-simcore/issues/4012
• https://github.com/ITISFoundation/osparc-simcore/pull/3983
• https://github.com/ITISFoundation/osparc-simcore/pull/3703
• https://github.com/ITISFoundation/osparc-simcore/pull/3982
• https://github.com/ITISFoundation/osparc-simcore/pull/3985
• https://github.com/ITISFoundation/osparc-simcore/pull/4004
• https://github.com/ITISFoundation/osparc-simcore/pull/4005

  Enhancements

• https://github.com/ITISFoundation/osparc-simcore/pull/3978

  Dependencies

• https://github.com/ITISFoundation/osparc-simcore/pull/3949

  Ongoing

• https://github.com/ITISFoundation/osparc-issues/issues/877
• https://github.com/ITISFoundation/osparc-simcore/issues/3932
• https://github.com/ITISFoundation/osparc-simcore/issues/3931
• https://github.com/ITISFoundation/osparc-simcore/issues/3837
• https://github.com/ITISFoundation/osparc-simcore/issues/3943
• https://github.com/ITISFoundation/osparc-simcore/issues/3626
• https://github.com/ITISFoundation/osparc-simcore/issues/1580
• https://github.com/ITISFoundation/osparc-simcore/issues/2410

  Devops

  Full list: https://git.speag.com/oSparc/osparc-ops-environments/-/issues?milestone_title=Mithril&scope=all&state=closed

[pcrespov]

Goal for sprint Jelly Beans

Backend organized in https://github.com/orgs/ITISFoundation/projects/9/views/34

DevOps organized in https://git.speag.com/oSparc/osparc-ops-environments/-/issues?scope=all&state=opened&milestone_title=Jelly%20Beans

[sanderegg]

Update on sprint Jelly Beans

Development

• Details in https://github.com/orgs/ITISFoundation/projects/9/views/34.
• Highlights are:
  • Increased type checking and code coverage (~80%)
  • Upgrade to python 3.10 and 271 libraries: vulnerabilities, new features, bug fixes, and pruning (planning license tracing)
  • Multiple releases to staging and production
  • Still problems with CI (flaky tests) which is significantly delaying** the development workflow!
  • Improved logs monitoring: towards zero error-logs (@matusdrobuliak66 demo)
    • Operations: see https://git.speag.com/oSparc/osparc-ops-environments/-/issues?scope=all&state=closed&milestone_title=Jelly%20Beans

[pcrespov]

Goal for sprint Pastel de Nata

• Details in https://github.com/orgs/ITISFoundation/projects/9/views/34

[pcrespov]

Update for sprint Pastel de Nata

• releases
• mypy: https://github.com/ITISFoundation/osparc-simcore/issues/4071
• flaky: https://github.com/ITISFoundation/osparc-simcore/issues/3421
• upgrades dependencies
• SEE https://github.com/orgs/ITISFoundation/projects/9/views/34

[pcrespov]

Update Watermelon

Highlights

• 🔨 Tooling:
  • Almost finished with full-repo type-checking , i.e. mypy
  • Integrated ruff: an extremly fast python linter
• 🚀 Releases
  • 4x releases to staging
  • 2x releases to production + 1 hotfix
• ⬆️ Dependency upgrades (requirements.txt)
• 🏷️ New convention: Redefined services labels names to follow inversed url notation
• ✅ json-schemas for all service configurations as artifacts of CI -> can validate stack configuration before deploy!
• ♻️ Towards sqlalchemy 2.0

In detail

• maintenance view
• devops view

[sanderegg]

Update Sundae

Highlights:

• completed mypy type-checking on osparc-simcore repository, CI steps checks for all python-based packages
• improved operations tooling for releases

In details:

• Maintenance view

[sanderegg]

Update Baklava

In details:

• Maintenance view

[elisabettai]

@sanderegg, @pcrespov, @matusdrobuliak66. I am closing this one. For Y7 (simcore)-related maintenance, please use this new one #1108

@mrnicegyu11, @YuryHrytsuk: for operations, we have a new issue in Y7 as well: #1109. I guess we can move the tasks still open here in that new one. I can do that, if that's useful.

@sanderegg, @pcrespov, @matusdrobuliak66, @mrnicegyu11, @YuryHrytsuk. In Y7, we also have a series of new cases, which are a bit of mixture of comp. backend, operations, monitoring, etc... Those should cover the work we're doing for the release. Those new cases are:

1100

1101

1102

Maybe I will try to link the new ones with the work packages we already have. The goal is not to add more work/useless issues, only making reporting on those new milestones easier. 😉