[DMP 2024]: CI-CD pipeline for AMRIT

[drtechie]

Ticket Contents

Description

Digital Bharat Collaborative (DBC) is an initiative of Piramal Swasthya, the healthcare arm of the Piramal Foundation. DBC's focus areas include building a robust digital public health delivery platform, improving data-driven governance and empowering healthcare workers. One of DBC's most successful initiatives is AMRIT (Accessible Medical Records via Integrated Technologies). AMRIT is a first-of-its-kind integrated health technology platform that supports clinical excellence for over 2 Cr beneficiaries. AMRIT is deployed across multiple on-premise servers and state data centres in India.

Building and deploying the latest code involves a lot of manual effort. The fact that AMRIT is deployed across several servers adds additional complexity in managing releases. Currently, several mission-critical services and helplines like 104 and 1097 take a downtime while deploying manually. CI-CD will ensure minimal downtime, allow us to release often and maintain a stable service to our thousands of health workers.

This project aims to integrate a robust CI-CD pipeline to improve development efficiency and reduce deployment risks.

Goals & Mid-Point Milestone

Goals

• [ ] Research and selection of CI-CD tools and understanding of current AMRIT deployments
• [ ] Tweaking of existing GitHub Actions to include triggers for unit testing and static code analysis
• [ ] Creation of Jenkins pipeline to build war files for deployment
• [ ] Add database schema migrations to the AMRIT platform
• [ ] [Goals Achieved By Mid-point Milestone are above]
• [ ] Handling remote database schema migrations via the deployment pipeline
• [ ] Deployment of war files to WildFly server in the development environment
• [ ] Promotion/replication of CI-CD workflow to higher environments

Setup/Installation

https://psmri.github.io/PSMRI/developer-guides/technical-overview/

Expected Outcome

• Comprehensive documentation covering the chosen CI/CD tools, pipeline setup, configuration details, and usage instructions.
• A few clicks in Jenkins must build and deploy the latest code to selected data centres.
• CI-CD pipeline(s) that care of code testing, building and deployment to data centres.

Acceptance Criteria

• Any sensitive information like DB passwords and API keys must be managed securely via the pipeline.
• The pipeline must automate code testing, building, and deployment processes.
• Migrations of database schema are taken care of in the pipeline.
• The deployment pipeline must require near-zero manual intervention.

Implementation Details

• You will receive access to a Jenkins server with a pipeline for configuring further.
• Existing GitHub Actions can be tweaked further for code testing.
• The services are hosted on WildFly servers. You will receive a test environment to try out the pipelines.

Mockups/Wireframes

NA

Product Name

AMRIT

Organisation Name

Piramal Swasthya Management Research Institute

Domain

⁠Healthcare

Tech Skills Needed

DevOps, Jenkins, SQL

Mentor(s)

@drtechie

Category

Backend, CI/CD, Database, Documentation, Deployment

[rohitkori]

Hey @drtechie, I am interested in working on this ticket. I have prior knowledge of DevOps tools like Docker, Kubernetes, GitHub workflows and Jenkins. Can you please guide me on how can I proceed with this project under DMP 2024?

[drtechie]

@rohitkori that's good to know. Appreciate your interest in this issue.

Please follow this link to register for the DMP 2024 program. You'll need to submit the design proposal on Unstop to apply for the project and contribute.

If you have any specific questions or doubts about the project, feel free to ask.

Thanks.

[maheshkasabe]

Hello @drtechie, I am intrested in this project, i have a good amount of experience to work on this project since I am and was a part of Google Summer of Code Program in 2023 and I am a mentor for the same organization in 2024 GSoC Program, in GSoC 2023 i worked on a similar project where i had to create a CI/CD pipeline for Graphspace Project i succefully completed it and similarly i was also an intern in Linux Foundation Mentorship Program 2023 where I worked on similar project! here is my linkedin : https://www.linkedin.com/in/maheshkasbe/

[manuprati]

Hi, I am interested in the project. I have knowledge of all the technologies required for the project viz. CI/CD, Python, Java, SQL, Oracle 9i, Docker, Kubernetes, GitHub workflows and Jenkins. As I am keen to learn new things and once understand, I will be able to expertise the same. If you feel me fit kindly guide me through the process. Regards.

[drtechie]

@manuprati @maheshkasabe

Please follow this link to register for the DMP 2024 program. You'll need to submit the design proposal on Unstop to apply for the project and contribute.

If you have any specific questions or doubts about the project, feel free to ask here or message me on Discord (ID: drtechie).

Thanks.

[Pankajjsuthar]

Hi, I am highly interested in the project, I have prior knowledge about CI/CD pipelines and will be happy to develop more with the project.

[nitishmalang]

Hello @drtechie I am interested working in this issue, I have also done gsoc 2023 and got selected for 1st round of summer of bitcoin 2024 i am interested in working on this issue I have already worked with ci/cd pipelines

[AbhimanyuSamagra]

Do not ask process related questions about how to apply and who to contact in the above ticket. The only questions allowed are about technical aspects of the project itself. If you want help with the process, you can refer instructions listed on Unstop and any further queries can be taken up on our Discord channel titled DMP queries. Here's a Video Tutorial on how to submit a proposal for a project.

[Kunalhanma]

Hi, I have keen interest in this project and as per required skills denoted for project I can manage database and Jenkins. Please consider me as a contributer it will be mean alot

[khandolly786]

Hey @drtechie I want to show my highly interest in this program as I belong from the IT background and I also have the high knowledge of Jenkins and CI/CD pipeline in my last semester seminar project also... I'm genuinely excited and looking forward for my further opportunities in this arena. Through this program I can be my best version and put my best in this and all... Best, Darakhshan...

[drtechie]

Weekly Goals

Week 1

• [x] Access all relevant servers - Windows server via RDP, Jenkins server credentials and VPN
• [x] Create a new project pipeline in Jenkins to deploy to the Windows test server
• [x] Build war files for all service lines using JDK_17 and the latest Node

Week 2

• [x] Build war files with correct configurations for the HWC service line
• [ ] Deploy manually on the Windows Test server and confirm that the HWC service line is operational

Week 3

• [x] Build war files for ALL service lines with working configuration as tested by deploying to WildFly
• [x] Deploy war files remotely via the administration API of WildFly

Week 4

• [ ] Identify base DB schema and prepare a new Spring Boot project for migrations
• [ ] Add Flyway and create a base DB schema

Week 5

• [ ] Create migrations that will run incrementally on top of the base DB schema
• [ ] Publish the new Spring Boot application to DB GitHub repo
• [ ] Incorporate DB commands in the Jenkins pipeline to run on the WildFly server

Week 5

• [x] Create Jenkins pipelines for Dev and UAT environment
• [ ] Run and deploy the latest main branches to Dev and UAT environment

Week 6

• [ ] Create Jenkins pipelines for 2 other production environments
• [ ] Deployment and testing of the production environments

Week 7

• [ ] Review GitHub Actions - add triggers for unit testing in PRs
• [ ] Review GitHub Actions - add static code analysis for API
• [ ] Review GitHub Actions - add static code analysis for UI

[anandamideShakyan]

Weekly Learnings & Updates

Week 1

• Read about WildFly servers, including how to create management users.
• Began exploring various code repositories.
• Started working on Jenkins pipelines.
• Upgraded to JDK 17 and Node 20.

Week 2

• Investigated different environment variables for each repository and added them to Jenkins jobs.
• Built each repository using JDK 17 and Node 20.
• Troubleshot minor build issues in repositories.
• Reported repositories with compatibility issues that failed the build process to the development team.
• Created a Jenkins job for the HWC service line, which is in progress due to repository build failures.
• Continued troubleshooting issues with WildFly management user creation.

Week 3

• Continued troubleshooting repositories with minor build issues and raised two pull requests.
• Gained a comprehensive understanding of the WildFly deployment process from the development team.
• Resolved issues in the HWC service line, ensuring all repositories are now building correctly.
• Successfully resolved the WildFly management user creation issue.

Week 4

• Identified issues with the manual deployment process and reported them.
• Started reading about the WildFly Management API.
• Collaborated with the development team to identify errors and began working on fixes.
• Raised four pull requests to address errors in the manual deployment process.

Week 5

• Encountered a forbidden error when attempting to access manually deployed artifacts.
• Discussed the issue with the development team and identified recent changes in the database schema as the root cause.
• Began working on the continuous deployment process.

Week 6

• Installed JBoss CLI on the local machine and explored its usability for continuous deployment of artifacts.
• Resolved a connection timeout error in the JBoss CLI.
• Read about wildfly http management api and used it in the jenkins job
• Successfully deployed artifacts from the /target repository to the WildFly server through a continuous deployment Jenkins job.