search

kubeflow / pipelines

Machine Learning Pipelines for Kubeflow
https://www.kubeflow.org/docs/components/pipelines/
Apache License 2.0
3.5k stars 1.58k forks source link

Tracking: PostgreSQL support in KFP #9813

Open zijianjoy opened 11 months ago

zijianjoy commented 11 months ago

PostgreSQL request has become the top upvoted issue on KFP repo: https://github.com/kubeflow/pipelines/issues/7512. This issue is for tracking the work of this integration.

zijianjoy commented 11 months ago

cc @chensun

github-actions[bot] commented 7 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

EshaanAgg commented 4 months ago

Hi! I am Eshaan Aggarwal, an avid Open Source enthusiast from India. I am a web developer proficient in GoLang and PostgreSQL and have recently started learning about Kubernetes. I would love to contribute to this issue and hopefully join Kubeflow as a GSoC '24 mentee. Are there any pre-tests or other beginner-friendly contributions I can make to get acquainted with this project and as a proof of skill?

UditNayak commented 4 months ago

Hello, I'm Udit. Proficient in Python, Golang, and PostgreSQL, I've recently completed a comprehensive course on Kubernetes. The skills acquired perfectly align with the requirements of this project, making it an ideal platform for me to apply and further enhance my knowledge. I'm enthusiastic about contributing to this GSoC project, especially on the specified issue. Eagerly anticipating the chance to contribute to its development!

zijianjoy commented 4 months ago

Hello @EshaanAgg and @UditNayak , thank you for your interest and I am assuming @rimolive will be your mentor.

As a start, I would recommend learning:

Then, my advice to the development will be in following orders:

  1. Make sure you can bring up KFP in the kubernetes environment
  2. Make sure you can bring up a postgresql instance and access to it manually in the kubernetes environment
  3. Make changes in KFP API server so it can read postgresql connection config from parameter/envionrment-variable. Then KFP API server should establish connection with the postgresql instance in the same cluster with such connection config.
  4. Make corresponding GORM change so that CRUD (create/read/update/delete) operation of KFP can be executed correctly using postgresql. (It is a good time to write some unit test or E2E test)
  5. Perform the similar actions as above for cache server.

I believe @rimolive can facilitate more once you dive deep into the project. But feel free to take any task you want to work on and ask questions along the way. Have fun!

rimolive commented 3 months ago

In addition to what @zijianjoy said, please join us on our Slack. We have the #gsoc-participants channel to welcome everyone interested in the GSoC projects.

VDliveson commented 3 months ago

@zijianjoy I am a web developer from India, and I would like to work on this issue. I know python,golang and a bit of kubernetes. How to get started on this

Irshu786 commented 3 months ago

@zijianjoy Im currently doing ops in kubeflow, loved the concept of changing db, Will be working on these.

Make sure you can bring up KFP in the kubernetes environment

Make sure you can bring up a postgresql instance and access to it manually in the kubernetes environment

Make changes in KFP API server so it can read postgresql connection config from parameter/envionrment-variable. Then KFP API #server should establish connection with the postgresql instance in the same cluster with such connection config.

Make corresponding GORM change so that CRUD (create/read/update/delete) operation of KFP can be executed correctly using postgresql. (It is a good time to write some unit test or E2E test)

Perform the similar actions as above for cache server.

SnehaAgg0212 commented 3 months ago

@zijianjoy I am a BE developer from India and am comfortable with python, goLang, java, Kubernetes and SQL databases. I would like to contribute to this issue. What are the steps?

jiduyuting commented 3 months ago

hello@zijianjoy I'm a junior studen from China major in computer science and technology, I'm very interested in open source project and want to do some contribution to this project, at the same time to improve my skill. I have joined our school lab which connect with database and cloud. So I frequently contact with postgresql and kubernetes. I'm doing the steps you refered above,but I can't join in the #gsoc-participants,can this have influence?

Irshu786 commented 3 months ago

Hey Guys,

Everyone has setup local kubeflow setup ?

On Mon, Mar 25, 2024 at 8:50 AM Jiduyuting @.***> wrote:

@.*** I'm a junior studen from China major in computer science and technology, I'm very interested in open source project and want to do some contribution to this project, at the same time to improve my skill. I have joined our school lab which connect with database and cloud. So I frequently contact with postgresql and kubernetes. I'm doing the steps you refered above,but I can't join in the #gsoc-participants,can this have influence?

— Reply to this email directly, view it on GitHub https://github.com/kubeflow/pipelines/issues/9813#issuecomment-2017138982, or unsubscribe https://github.com/notifications/unsubscribe-auth/AXRWTPO2DUQDZHVE6JBLTCLYZ6JZVAVCNFSM6AAAAAA3DJ3CPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJXGEZTQOJYGI . You are receiving this because you commented.Message ID: @.***>

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

rimolive commented 1 month ago

/lifecycle frozen

sagnik3788 commented 1 week ago

@zijianjoy anyone working on this issue ?

rimolive commented 1 week ago

@sagnik3788 This is part of the Google Summer of Code. You can find details in https://www.kubeflow.org/events/gsoc-2024/#project-9-postgresql-integration-in-kubeflow-pipelines.