search

spring-cloud / spring-cloud-dataflow

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
https://dataflow.spring.io
Apache License 2.0
1.11k stars 583 forks source link

Convert JdbcDataflowTaskExecutionMetadataDao to use Spring Data repositories #3560

Open mminella opened 5 years ago

mminella commented 5 years ago

To get an MVP out quickly, the JdbcDataflowTaskExecutionMetadataDao class was implemented using JDBC. However, all the other DAOs in Spring Cloud Data Flow have been converted to Spring Data JPA. This DAO should be migrated as well.

siddhantsorann commented 3 years ago

@sabbyanandan @mminella I'd like to pick this one up if it's still up for grabs.

sabbyanandan commented 3 years ago

@siddhantsorann: That'd be a lofty goal, but feel free to give it a go. 😄

siddhantsorann commented 3 years ago

@sabbyanandan: Haha let me have a look then. Can you also refer me to any other class which was migrated to Spring Data JPA?

siddhantsorann commented 3 years ago

From what I gather we will have to create an entity as well for TaskExecutionMetadata and implement a JPA Repo on that?

mminella commented 3 years ago

The challenge is that most of the related entities are not JPA mapped. Nothing that SCDF gets from Spring Cloud Task is for example.

siddhantsorann commented 3 years ago

Right, I can see that Task_execution_metadata has a reference to task_execution which is also not a JPA Entity. How would you suggest we go about this in that case?

siddhantsorann commented 3 years ago

@mminella So the entity task_execution_metadata will be the first entity from scdf to be migrated to JPA from DAO? I can start building an entity and repo for it in that case.

mminella commented 3 years ago

It's been my experience that doing half JPA and half traditional JDBC causes data consistency issues between the two solutions (given caching/etc) so it's best to do an all or nothing approach which is where the complexity of this issue comes from.

siddhantsorann commented 3 years ago

@mminella That definitely makes sense. I would also suggest that we should look at moving all entities to JPA. I don't mind having a discussion and planning it out. Would still be eager to pick up such a migration.