Bulk Upload- Deduplication Service

[manukalia29]

What is the feature request for?

Registry Core

What problem/inconvenience (if any) will this feature solve?

Not having de duplication service causes data inaccuracies, reduced efficiency (manual verification), poor user experience,

Describe the feature clearly.

De-duplication service will help to identify and remove duplicate entries from a large dataset before it is uploaded or imported into a system. The de-duplication service can use algorithm to identify potential duplicates in the dataset uploaded based on specific criteria, such as names, addresses, or unique identifiers. Once the potential duplicates are identified, the deduplication service can flag them for manual review.

Needs discussion --Where we will configure the attributes that need to check the duplicates? and messages to be shown

[jayadinni]

Please also enable unique identifier in the form builder:

[Renuka-S]

• [ ] @tejash-jl @varadeth Create a new schema in education with unique identifier to test the feature. Start Date: 2nd June
• [ ] @Pritzz4 Can confirm the duplication workflow after the setup to close the issue