Change hashing strategy for StudyLocusId generation in StudyLocus object

[project-defiant]

As a developer I want change the hashing strategy to md5 when calculating StudyLocusId because current implementation requires casting big integer to a string to be ingested by the platform.

Background

This task is required due to the fact we do not create proper hashes within StudyLocusId field of StudyLocus object.

Tasks

• [x] Fix column names that are required to generate the hash (studyId, variantId and finemappingMethod).
• [x] Change schema of the StudyLocus object to have StudyLocusId as a StringType.
• [x] Change hashing strategy.
• [ ] Recreate hashes of all datasource credible_set datasets.

Acceptance tests

• [ ] When all of staging bucket credilble_sets have md5 hashes recalculated
• [ ] StudyLocusId field should be stringType

[project-defiant]

@DSuveges FYI

[d0choa]

Reassigning to @vivienho based on discussions with @DSuveges

@DSuveges the ticket might not have the right context, so @vivienho might need some extra guidelines

The change in the hashes will break all the data compatibility, so @project-defiant, you might want to keep an eye on it

Related to https://github.com/opentargets/issues/issues/3535

[project-defiant]

@d0choa we should not merge this unless we are sure we can reparse all indices for credible sets, or recalculate entire study_locus

[DSuveges]

I think we'll re-calculate the entire dataset a few time till the release.

[d0choa]

It's never a good time to make this change. But it will get a lot worse soon