Data related tasks for 23.09 Platform release

[DSuveges]

This is a collection of the data team's tasks for the upcoming 2023 September release. For more context on the priorities of the planned features/updates, see release intention document here. Most data team related issues are in the should/could/won't.

Timeline: The proposed ETL run is expected on the 21th of August. By that time, all data team tasks up to running the parsers, should be completed.

New feature, updates, fixes:

• [x] We might need to explore the observed bug with known drugs (ChEMBL) as the new ChEMBL release is out. This might be a non-issue as we are approaching to the release date (#3005). @ireneisdoomed
• [X] Update stop reasons predictions for ChEMBL submission. @ireneisdoomed
• [X] Update chemical probes data to follow Probes&Drugs 01.2023 release. @ireneisdoomed
• [X] Add a publication reference to evidence from Genebass and Finngen (#2739). @ireneisdoomed
• [X] Change the description of the direction of effect for target safety events (#2921)
• [x] Scope and refactor the baseline expression [@buniello /Data+BE+FE]. Considering how much downstream dependencies this task has, this should be completed soon #3024 . @tskir @buniello Dropped from the 23.09 release following the decision to iterate more by @buniello and @d0choa
• [x] There's a potential bug reported on our interaction widget, as we are not reporting the correct STRING interaction score #3026 @DSuveges
• [x] New data from Validation Lab in PPP [Data+FE] #2990 We had a meeting with the Validation lab last week, the data is expected to arrive in a few weeks. Given the data, this update has high priority. No schema update is expected. @DSuveges @ehsanb29 Dropped from 23.09 release intentions.
• [x] Association from a new gene burden study could be curated and added to our gene burden dataset. No schema update is expected, however curation repo needs to be updated and released (no ticket yet). @ireneisdoomed I decided not to prioritise this. The study is based on a smaller cohort than what we currently have. It is a systematic analysis, so there were many diseases to map, and the study was anyway not that recent.
• [x] Moving public CRISPR screen from PPP to public. This task has a trivial, one off solution and a very complex general solution. We need to see what we have time for. No schema change is expected. #2991 @ehsanb29 Dropped from 23.09 release intentions.
• [x] New Project score evidence for PPP #3027. This task might be not too complicated. The new evidence model should either follow the old schema or put something together from the existing fields. So there MIGHT be updates required from the frontend. @ehsanb29 @DSuveges Dropped from 23.09

Standard procedures:

• [x] Release schema repository if updates were implemented. @tskir
• [x] Release curation repository if updates were implemented. @tskir
• [x] Update the evidence parser repository with eg. EFO, schema/curation version. @tskir
• [x] Run evidence parsers pipelines. @tskir
• [x] Release the evidence parser repository. @tskir
• [x] Check that all external evidence is submitted by the end of the submission window (11 Aug). @tskir
• [x] Run pre-ETL metrics. @tskir
• [x] Run post-ETL metrics. @tskir

[tskir]

Hi @DSuveges, there are three tickets in the description (#2990, #3026, #3027) which are listed as being either partially or fully assigned to you. Given that I am being responsible for this release from the data team side, I just wanted to double check that the assignment is current and you'll continue to be responsible for those tickets? Or shall I take them over?

[tskir]

Status of standard procedures

• Schema repository release: blocked by the baseline expression schema PR https://github.com/opentargets/json_schema/pull/172 which needs to be reviewed and merged
• Curation repository release: no updates since last release. However, for consistency, I've created a 23.09 tag pointing to the same commit as the 23.06 release.
• Evidence parser release: blocked by schema repository release, see above
• Evidence generation: in progress, currently running

[tskir]

Data delivered, code updated & all necessary metrics done. The issue is complete.