Convert OTAR EFO OWL to JSON

[d0choa]

As part of removing data transformations from PIS @javfg removed the OWL -> JSON conversion that was currently performed by Apache Jena.

We will need to find an alternative method to produce this transformation. As the ideal solution we are currently exploring whether the EFO team could create a JSON for the OTAR slim in the same way they make a JSON for the official release.

Action can be followed here: https://github.com/EBISPOT/efo/issues/2282

If this option does not fly we will need an alternative

[javfg]

I looked into this when I removed Jena, I'll write some thoughts on it soon.

[prashantuniyal02]

Hi @javfg , Just to keep something in mind that since the request to create the JSON file was very close to EFO's September release, the JSON file was not be tested thoroughly by them. The conversion they used was simply the convert command from ROBOT

In the upcoming October release, it will be tested thoroughly.

[javfg]

I've created https://github.com/opentargets/ontoform for this purpose.

It's based on @jdhayhurst idea of using Polars for POS conversion of parquet to JSON. It is extremely fast (about 2 seconds) for EFO, and should be as good for the rest of ontologies. Ensembl processing may take a bit longer but it should be manageable.

We can extend that application easily by just adding new ontologies and the CLI will pick them up automatically. What remains is to decide where to run that:

• As a stand-alone task between PIS and ETL (we may be able to use cloud run for this)
• As a script before we run the actual ETL, in the same machines
• We could also create a custom PIS task that ran this (this doesn't make a lot of sense)

[javfg]

I'll close this issue, as the conversion is done. We can track the part about running it in https://github.com/opentargets/issues/issues/3496.