search

opentargets / issues

Issue tracker for Open Targets Platform and Open Targets Genetics Portal
https://platform.opentargets.org https://genetics.opentargets.org
Apache License 2.0
12 stars 2 forks source link

Update Dataproc and PySpark in Genetics #3189

Open ireneisdoomed opened 5 months ago

ireneisdoomed commented 5 months ago

Dataproc's image 2.2 has been public for one month and it comes with PySpark 3.5. They have important features like support for all SQL functions and a basic testing suite.

This is a nice to have.

ireneisdoomed commented 5 months ago

Currently blocked. Hail does not support PySpark>3.4.