search

OHDSI / WhiteRabbit

WhiteRabbit is a small application that can be used to analyse the structure and contents of a database as preparation for designing an ETL. It comes with RabbitInAHat, an application for interactive design of an ETL to the OMOP Common Data Model with the help of the the scan report generated by White Rabbit.
http://ohdsi.github.io/WhiteRabbit
Apache License 2.0
185 stars 90 forks source link

Add Spark support #334

Open alondhe opened 2 years ago

alondhe commented 2 years ago

Is your feature request related to a problem? Please describe. Recently, we've added Spark SQL support to SqlRender and a few OHDSI tools. We'd like to do the same for WhiteRabbit.

Describe the solution you'd like My team will raise a PR to add Spark, with the necessary JDBC connection string format, as well as any SQL translations needed.

Describe alternatives you've considered We don't have any other database platforms, so we'd need this added in order to use the tool.

Additional context None

zwelshman commented 2 years ago

@alondhe I'm particularly interested in a SparkSQL/PySpark implementation. Is there anything I can help with?

natb1 commented 2 years ago

Something like this would be needed to enable https://github.com/OHDSI/Perseus/issues/11 I will try to get more information via teams. @zwelshman depending on the existing progress, is this something you would be interested in working together on?