Creating large sample data (>250 patients) for generated Cohort Definition fails on Databricks

[guybartal]

Expected behavior

Creating large sample data (>250 patients) for generated Cohort Definition on Databricks should work like other dialects does (MS SQL and Azure Synapse).

Actual behavior

example for smaller population sample which works:

No logged errors spotted on WebAPI and no failed jobs on Databricks side.

Steps to reproduce behavior

• WebAPI version 2.12.1
• Databricks Cluster version: 12.2 LTS (includes Apache Spark 3.3.2, Scala 2.12)
• Databricks JDBC spark drivers version 2.6.22
• Define Cohort Definition
• Generate Cohort on Databricks
• Create new sample with 500 patients on Databricks
• See popup failure