search

Azure / azure-sdk-for-java

This repository is for active development of the Azure SDK for Java. For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/java/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-java.
MIT License
2.34k stars 1.98k forks source link

Databricks cosmos plugin not working #37994

Open SomanathSankaran opened 10 months ago

SomanathSankaran commented 10 months ago

Tried to execute cosmos metrics plugin enablement scripts

!/bin/bash

STAGE_DIR="/dbfs/FileStore/plugins"

echo "BEGIN: Upload Spark Plugins" cp -f $STAGE_DIR/*.jar /databricks/jars || { echo "Error copying Spark Plugin library file"; exit 1;} echo "END: Upload Spark Plugin JARs"

Error "databricks_error_message": "Spark error: Driver down cause: driver state change (exit code: 10)"

Spark encountered an error on startup. This issue can be caused by invalid Spark configurations or malfunctioning init scripts. Please refer to the Spark driver logs to troubleshoot this issue, and contact Databricks if the problem persists.

Internal error message: Spark error: Driver down cause: driver state change (exit code: 10)

kushagraThapar commented 9 months ago

@xinlian12 can you please take a look at this, thanks!

ShasTheMass commented 7 months ago

Hey guys, anything on this, I am getting a very similar error with 11.3 (spark 3.3.0). Are there any recommended configs e.g. adequate amount of memory and type of node (e.g. AWS EC2 i3.xlarge)?

kushagraThapar commented 7 months ago

@tvaron3 / @xinlian12 please take a look at this, thanks!