search

databricks / cli

Databricks CLI
Other
132 stars 50 forks source link

Add an error if state files grow bigger than the export limit #1795

Open andrewnester opened 3 days ago

andrewnester commented 3 days ago

Changes

Currently API limits on exporting files from workspaces are set at 10 MBs while uploading to is 500 MBs. We want to prevent users running into deadlock when they won't be able to pull state file anymore so we prevent from uploading large state files (over 10 MBs) to Databricks workspace.

andrewnester commented 4 hours ago

@pietern I've added a unit test but also confirmed it manually