search

databricks-demos / dbdemos

Demos to implement your Databricks Lakehouse
Other
255 stars 80 forks source link

02-Data-preparation note cell "Extract the dataset using sh command" FAILS #33

Closed Free-Radical closed 1 year ago

Free-Radical commented 1 year ago

Usign databricks dbdemos-lakehouse-c360 follwing error from cell:

rm: cannot remove '/dbfs/dbdemos/product/llm/gardening/raw': No such file or directory

shell script is not in the expected directory (/tmp/gardening) when executing

cp -f Posts.xml /dbfs/dbdemos/product/llm/gardening/raw

I fixed it by adding new cell immediately after with following commands:

%sh pwd mkdir -p /dbfs/dbdemos/product/llm/gardening/raw cp -f /tmp/gardening/Posts.xml /dbfs/dbdemos/product/llm/gardening/raw

QuentinAmbard commented 1 year ago

Hi, thanks for letting us know, I'm adding this instead, it should ignore the error:

rm -rf /dbfs/dbdemos/product/llm/gardening/raw || true

Will be in the next release ~toay