Not a solution to the "cancel a single cell" problem

[GermanCM]

The solution provided by the team for the issue of not being able to execute other cells after cancelling out another cell (and having this one "frozen" for a long time) is not a solution good enough, since clearing the state of a notebook means, not few times, loosing lots of processed data stored in memory. This issue seems to happen when executing code on a koalas or pandas_on_spark dataframe, more often than on pandas dataframes for instance.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: ec2c60b2-01d2-f14d-827d-7f9f36f8a899
• Version Independent ID: ff894427-4deb-2200-3907-7fe1de14f715
• Content: Cannot run notebook commands after canceling streaming cell - Azure Databricks
• Content Source: databricks/kb/notebooks/streaming-notebook-stuck.md
• Service: azure-databricks
• GitHub Login: @mssaperla
• Microsoft Alias: saperla

[ShaikMaheer-MSFT]

@GermanCM Thank you for sharing feedback. We are actively validating it. We are share updates soon.

[ShaikMaheer-MSFT]

@mssaperla Could you please validate customer feedback and enhance document as appropriate? Thank you.

[GermanCM]

Any new on it?

[GermanCM]

As far as I could check today, it seems that many problems are solved when using the input parameter _indexcol when converting from spark to pandas_on_spark, including the issue explianed here of cancelling a cell running on such pandas_on_spark dataframes.