How to get it to work with Databricks notebook importing?

[jlu-barracuda]

In Databricks, notebooks are imported using %run notebook-im-importing, but this tool specifically expects there to be a .ipynb extension after the notebook name. How can I get around this?

[MarcoGorelli]

Hi @jlu-barracuda - I don't use databricks so I don't really understand the question, could you elaborate please?

[jlu-barracuda]

Thanks for the quick response. To give you the background, Databricks has direct git integration that checks in notebooks as python files that contain python-ish markers that allows readability of a notebook in a git repo, and when they are pulled from the repo in Databricks, they get automatically converted back to Databricks notebooks. The python files look something like:

# Databricks notebook source
# This is the first cell

import x from y

# COMMAND ----------
# This is a new cell

# MAGIC %run another_nb

# COMMAND ----------
# This is a third cell

var_1 = function_from_another_nb(1000)

I'm trying to add a Github action that lints the files. When I pull the repo via a CLI, as expected, they stay python files with the above format. I found a tool that converts the above to .ipynb that works perfectly. However, the only remaining issue is that the syntax for importing functionalities from another notebook in jupyter or ipython notebooks (which nbQa works beautifully on) is %run another_nb.ipynb. So when I use nbQA on the .ipynb version of the above file, I get F821 undefined name 'function_from_another_nb'. Is there a way to get nbQA to see %run another_nb in a file and interpret that as %run another_nb.ipynb?

[MarcoGorelli]

So when I use nbQA on the .ipynb version of the above file

which command are you running?

[jlu-barracuda]

nbqa flake8

[MarcoGorelli]

I suggest you # noqa it, if flake8 doesn't support this then there's not much we can do here

thanks anyway for your report!