[BUG] GPU Nightly builds failing

[miguelgfierro]

Description

tests/unit/examples/test_notebooks_gpu.py .                              [ 50%]
tests/smoke/examples/test_notebooks_gpu.py F                             [100%]

=================================== FAILURES ===================================
________________________________ test_npa_smoke ________________________________

notebooks = ***'als_deep_dive': '/mnt/azureml/cr/j/8e37d7f372a94e03bf052e14ca5b0be0/exe/wd/examples/02_model_collaborative_filtering...rk_movielens': '/mnt/azureml/cr/j/8e37d7f372a94e03bf052e14ca5b0be0/exe/wd/examples/06_benchmarks/movielens.ipynb', ...***
output_notebook = 'output.ipynb', kernel_name = 'python3'

    @pytest.mark.notebooks
    @pytest.mark.gpu
    def test_npa_smoke(notebooks, output_notebook, kernel_name):
        notebook_path = notebooks["npa_quickstart"]
>       pm.execute_notebook(
            notebook_path,
            output_notebook,
            kernel_name=kernel_name,
            parameters=dict(epochs=1, seed=42, MIND_type="demo"),
        )

tests/smoke/examples/test_notebooks_gpu.py:190: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/azureml-envs/azureml_65a72d7faa653dcf6da20162d218b48c/lib/python3.9/site-packages/papermill/execute.py:94: in execute_notebook
    parameter_predefined = _infer_parameters(nb)
/azureml-envs/azureml_65a72d7faa653dcf6da20162d218b48c/lib/python3.9/site-packages/papermill/inspection.py:42: in _infer_parameters
    language = nb.metadata.kernelspec.language
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = ***'display_name': "Python 3.7.11 64-bit ('tf2': conda)", 'name': 'python3'***
key = 'language'

    def __getattr__(self, key):
        """Get an attr by calling :meth:`dict.__getitem__`.

        Like :meth:`__setattr__`, this method converts :exc:`KeyError` to
        :exc:`AttributeError`.

        Examples
        --------
        >>> s = Struct(a=10)
        >>> s.a
        10
        >>> type(s.get)
        <... 'builtin_function_or_method'>
        >>> try:
        ...     s.b
        ... except AttributeError:
        ...     print("I don't have that key")
        ...
        I don't have that key
        """
        try:
            result = self[key]
        except KeyError:
>           raise AttributeError(key) from None
E           AttributeError: language

/azureml-envs/azureml_65a72d7faa653dcf6da20162d218b48c/lib/python3.9/site-packages/nbformat/_struct.py:125: AttributeError
----------------------------- Captured stdout call -----------------------------
INFO:papermill:Input Notebook:  /mnt/azureml/cr/j/8e37d7f372a94e03bf052e14ca5b0be0/exe/wd/examples/00_quick_start/npa_MIND.ipynb
INFO:papermill:Output Notebook: output.ipynb
------------------------------ Captured log call -------------------------------
INFO     papermill:execute.py:84 Input Notebook:  /mnt/azureml/cr/j/8e37d7f372a94e03bf052e14ca5b0be0/exe/wd/examples/00_quick_start/npa_MIND.ipynb
INFO     papermill:execute.py:85 Output Notebook: output.ipynb
============================== slowest durations ===============================
1.26s call     tests/unit/examples/test_notebooks_gpu.py::test_gpu_vm
0.03s call     tests/smoke/examples/test_notebooks_gpu.py::test_npa_smoke

In which platform does it happen?

How do we replicate the issue?

See: https://github.com/recommenders-team/recommenders/actions/runs/6884390439/job/18726805379

Expected behavior (i.e. solution)

Other Comments

[loomlike]

I'm working on this, along w/ notebook fixes for #2035

[SimonYansenZhao]

Is this issue fixed? @loomlike @miguelgfierro

[miguelgfierro]

@SimonYansenZhao staging is passing, running main right now:

• CPU: https://github.com/recommenders-team/recommenders/actions/runs/7307018860
• GPU: https://github.com/recommenders-team/recommenders/actions/runs/7307015824
• Spark: https://github.com/recommenders-team/recommenders/actions/runs/7307019104

If they all pass, we can close this issue.