[BUG] Operationalization notebooks don't run

[anargyri]

Description

Running notebooks examples/05_operationalize/als_movie_o16n.ipynb and examples/05_operationalize/lightgbm_criteo_o16n.ipynb fails in different ways.

It seems that the code is not in sync with the version of AzureML SDK prescribed in the conda env.

In which platform does it happen?

• Jupyter on Linux
• Databricks

How do we replicate the issue?

examples/05_operationalize/als_movie_o16n.ipynb

• Open notebook on Jupyter or VSCode
• Run cells (including az login)
• Workspace.create() fails

examples/05_operationalize/lightgbm_criteo_o16n.ipynb

• This notebook requires running examples/02_model_content_based_filtering/mmlspark_lightgbm_criteo.ipynb first on Databricks
• The latter fails on Databricks (although it succeeds on Linux) at the load_spark_df() call.

Expected behavior (i.e. solution)

The notebooks should complete successfully on Jupyter (als_movie_o16n.ipynb) and Databricks (mmlspark_lightgbm_criteo.ipynb, lightgbm_criteo_o16n.ipynb).

To this end, the code should be rewritten to be consistent with the latest AzureML SDK and Databricks.

Other Comments

[miguelgfierro]

@Inhenn proposal for tasks:

• [ ] Understand Cosmos, AzureML, AKS and Databricks
• [ ] Make sure you can create instances of Cosmos, AzureML, AKS and Databricks
• [ ] Remove pydocuemntdb https://github.com/microsoft/recommenders/blob/main/setup.py#L41 and do a PR. Please take a look at the contribution guideline.
• [ ] Adjust the rest of the code that has pydocumentdb and replace it with azure-cosmos. See the code here: https://github.com/microsoft/recommenders/search?q=pydocumentdb and do a PR
• [ ] Make sure that als_movie_o16n.ipynb runs and do a PR (if needed)
• [ ] Make sure that lightgbm_criteo_o16n.ipynb runs and do a PR (if needed)