MLB notebook fails on API enablement when run in Vertex AI Workbench

[markbpryan]

MLB example is brilliant! However, running the notebook (https://github.com/statmike/vertex-ai-mlops/blob/main/Applied%20GenAI/Vertex%20AI%20GenAI%20For%20Document%20Q%26A%20-%20MLB%20Rules%20For%20Baseball.ipynb) in Vertex AI Workbench fails on this cell with authorization error:

# Enable Document AI For This Project
!gcloud services enable documentai.googleapis.com
# Enable Vertex AI For This Project
!gcloud services enable aiplatform.googleapis.com

Using the same approach used for setting the project ID corrects the problem and allows the APIs to be enabled without errors when the notebook is run in the Vertex AI Workbench:

try:
    import google.colab
    from google.colab import auth
    auth.authenticate_user()
    # Enable Document AI For This Project
    !gcloud services enable documentai.googleapis.com
    # Enable Vertex AI For This Project
    !gcloud services enable aiplatform.googleapis.com
except Exception:
    pass

[statmike]

Hi @markbpryan, Thank you for sharing your experience with the notebook. Are you ok sharing the error you experience with the !gcloud sevices ... commands? I think it was likely a permissions issue with the service account the Vertex AI Workbench notebook instance is running as. By moving these command under the try: import google.colab that likely pass because they never run in the Vertex AI Workbench environment since it fail the import of google.colab.

If the rest of the notebook still ran then it is likely because both Document AI and Vertex AI APIs were already enabled.

I updated the readme and each of the notebooks to have a prerequisites section with some additional clarity around the needs for role/permissions related to the services used in the notebook. I will add it to my todo list to give prescriptive minimal permissions needed for each later on.

Thank You, @statmike

[markbpryan]

Thanks - documenting the prerequisites should take care of it. With that, I'm going to close this issue. I still haven't been able to get this notebook to run to the end in Vertex Workbench (it's currently choking on importing PyPDF2), but I suspect the root cause could be some recent changes in Workbench levels, so I am going to try to work around that. In the mean time, thanks for looking at the issues I've reported and I'll close this issue.