Using API tokens with different jupyter notebook environments

[simon-clematide]

The current setup of the datalab is using it via its repository. This allows to have a .env file etc. for configuration. For google colab it is a bit more difficult. But there we could provide access colab SECRETS. I suggest to have a generic way how to get the required tokens:

• look into the environment, test whether we are running inside colab, and try colab userdata.

Maybe something like

import os

def get_token(var_name):
    """
    Retrieves the token by checking multiple sources:
    1. Environment variables
    2. Colab's SECRETS (if running in Colab)
    """
    try:
        # Check if running in Colab
        import google.colab
        if var_name in os.environ:
            return os.environ.get(var_name)
        else:
            try:
                # Access Colab's SECRETS
                from google.colab import auth
                auth.authenticate_user()  # Authenticate Colab user if necessary
                return os.environ.get(var_name, "Token not found in Colab SECRETS")
            except KeyError:
                return "Token not found in Colab SECRETS"
    except ImportError:
        # Fallback for non-Colab environments
        return os.getenv(var_name, "Token not found in environment")

# Example usage:
# token = get_token('API_KEY')
# print(token)

[EmanuelaBoros]

On it. (I thought one doesn't need an API key if the model is public)

[simon-clematide]

I think one does not need a HF_TOKEN, but it warns anyway. But not really entirely sure. But we should be certain about it, and not say "maybe you need a HF_TOKEN":-)