TUM-DAML / seml

SEML: Slurm Experiment Management Library
Other
165 stars 29 forks source link

create_neptune_observer does not use api_token argument #91

Closed heborras closed 2 years ago

heborras commented 2 years ago

Expected Behavior

Passing an api_token to the create_neptune_observer should make the function use this argument.

Actual Behavior

The argument is ignored and the default setting or the setting defined in the config file is used.

Steps to Reproduce the Problem

  1. Install seml
  2. Add the create_neptune_observer to the example
  3. Start the example experiment
  4. The example experiment then crashes complaining about an invalid API token
  5. Moving the API token to the user config fixes the error

Error message:

Starting job 68847
SLURM assigned me the node(s): ceg-brook02
Experiments are running under the following process IDs:
Experiment ID: 82       Process ID: 2474

Traceback (most recent calls WITHOUT Sacred internals):
  File "/home/hborras/.conda/envs/seml/lib/python3.7/site-packages/neptune/internal/api_clients/credentials.py", line 95, in _api_token_to_dict
    return json.loads(base64.b64decode(api_token.encode()).decode("utf-8"))
  File "/home/hborras/.conda/envs/seml/lib/python3.7/base64.py", line 87, in b64decode
    return binascii.a2b_base64(s)
binascii.Error: Invalid base64-encoded string: length cannot be 1 more than a multiple of 4

During handling of the above exception, another exception occurred:

Traceback (most recent calls WITHOUT Sacred internals):
  File "/tmp/e9f90b9f-fc54-4079-a03f-22190d11122d/examples/example_experiment.py", line 24, in <module>
    ex.observers.append(seml.create_neptune_observer('hendrikb/'+db_collection, api_token=None))
  File "/home/hborras/seml-test2/seml/seml/observers.py", line 163, in create_neptune_observer
    neptune_obs = NeptuneObserver(api_token=api_token, project_name=project_name, source_extensions=source_extensions)
  File "/home/hborras/.conda/envs/seml/lib/python3.7/site-packages/neptunecontrib/monitoring/sacred.py", line 81, in __init__
    neptune.init(project_qualified_name=project_name, api_token=api_token)
  File "/home/hborras/.conda/envs/seml/lib/python3.7/site-packages/neptune/__init__.py", line 185, in init
    proxies=proxies,
  File "/home/hborras/.conda/envs/seml/lib/python3.7/site-packages/neptune/internal/api_clients/backend_factory.py", line 30, in backend_factory
    return HostedNeptuneBackendApiClient(api_token, proxies)
  File "/home/hborras/.conda/envs/seml/lib/python3.7/site-packages/neptune/utils.py", line 298, in wrapper
    return func(*args, **kwargs)
  File "/home/hborras/.conda/envs/seml/lib/python3.7/site-packages/neptune/internal/api_clients/hosted_api_clients/hosted_backend_api_client.py", line 69, in __init__
    self.credentials = Credentials(api_token)
  File "/home/hborras/.conda/envs/seml/lib/python3.7/site-packages/neptune/internal/api_clients/credentials.py", line 76, in __init__
    token_dict = self._api_token_to_dict(self.api_token)
  File "/home/hborras/.conda/envs/seml/lib/python3.7/site-packages/neptune/internal/api_clients/credentials.py", line 97, in _api_token_to_dict
    raise InvalidApiKey()
neptune.api_exceptions.InvalidApiKey:

----InvalidApiKey-----------------------------------------------------------------------

Your API token is invalid.

Learn how to get it in this docs page:
https://docs-legacy.neptune.ai/security-and-privacy/api-tokens/how-to-find-and-set-neptune-api-token.html

There are two options to add it:
    - specify it in your code
    - set an environment variable in your operating system.

CODE
Pass the token to neptune.init() via api_token argument:
    neptune.init(project_qualified_name='WORKSPACE_NAME/PROJECT_NAME', api_token='YOUR_API_TOKEN')

ENVIRONMENT VARIABLE (Recommended option)
or export or set an environment variable depending on your operating system:

    Linux/Unix
    In your terminal run:
        export NEPTUNE_API_TOKEN=YOUR_API_TOKEN

    Windows
    In your CMD run:
        set NEPTUNE_API_TOKEN=YOUR_API_TOKEN

and skip the api_token argument of neptune.init():
    neptune.init(project_qualified_name='WORKSPACE_NAME/PROJECT_NAME')

You may also want to check the following docs pages:
    - https://docs-legacy.neptune.ai/security-and-privacy/api-tokens/how-to-find-and-set-neptune-api-token.html
    - https://docs-legacy.neptune.ai/getting-started/quick-starts/log_first_experiment.html

Need help?-> https://docs-legacy.neptune.ai/getting-started/getting-help.html

Specifications

Details - Version: 0.3.6 - Python version: 3.7 - Platform: ``` (seml) [hborras@ceg-octane examples]$ uname -a Linux ceg-octane 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux ``` - Anaconda environment (`conda list`): ``` (seml) [hborras@ceg-octane examples]$ conda list # packages in environment at /home/hborras/.conda/envs/seml: # # Name Version Build Channel _libgcc_mutex 0.1 main _openmp_mutex 4.5 1_gnu ca-certificates 2021.10.26 h06a4308_2 certifi 2021.10.8 py37h06a4308_2 colorama 0.4.4 pypi_0 pypi debugpy 1.5.1 pypi_0 pypi docopt 0.6.2 pypi_0 pypi gitdb 4.0.9 pypi_0 pypi gitpython 3.1.27 pypi_0 pypi importlib-metadata 4.11.2 pypi_0 pypi jsonpickle 1.5.2 pypi_0 pypi libedit 3.1.20210910 h7f8727e_0 libffi 3.2.1 hf484d3e_1007 libgcc-ng 9.3.0 h5101ec6_17 libgomp 9.3.0 h5101ec6_17 libstdcxx-ng 9.3.0 hd4cf53a_17 munch 2.5.0 pypi_0 pypi ncurses 6.3 h7f8727e_2 numpy 1.21.5 pypi_0 pypi openssl 1.0.2u h7b6447c_0 packaging 21.3 pypi_0 pypi pandas 1.1.5 pypi_0 pypi pip 21.2.2 py37h06a4308_0 py-cpuinfo 8.0.0 pypi_0 pypi pymongo 4.0.1 pypi_0 pypi pyparsing 3.0.7 pypi_0 pypi python 3.7.0 h6e4f718_3 python-dateutil 2.8.2 pypi_0 pypi pytz 2021.3 pypi_0 pypi pyyaml 6.0 pypi_0 pypi readline 7.0 h7b6447c_5 sacred 0.8.2 pypi_0 pypi seml 0.3.6 pypi_0 pypi setuptools 58.0.4 py37h06a4308_0 six 1.16.0 pypi_0 pypi smmap 5.0.0 pypi_0 pypi sqlite 3.33.0 h62c20be_0 tk 8.6.11 h1ccaba5_0 tqdm 4.63.0 pypi_0 pypi typing-extensions 4.1.1 pypi_0 pypi wheel 0.37.1 pyhd3eb1b0_0 wrapt 1.13.3 pypi_0 pypi xz 5.2.5 h7b6447c_0 zipp 3.7.0 pypi_0 pypi zlib 1.2.11 h7f8727e_4 ```