How to access Lake pro with credentials

[dekacypher]

• Lake API version: 0.8.0
• Python version: 3.10.12
• Operating System: MacOS

Description

I've just bought a subscription plane in crypto lake and there's no documentation on how to access lake pro, and where enter the api credentials.

What I Did

I've tried this command on Google Colab, with no luck

from google.colab import userdata
userdata.get('secretName')

[leftys]

Hi, did you get this email we sent you two hours ago?

did you receive the registration email? The instructions are there together with your credentials and links to a few places to get started (documentation, examples).

The documentation about open interest data are on Data types page of our web.

Also did you get the registration email with subject Welcome to Lake? Everything is described there. Please check your spam folders as well. Also you are using two or three emails/accounts, one on outlook.com, one on ntnu.no domains, so you might want to check both as well.

[dekacypher]

Hi, yes I've got it on my mail, and have successfully creates the credentials file in the right path. I'm trying to fetch open interest data but when i run this code I, get an error:

lakeapi.use_sample_data(anonymous_access = False)

trades = lakeapi.load_data(
    table="origin_time",
    start=datetime.datetime(2022, 1, 1),
    end=datetime.datetime(2022, 5, 1),
    symbols=["BTC-USDT"],
    exchanges=['BINANCE'],
)
trades.set_index('received_time', inplace = True)
trades

{
    "name": "NoFilesFound",
    "message": "No data found for your query",
    "stack": "---------------------------------------------------------------------------
NoFilesFound                              Traceback (most recent call last)
/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb Cell 2 line 3
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=0'>1</a> lakeapi.use_sample_data(anonymous_access = False)
----> <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=2'>3</a> trades = lakeapi.load_data(
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=3'>4</a>     table=\"origin_time\",
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=4'>5</a>     start=datetime.datetime(2022, 1, 1),
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=5'>6</a>     end=datetime.datetime(2022, 5, 1),
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=6'>7</a>     symbols=[\"BTC-USDT\"],
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=7'>8</a>     exchanges=['BINANCE'],
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=8'>9</a> )
     <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=9'>10</a> trades.set_index('received_time', inplace = True)
     <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=10'>11</a> trades

File /opt/homebrew/lib/python3.11/site-packages/lakeapi/main.py:161, in load_data(table, start, end, symbols, exchanges, bucket, boto3_session, use_threads, columns, row_slice, drop_partition_cols, cached)
    158 for _ in range(2):
    159     try:
    160         # TODO: log & skip corrupted files
--> 161         df = lakeapi._read_parquet.read_parquet(
    162             path=f\"s3://{bucket}/{table}/\",
    163             partition_filter=partition_filter,
    164             categories=[\"side\"] if table == \"trades\" else None,
    165             dataset=True,  # also adds partition columns
    166             boto3_session=boto3_session,
    167             columns=columns,
    168             use_threads=use_threads,
    169             ignore_index=True,
    170             cached=cached,
    171         )
    172         break
    173     except botocore.exceptions.ClientError as ex:
    174         # When 404 file not found error happens, it means the boto cache of available files is wrong and we need
    175         # to clear it and try again.

File /opt/homebrew/lib/python3.11/site-packages/lakeapi/_read_parquet.py:576, in read_parquet(path, path_root, path_suffix, path_ignore_suffix, version_id, ignore_empty, ignore_index, partition_filter, columns, chunked, dataset, categories, safe, map_types, use_threads, last_modified_begin, last_modified_end, boto3_session, s3_additional_kwargs, pyarrow_additional_kwargs, cached)
    574     paths = _apply_partition_filter(path_root=path_root, paths=paths, filter_func=partition_filter)
    575 if len(paths) < 1:
--> 576     raise exceptions.NoFilesFound(\"No data found for your query\")
    577 _logger.debug(\"paths:\
%s\", paths)
    579 args: Dict[str, Any] = {
    580     \"columns\": columns,
    581     \"categories\": categories,
   (...)
    589     \"pyarrow_additional_kwargs\": pyarrow_additional_kwargs,
    590 }

NoFilesFound: No data found for your query"
}

[leftys]

Please delete the _lakeapi.use_sampledata call, that switches you to the small free sample dataset. Also table parameter should be trades, book, candles, funding etc (see https://crypto-lake.com/data/). Finally I recommend to update to the newest lakeapi package version. New version should have gave you a better error message in your case iirc.

Have fun exploring the data and we wish you successful research& trading! :)

[dekacypher]

Thank you so much for your help! I really appreciate it.

I made changes that you proposed, but now i get a different error:

trades = lakeapi.load_data(
    table="open_interest",
    start=datetime.datetime(2022, 1, 1),
    end=datetime.datetime(2022, 5, 1),
    symbols=["BTC-USDT"],
    exchanges=['BINANCE'],
)
trades.set_index('received_time', inplace = True)
trades

{
    "name": "NoFilesFound",
    "message": "No files Found on: s3://sample.crypto.lake/open_interest/.",
    "stack": "---------------------------------------------------------------------------
NoFilesFound                              Traceback (most recent call last)
/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb Cell 2 line 3
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=0'>1</a> get_ipython().run_line_magic('pip', 'install --upgrade lakeapi')
----> <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=2'>3</a> trades = lakeapi.load_data(
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=3'>4</a>     table=\"open_interest\",
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=4'>5</a>     start=datetime.datetime(2022, 1, 1),
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=5'>6</a>     end=datetime.datetime(2022, 5, 1),
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=6'>7</a>     symbols=[\"BTC-USDT\"],
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=7'>8</a>     exchanges=['BINANCE'],
      <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=8'>9</a> )
     <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=9'>10</a> trades.set_index('received_time', inplace = True)
     <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=10'>11</a> trades

File /opt/homebrew/lib/python3.11/site-packages/lakeapi/main.py:144, in load_data(table, start, end, symbols, exchanges, bucket, boto3_session, use_threads, columns, row_slice, drop_partition_cols, cached)
    141     assert exchanges[0].upper() == exchanges[0]
    143 if method == 'cloudfront':
--> 144     df = _load_data_cloudfront(
    145         table = table, start = start, end = end, symbols = symbols, exchanges = exchanges,
    146         boto3_session = boto3_session, use_threads = use_threads, username = username, cached = cached,
    147     )
    148 else:
    149     with botocache_context(
    150         cache=cache,
    151         action_regex_to_cache=[\"List.*\"],
   (...)
    155         supress_warning_message=is_anonymous_access,
    156     ):

File /opt/homebrew/lib/python3.11/site-packages/lakeapi/main.py:275, in _load_data_cloudfront(table, start, end, symbols, exchanges, boto3_session, use_threads, username, cached)
    258     boto3_session = boto3.Session(region_name=\"eu-west-1\")
    260 # credentials = boto3_session.get_credentials().get_frozen_credentials()
    261 # auth = AWSRequestsAuth(
    262 #     aws_access_key=credentials.access_key,
   (...)
    273 
    274 # if not exchanges or not symbols or not start or not end:
--> 275 available_data = list_data(table = table, start = start, end = end, symbols = symbols, exchanges = exchanges, boto3_session = boto3_session)
    276 df = pd.DataFrame(available_data)
    277 if not exchanges:

File /opt/homebrew/lib/python3.11/site-packages/lakeapi/main.py:416, in list_data(table, start, end, symbols, exchanges, bucket, boto3_session, last_modified_begin, last_modified_end)
    414 paths = lakeapi._read_parquet._apply_partition_filter(path_root=path_root, paths=paths, filter_func=partition_filter)
    415 if len(paths) < 1:
--> 416     raise lakeapi.exceptions.NoFilesFound(f\"No files Found on: {path}.\")
    417 return [_path_to_dict(path) for path in paths]

NoFilesFound: No files Found on: s3://sample.crypto.lake/open_interest/."
}

[leftys]

Maybe you didn't restart the notebook so that the use_sample_data call is still in effect? Because I still see sample.crypto.lake being accessed.

[leftys]

I see you already got it working, so closing this.