TileDB-Py built without GCS Support

[jacobgreen405]

I encountered an error when trying to point to a TileDB array from Google Cloud storage. The catch is, it only happens if tiledb-py was installed a specific way. For instance:

After installing TileDB into my conda environment with conda install -c conda-forge tiledb tiledb-py (TileDB version 2.10.3, tiledb-py version 0.16.3), then calling dask.array.from_tiledb('gcs://my-bucket/array.tldb') after setting os.environ('GOOGLE_APPLICATION_CREDENTIALS')='/path/to/token.json', I receive the error:

---------------------------------------------------------------------------
TileDBError                               Traceback (most recent call last)
Input In [5], in <cell line: 3>()
      1 os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = token
      2 tic1 = time.time()
----> 3 da = dsa.from_tiledb(root + data + '.tldb',
      4                      storage_options={'sm.compute_concurrency_level': 1, 'sm.io_concurrency_level': 1})
      5 toc1 = time.time()
      6 connectTime = toc1 - tic1

File /contrib/Jacob.Green/miniconda3/envs/cloud-data/lib/python3.10/site-packages/dask/array/tiledb_io.py:48, in from_tiledb(uri, attribute, chunks, storage_options, **kwargs)
     46     tdb = uri
     47 else:
---> 48     tdb = tiledb.open(uri, attr=attribute, config=tiledb_config, key=key)
     50 if tdb.schema.sparse:
     51     raise ValueError("Sparse TileDB arrays are not supported")

File /contrib/Jacob.Green/miniconda3/envs/cloud-data/lib/python3.10/site-packages/tiledb/highlevel.py:23, in open(uri, mode, key, attr, config, timestamp, ctx)
      9 def open(uri, mode="r", key=None, attr=None, config=None, timestamp=None, ctx=None):
     10     """
     11     Open a TileDB array at the given URI
     12 
   (...)
     21     :return: open TileDB {Sparse,Dense}Array object
     22     """
---> 23     return tiledb.Array.load_typed(
     24         uri,
     25         mode=mode,
     26         key=key,
     27         timestamp=timestamp,
     28         attr=attr,
     29         ctx=_get_ctx(ctx, config),
     30     )

File tiledb/libtiledb.pyx:3577, in tiledb.libtiledb.Array.load_typed()

File tiledb/libtiledb.pyx:3403, in tiledb.libtiledb.preload_array()

File tiledb/libtiledb.pyx:575, in tiledb.libtiledb._raise_ctx_err()

File tiledb/libtiledb.pyx:560, in tiledb.libtiledb._raise_tiledb_error()

TileDBError: [TileDB::VFS] Error: TileDB was built without GCS support

However, if I install TileDB with pip install tiledb, everything works as intended.

[nguyenv]

Hi @jacobgreen405,

We apologize for the inconvenience. We are currently working on fixing GCS support for the tiledb conda packages at https://github.com/conda-forge/tiledb-feedstock/pull/161.

As you mention, the current workaround is to use the PyPI wheels instead.

Thanks.

[igorol]

Hi all,

Is this issue solved? I see the same error with tiledb-py version 0.20.0.

I'm running on a Mac M2. Is it possible it's something related to tiledb-py for the arm64 architecture not being built with GCS support?

Thanks

[ihnorton]

TileDB built with GCS is available on all platforms from conda-forge (https://github.com/conda-forge/tiledb-feedstock/pull/184), as well as linux and macOS wheels installed via pip from PyPI (Windows should be available in TileDB 2.17). Please let us know if there are any issues.

[cameronraysmith]

I'm running on a Mac M2. Is it possible it's something related to tiledb-py for the arm64 architecture not being built with GCS support?

TileDB built with GCS is available on ... macOS wheels installed via pip from PyPI. Please let us know if there are any issues.

https://github.com/TileDB-Inc/TileDB-Py/blob/a355893106ac50b4d0d90f492d9bb3a010096919/misc/azure-release.yml#L33-L39

from Feb 2, 2023 https://github.com/TileDB-Inc/TileDB-Py/pull/1614/files#diff-4d387a144a0b721ed0135142faa6865225e425f757cf4d74e3891aae73c8e277R32

[ihnorton]

@cameronraysmith thanks for the comment. I've opened #1899 to see about (re-)enabling in the wheel builds.

[cameronraysmith]

Thank you @ihnorton .