TileDB-Inc / TileDB-VCF

Efficient variant-call data storage and retrieval library using the TileDB storage library.
https://tiledb-inc.github.io/TileDB-VCF/
MIT License
88 stars 15 forks source link

Azure blob storage credential parsing problem #603

Closed vstathias closed 11 months ago

vstathias commented 11 months ago

Error when using the credentials to connect to an azure blob

azure_cred = ["vfs.azure.storage_account_name=myblob", "vfs.azure.storage_sas_token=?sv=123456789=="]
config = tiledbvcf.ReadConfig(tiledb_config=azure_cred)
ds = tiledbvcf.Dataset(uri,
                       mode = "w",
                       cfg=config)
ds.create_dataset()

RuntimeError: TileDB-VCF exception: Error setting TileDB config parameter; bad value 'vfs.azure.storage_sas_token=?sv=123456789==

This could be due to the utils::split function, parsing multiple "=" that typically exist in Azure SAS & Azure Account Keys. See utils.cc, lines 318-331:

void set_tiledb_config(
    const std::vector<std::string>& params, tiledb_config_t* cfg) {
  tiledb_error_t* err;
  for (const auto& s : params) {
    auto kv = utils::split(s, '=');
    if (kv.size() != 2)
      throw std::runtime_error(
          "Error setting TileDB config parameter; bad value '" + s + "'");
    utils::trim(&kv[0]);
    utils::trim(&kv[1]);
    tiledb_config_set(cfg, kv[0].c_str(), kv[1].c_str(), &err);
    tiledb::impl::check_config_error(err);
  }
}
gspowley commented 11 months ago

Thank you @vstathias for the report and debug info. PR #605 should fix the issue.

pilare commented 11 months ago

@gspowley thank you for an immediate response and fix. Are you team plan to do some new release with that PR #605 included?** or we need to try to use Makefile, to compile it ourselves? There is no instruction on it and we haven't yet set up environment for it so I'm afraid it will take us some time.

gspowley commented 11 months ago

Yes, we're working on a 0.26.2 release, which will include the fix. It should be available within the next day or so.

vstathias commented 11 months ago

Perfect, thank you @gspowley!

gspowley commented 11 months ago

Please let us know if 0.26.2 resolved the issue.

vstathias commented 11 months ago

It works, thank you!