Cloud Storage Integeration of VDMS through TileDB configuration

[vinothalagupandi]

What is the procedure to store the Tile DB artefacts such as Images , Entity, Descriptors, Videos to Cloud Storage services by configuring TileDB

[Ragaad]

For descriptors you have to create a descriptor set with either ["TileDBDense", "TileDBSparse"] engine. then it will add any descriptor to the new set . for Images, you need to make sure the format flag of the AddImage is "tdb" in the case it will store the image in the Tiledb format. We do not support Tiledb Video yet, but we are open to do so, for entities all entities are stored in our graph database PMGD.

[vinothalagupandi]

Thanks for the reply but my request is how to store the data/images/descriptors etc (entire “db” folder ) to cloud storage such as azure blob/s3/gcp instead of storing in local file systems (/db/images/jpg) is there any way to change it via vdms or tile db configuration? As VDMS pushes data to local storage using tile db how to do the same to cloud storages

[Ragaad]

We did some prototypes where we store the image/video to s3 bucket. it is not in the release yet, but if this match your request let us know, we could consider releasing it?

[vinothalagupandi]

Yeah Storing all data in s3 kinda storage than in local whenever we add images using vdms API is my exact request.Can you share this ? Thanks

[Ragaad]

the code is quite old now, we will update it and integrate it with the newest VDMS API and release it with the coming release. If you need to see the old code, it is in our open-source project VCL. Agian, this is not integrated yet with the newest VDMS code.

https://github.com/IntelLabs/vcl/blob/feature_dfs/src/RemoteConnection.cc

[vinothalagupandi]

I will check that for sure.please give feature to give the s3 credentials in vdms-config.json file and let vdms read those credentials from that config file to make upload operation in further release.kindly consider this support and i am looking forward to the new release thank you . example config-vdms.json format { "port": 55555, "max_simultaneous_clients": 100, "db_root_path": "db", "more-info": "github.com/IntelLabs/vdms", store-s3 : true , access-key: a, secret-key:b } if store-s3 is false then vdms should use local storage and vice versa

[Ragaad]

Thank you for this, we will update the code to refelt this addition and let you know

[ifadams]

@cwlacewe closing as stale, though did this get resolved through revival of S3 connector integration? @rolandoquesada?