search

BlueBrain / nexus

Blue Brain Nexus - A knowledge graph for data-driven science
https://bluebrainnexus.io/
Apache License 2.0
274 stars 73 forks source link

Azure Blob Storage storage implementation #4477

Open kshalot opened 8 months ago

kshalot commented 8 months ago

Description

We are looking to use Blue Brain Nexus in the Microsoft Azure cloud. One thing which is relevant for us is to use an object store for file storage which is covered in Nexus only via the S3Storage implementation. For Azure-based deployments, it's ideal to use Azure Blob Storage which is S3's counterpart. Interacting with ABS is already implemented via a client library, so in theory it's only necessary to wrap them in Nexus abstractions.

I wanted to get a feel of whether the storage classes in Nexus are curated in a way that would prevent contribution adding support for other providers. Also, I'm curious of the approximate effort necessary to implement this feature, as this is something that our team could potentially take on.

For now we are weighing our options. An easy/hacky alternative for us is to simply not push files directly through Nexus and only create the appropriate resources to represent the file as some ABS entity.

imsdu commented 8 months ago

Hello :),

Thanks for your interest in Nexus, it is nice to discover that we have unknown users out there besides BBP and known partners.

We don't currently deploy on Azure so unfortunately it will be difficult to make time for this as we are already quite busy.

Even the S3 implementation is very likely to be removed in 1.10 as we don't use it and it is getting in the way of technical migrations we are working on.

If you want to contribute to the repository, you would also have to go through legal steps via BBP/EPFL (i.e a CLA, we have only done it once, I don't remember the details) and of course to know some Scala).

As a alternative, maybe Azure provides a component acting as a bridge allowing to mount the Blob storage as a regular volume so that it is seen as a local storage by Nexus ?

imsdu commented 8 months ago

I am also curious of your current usage of Nexus, how you run it, feel free to contact us at the address: bbp-nexus-support@groupes.epfl.ch or use discussions if you want to share some details :)