search

microsoftgraph / msgraph-sdk-go

Microsoft Graph SDK for Go
https://docs.microsoft.com/en-us/graph/sdks/sdks-overview
MIT License
239 stars 36 forks source link

CloudQuery Source Plugin? #386

Closed yevgenypats closed 1 year ago

yevgenypats commented 1 year ago

Hi Team, hopefully this is right place to ask, if not, I'd appreciate if you can direct me.

I'm the founder of cloudquery.io, a high performance open source ELT framework.

We are maintaining an Azure plugin, which is widely used and users use it to sync their APIs to a database/data-lake and create asset inventory, operational data lake and other use-cases.

As we have limited capacity to maintain all plugins, we are usually looking for the official vendor to help maintain it (similar to terraform provider).

I was curious if this would be an interesting collaboration, where we can help with the initial version (already implemented) and you will help maintain it?

This will give your users the ability to sync/ELT Azure APIs to any of their data-lakes/data-warehouses/databases easily using any of the growing list of CQ destination plugins. I believe this will also help with various underlying managed asset inventory solutions where you currently (I assume) maintain some kind of in-house ELT solution, while giving your users more flexibility and taking advantage of new data-lake based approaches.

Im linking here recent requests from our users:

(P.S I know this is the msgraph SDK and not the azure one but we will be happy to help create an msgraph cloudquery source plugin given there is interest and as long as you can help us maintain it)

Best, Yevgeny

baywet commented 1 year ago

Hi @yevgenypats Thanks for reaching out and for your interest in the Go SDK.

First, this repo is for the Microsoft Graph SDK, if you're looking for the Azure SDK this is the right repo (blob storage, etc...)

Then, Microsoft Graph APIs are not meant for heavy extraction scenarios. For that, Microsoft Graph Data connect is what people want to have a look at.

Finally, we're not staffed to be able to help maintaining a 3rd party eco-system. This would be too much of a stretch for us because of the vastness of the ecosystem. It'd also make for an unfair advantage for the solutions we would provide support to compared with the ones that wouldn't get that kind of support.

I think the best bet now if your team is not staffed to support the extension, and because we can't support integrations with other solutions, would be to try to build a team of community members who care about this problem space (intersection of cloud query and Microsoft Graph) to have them build the integration.

Let us know if you have additional questions/remarks.

yevgenypats commented 1 year ago

Hi @yevgenypats Thanks for reaching out and for your interest in the Go SDK.

First, this repo is for the Microsoft Graph SDK, if you're looking for the Azure SDK this is the right repo (blob storage, etc...)

Then, Microsoft Graph APIs are not meant for heavy extraction scenarios. For that, Microsoft Graph Data connect is what people want to have a look at.

Finally, we're not staffed to be able to help maintaining a 3rd party eco-system. This would be too much of a stretch for us because of the vastness of the ecosystem. It'd also make for an unfair advantage for the solutions we would provide support to compared with the ones that wouldn't get that kind of support.

I think the best bet now if your team is not staffed to support the extension, and because we can't support integrations with other solutions, would be to try to build a team of community members who care about this problem space (intersection of cloud query and Microsoft Graph) to have them build the integration.

Let us know if you have additional questions/remarks.

Hey @baywet ! I think this makes total, sense and I'll try to open the issue in the azure-sdk-for-go which is the right repo.

Re unfair advantage. 1) we are open source framework. 2) I think it's more of a business decision and I don't know who's the right contact but I guess this like anything else would depend on customers request (For example if this would be top priority and customers would ask for integrations for two ELT vendors then potentially Azure would put more resources on that to maintain, so everything goes back to customers needs (most of the time)).

Anyway, I think this makes sense and I'll close this issue, thanks for the detailed response!