search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.52k stars 4k forks source link

Firestore source request #3323

Closed Khamaldeeen closed 7 months ago

Khamaldeeen commented 3 years ago

I would like to have an integraiton between firestore database and power bi

Describe the context around this new integration

Describe the alternative you are considering or using

Continue to manually export the data from the firestore to my local instance in a JSON format and load the wrangled files into power bi

┆Issue is synchronized with this Asana task by Unito

marcosmarxm commented 3 years ago

hey @Khamaldeeen thanks for the request! I added this source to our roadmap. Do you want to contribute creating this connector? I can help with the initial steps

Khamaldeeen commented 3 years ago

Hi @marcosmarxm that would be fantastic. I will be glad to share my ideas.

cheyura commented 3 years ago

Webhook-based? (no/partially/yes) Partially https://firebase.google.com/docs/functions/firestore-events

Available authentication modes (API key/Oauth/other) While the easiest way to use Cloud Firestore is to use one of the native client libraries, there are some situations when it is useful to call the REST API directly.

The REST API can be helpful for the following use cases:

Accessing Cloud Firestore from a resource-constrained environment, such as an internet of things (IoT) device, where running a complete client library is not possible. Automating database administration or retrieving detailed database metadata.

Creating an account Cloud Firestore offers free quota that allows you to get started at no cost. The free quota amounts are listed below. https://firebase.google.com/docs/firestore/quotas

Sandbox account? No

How to populate the account with data? API,UI

Available streams for sync Below are brief descriptions of the two most important method groups. For a complete list, see the REST API reference or use the API Explorer.

v1.projects.databases.documents Perform CRUD operations on documents, similar to those outlined in the add data or get data guides.

v1.projects.databases.collectionGroups.indexes Perform actions on indexes such as creating new indexes, disabling an existing index, or listing all current indexes. Useful for automating data structure migrations or synchronizing indexes between projects.

Also enables retrieval of document metadata, such as the list of all fields and subcollections for a given document.

Integration supports incremental sync? No

Other information/blockers No

julienfouilhe commented 2 years ago

@igrankova I'm sorry to comment on a closed issue, but it's been closed as completed but I can't find the firestore source in Airbyte's list, is that normal? I'd be happy to contribute a Firestore source but I wanted to ensure it's not already done. Thanks!

igrankova commented 2 years ago

hi, @julienfouilhe, as I see, we have a destination Firestore, but no source. @YowanR we have a new contribution offered. can you help with the steps what to do, because I'm not in this process.

lightstrike commented 2 years ago

@julienfouilhe I'm also looking to create a Firestore source, let me know if you want to collaborate on this!

folkevil commented 1 year ago

Hi guys, have any update for Firebase source ? I see the document in website but can't find repo for this source.

andremosko commented 7 months ago

Hi! Any updates for Firestore as a source? Thanks =)