🐛 [Stream-Firestore-BigQuery] Import existing Firestore documents into BigQuery

DylanMontoya commented 1 month ago

[READ] Step 1: Are you in the right place?

Issues filed here should be about a feature request for a specific extension in this repository. To file a feature request that affects multiple extensions or the Firebase Extensions platform, please reach out to Firebase support directly.

[REQUIRED] Step 2: Extension name

This feature request is for extension: firestore-bigquery-export

What feature would you like to see?

Previously the functionality started with the following options, where it was indicated if when installing the extension the existing data was to be imported and then the name of the collection as well:

Import existing Firestore documents into BigQuery?
yes or no

Existing Documents Collection(opcional)
colletion name

Currently these functions are disabled and I would like them to be disabled again as it is much easier to structure the data you see in Bigquery.

How would you use it?

The alternative use without this functionality is from the following command and I have had good results, however to be more practical I would like to add the two configuration parameters mentioned above.

gcloud firestore export gs://nombrebucket --collection-ids=nombrecoleccion

pr-Mais commented 1 month ago

Hi @DylanMontoya, I believe you should be using the import script which does the same as the disabled backfill functionality. That functionality is currently disabled due to cost issues, and we're working on an alternative approach to enable it back.

pr-Mais commented 1 month ago

I will be closing this one in favor of https://github.com/firebase/extensions/issues/2029 which is tracking the same request.

firebase / extensions