search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.12k stars 4.12k forks source link

Google Directory - users stream doesn't work (normalization) #27673

Open jcolinger3 opened 1 year ago

jcolinger3 commented 1 year ago

Connector Name

Google Directory

Connector Version

0.2.1

What step the error happened?

During the sync

Revelant information

The users part of the Google Directory connection seems to fail for normalization/reset stream. Including logs for reset/sync. b19d0480_d7ba_4b12_95b1_45479c6e23c3_logs_138_txt.txt b19d0480_d7ba_4b12_95b1_45479c6e23c3_logs_135_txt.txt

Relevant log output

Failure Origin: normalization, Message: Something went wrong during normalization
2023-06-23 20:25:55 ERROR i.a.w.g.DefaultNormalizationWorker(run):86 - Normalization failed for job 139.

Contribute

jcolinger3 commented 1 year ago

Presented this error during office-hour 6/23. @marcosmarxm seemed to identify the issue as the "users". Tested by only using groups and groupmembers, and the sync/normalization worked fine.

jcolinger3 commented 1 year ago

@marcosmarxm Is there anything I can do to assist with this? I'm not a programmer, but I'm curious about this issue/connector for Google Directory. It seemed like you were able to diagnose the issue very quickly during the office hour. If the fix possible? Just wondering because we are a school that like this feature to audit Users in Google Directory (rather than manual download).

jcolinger3 commented 1 year ago

After testing on our end... I suspect the issue is with the json not being able to be transformed. We found a combination of fields that work... but I believe that there is still some improvement to be made here... at least to documentation to help others.