airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.16k stars 4.13k forks source link

Source Appstore: Invalid vendor ID error #2162

Closed sherifnada closed 3 years ago

sherifnada commented 3 years ago

Expected Behavior

When I create credentials for the Appstore source connector according to the documentation, I expect that it will work as expected and that I can sync all streams.

Current Behavior

Steps to Reproduce

  1. Follow the documentation to create an app key with the Finance role
  2. create a config.json matching the appstore spec and store it in some directory e.g /tmp/config.json
  3. Navigate to the appstore connector directory and activate the virtual environment: cd airbyte-integrations/connectors/source-appstore-singer && python -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt && pip install .[main]
  4. python main_dev.py check --config /ab_tmp/config.json
  5. The check will fail with an error saying the vendor is invalid
  6. change the code in source_appstore_singer/source.py's check_config method to make the reportType: "SALES" and version: 1_0 and run the check again, it will succeed

Severity of the bug for you

High - blocking a user from using airbyte

Airbyte Version

This is happening on all published versions of the connector

sherifnada commented 3 years ago

@Muriloo as someone who frequently uses this connector, have you ever seen this issue before?

sherifnada commented 3 years ago

This could maybe be explained if the app doesn't offer subscriptions. Confirming with the user.

sherifnada commented 3 years ago

the user confirmed they don't offer subscriptions on their app.

sherifnada commented 3 years ago

with this info, it seems like the Apple API returns wrong exception messages e.g invalid vendor when the real problem is that subscriptions aren’t enabled for the account. The fix will be to probe the API for all reports it can generate then offer to the user which reports they can sync based on which ones didn’t throw an exception. Silly but I think that’s what can be done with the tools the API gives us.

oseh-stake commented 2 years ago

Also interested