airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.68k stars 4.02k forks source link

fix appstore connector key handling #2015

Closed jrhizor closed 3 years ago

jrhizor commented 3 years ago

Tell us about the problem you're trying to solve

Currently the appstore connector fails because when pasting a file into the private_key field removes the newlines, and the underlying library requires the newlines to handle the file.

Describe the solution you’d like

  1. Allow configuring multiline text inputs in specs, example:

      "private_key": {
        "type": "string",
        "description": "Private_key is the contents of the <a href=\"https://developer.apple.com/documentation/appstoreconnectapi/creating_api_keys_for_app_store_connect_api\">key file</a> you use to connect to appstore's API.",
        "airbyte_secret": true,
        "multiline": true
      },
  2. Support textarea inputs in the UI for multiline that propagate the newline characters in the UI.

  3. Bump the version of the appstore connector so the resulting tap_config.json files look like:

    {
    "key_id": "aa",
    "private_key": "-----BEGIN PRIVATE KEY-----\nsomething\nsomething\n-----END PRIVATE KEY-----",
    "issuer_id": "aa",
    "vendor": "0",
    "start_date": "2020-11-16T00:00:00Z"
    }

Describe the alternative you’ve considered or used

I considered re-adding the newlines in files, but this seems buggy and doesn't address the general concern.

jrhizor commented 3 years ago

File upload support might be equally good as text areas for this.

jamakase commented 3 years ago

Just noticed @jrhizor @cgardens we have here secret + multine, how are we going to handle such case? In case of multiline + secret - should we make it visible when you specify it and masked with * when you edit? What's the expected behaviour?

cgardens commented 3 years ago

good question.

should we make it visible when you specify it and masked with * when you edit

i think your suggestion is good.

sherifnada commented 3 years ago

@cgardens are we ready to change the type of the input in the appstore source to multiline?

cgardens commented 3 years ago

@sherifnada it's on master but not on an officially released version. so your call on how you want to roll it out. thinking about it now this isn't going to be backwards compatible 😭

sherifnada commented 3 years ago

This issue was closed by accident. We should upgrade the appstore connector to use this by adding multiline: true to the key spec.