search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.97k stars 4.1k forks source link

Source Salesforce: Support oauth'ing into sandbox environment #7722

Closed sherifnada closed 2 years ago

sherifnada commented 2 years ago

Tell us about the problem you're trying to solve

When oauth'ing a salesforce user into a sandbox environment, we need to send them to test.salesforce.com rather than login.salesforce.com. Our current server implementation of salesforce oauth always assumes they are not trying to use a sandbox environment and redirects them to login.salesforce.com.

Unfortunately, many enterprise users are not allowed to use production environments for testing new tools. Therefore, they would not be able to adopt Airbyte Cloud without being able to access the sandbox environment.

Ideally, we would wait for #6971 to be implemented and use the functionality offered there to implement this flow. However, since that could take some time, we need to come up with a way to allow new Airbyte Cloud customers to start Once #6971 is implemented, we need to look at the user's input config and determine which oauth configuration to use.

Describe the solution you’d like

This fix has the following components:

  1. Disable the sandbox option of the Salesforce connector and always assume we are using the production environment (remove it from the spec but not from the code, and allow additional properties = true. if the undocumented option is passed then the connector should use sandbox)
  2. Create a fork of the salesforce connector, source-salesforce-sandbox (similar to the source-file-secure fork of the source-file connector) which always assumes we are using the salesforce sandbox environment. By the end we should have two salesforce connectors, source-salesforce which always assumes NOT sandbox, and source-salesforce-sandbox which always assumes sandbox environment. Publish both connectors normally like we do with any connector.
  3. Create a new oauth flow in the server which can log the user in for the sandbox environment. For testing this flow, use the Salesforce (Sandbox) Lastpass credentials. The only difference between this flow and the current salesforce flow is that the current salesforce flow uses the login.salesforce.com endpoint whereas the new flow should use test.salesforce.com. That's it. Otherwise, it can use the same credentials.
  4. Once you've tested the flow, create a PR and merge as normal on salesforce.
  5. Create a ticket to revert 1 & 2 once #6971 is implemented. That is, once that ticket is implemented, we should go back to having a single salesforce connector with a sandbox toggle whose oauth flow is controlled based on the toggle.
VasylLazebnyk commented 2 years ago

blocked by #7915

sherifnada commented 2 years ago

@vitaliizazmic the UI logic has been merged, this issue should no longer be blocked