search

appsmithorg / appsmith

Platform to build admin panels, internal tools, and dashboards. Integrates with 25+ databases and any API.
https://www.appsmith.com
Apache License 2.0
34.35k stars 3.72k forks source link

[Feature] Proxy support for API datasources #3311

Open ngauje opened 3 years ago

ngauje commented 3 years ago

Summary

This issue to support proxy configuration on datasources. In some environments some APIs can only be accessible through an HTTP proxy. It could be a proxy to go outside a private network (on internet for example), or a proxy to access a specific subset of services inside this company.

Motivation

As an Appsmith user in a corporate network, I would like to access to an internal API behind an HTTP proxy.

Additional Context

I tried the Java way of configuring a proxy (https://docs.oracle.com/javase/6/docs/technotes/guides/net/proxies.html) with no success. The risk with this approach is to proxify all the network flows. A per-datasource configuration seems to be more optimized.

Discussion started on Discord about this: https://discord.com/channels/725602949748752515/760761686549463060/814451065638223872

mohanarpit commented 3 years ago

We use Netty client libraries in order to initiate HTTP connections via Appsmith. Unfortunately, Netty doesn't support Java's system proxy settings.

We'll prioritize this feature & try to get this done via Java's library itself internally.

nidhi-nair commented 3 years ago

Additional information: Netty does not pick up proxy configuration via environment variables. The discussion around this issue seems to suggest this method of programmatic configuration. (https://github.com/reactor/reactor-netty/issues/887#issuecomment-549439355)

mohanarpit commented 3 years ago

@nidhi-nair I have a POC in the branch feature/api-proxy. Check the commit d3efb3787658a399d8f57e5ff3d76a219ae20732. I didn't push this change because I got entangled in dependency hell when trying to compile the entire code base. The tests run successfully but the server code as a whole wouldn't compile.

Maybe this will give you some direction in terms of the actual fix.

WernerEngelsB commented 2 years ago

Is there still any progress in this topic? Within an Corporate-Network most APIs will be behind a proxy to leave the corporate network-zone...

sharat87 commented 2 years ago

@nidhi-nair @mohanarpit, I don't think this issue is about setting a proxy at the instance level, with an environment variable. This is about configuration a proxy for specific API datasource, and no other. This should be set at the WebClient level, not at the java level.

nidhi-nair commented 2 years ago

You're right about this needing to be WebClient level @sharat87. Arpit's POC does exactly this IMO. We'll bring this issue back into priority on our backlog since the branch already exists.

@WernerDreierB thanks for bringing focus back to this. We'll keep this issue updated! :)

WernerEngelsB commented 2 years ago

Ping... this is still an issue in 1.7.9 A proxy can be set, however, it seems that the "no-proxy"-variable seems to be ignored. Therefore I can either have "only internet facing apis" or "only corporate-network facing apis", but not both! It's impossible to run appsmith in an corporate environment when proxy-settings are not properly configurable. Please prioritise.

nidhi-nair commented 2 years ago

Hey @WernerDreierB , we have an open PR for this issue here. Also, we seem to have a duplicate issue (or rather one with a wider scope) here, if you'd like to subscribe to it.

Apologies for the delay here!