[proposal]better control http and grpc related dependencies

[skyao]

background

Since we have HTTP API and gRCP API in dapr, in Dapr Java sdk project, we both have some dependencies to support Dapr HTTP API and Dapr gRPC API:

• sdk-autogen project: depends on grpc-netty-shaded / grpc-protobuf / grpc-stub for Dapr gRPC API

      <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-netty-shaded</artifactId>
      <version>${grpc.version}</version>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-protobuf</artifactId>
      <version>${grpc.version}</version>
    </dependency>
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-stub</artifactId>
      <version>${grpc.version}</version>
    </dependency>

• Sdk project: depends on okhttp and jackson for Dapr HTTP API

    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>2.12.6.1</version>
    </dependency>
    <dependency>
      <groupId>com.squareup.okhttp3</groupId>
      <artifactId>okhttp</artifactId>
      <version>4.9.0</version>
    </dependency>

  and also depends on dapr-sdk-autogen for Dapr gRPC API:

  <dependency>
  <groupId>io.dapr</groupId>
  <artifactId>dapr-sdk-autogen</artifactId>
  <version>${project.version}</version>
  </dependency>

In dapr java sdk project, now we don't have fine-grained dependency control and all the subprojects are depend on each other simply. So for each application, whether it depends on the sdk/sdk-springboot/sdk-actors project, it will end up depending on all the http and grpc dependencies:

Problems

The problem is that the developers will find that both HTTP and gRPC related dependencies are included in the application's classpath even only HTTP API or gRPC API is used. It will introduce unnecessary dependencies into application and make the package more bigger.

A more serious problem is that these dependencies increase the probability of dependency conflicts. An unused dependency causing a dependency conflict... It doesn't make sense.

In critical projects, dependencies are often required to be reviewed. Introducing dependencies that are not used will be challenged.

To exclude some transitive dependencies, the developers have to set exclusions when they use the dependencies. For example, to introduce dapr sdk without gRPC API support:

    <dependency>
      <groupId>io.dapr</groupId>
      <artifactId>dapr-sdk</artifactId>
      <version>${project.version}</version>
      <exclusions>
        <exclusion>
          <groupId>io.dapr</groupId>
          <artifactId>dapr-sdk-autogen</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

Or to introduce dapr sdk without http API support:

    <dependency>
      <groupId>io.dapr</groupId>
      <artifactId>dapr-sdk</artifactId>
      <version>${project.version}</version>
      <exclusions>
        <exclusion>
          <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
        </exclusion>
        <exclusion>
          <groupId>com.squareup.okhttp3</groupId>
            <artifactId>okhttp</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

This is not friendly to devepers and easy to make mistakes.

Proposal

split into http and gRPC parts

To better control http and gRPC related dependencies and NOT introduce unnecessary dependencies into Daprd based Java applications, I suggest to clearly split http and gRPC dependencies, so that users can explicitly choose to use either http or gRPC, or both if necessary.

We should split sdk project into three new projects:

• sdk-grpc: include all the code for Dapr gRPC API and depends on sdk-autogen
• sdk-http: include all the code for Dapr HTTP API
• sdk-common: include common code for sdk-grpc and sdk-http
• sdk: current code will move into sdk-grpc and sdk-http projects, keep this sdk for backward compatible or we can mark it as "deprecated" and remove it later safely.

Similarly, other projects like sdk-actors and sdk-springboot need to be split in this way:

how to use dapr java sdk

To use only dapr http api, the applications should depends on sdk-springboot-http / idk-actors-http / sdk-http :

To use only dapr gRPC api, the applications should depends on sdk-springboot-grpc / idk-actors-grpc / sdk-grpc :

For legacy applications which use dapr java sdk before this proposal, they can continue to depends on sdk-springboot / sdk-actors / sdk :

In particular, if the applications really need to use both HTTP API and gRPC API, suggest to use both sdk-xxx-http and sdk-xxx-grpc, since we will mark sdk-springboot / sdk-actors / sdk as "deprecated" and plan to remove them in the future.

default behavior of dapr java sdk

In dapr java sdk, gRPC client is used by default for all building blocks but except service invoke: HTTP client will be used for service invoke by default.

See the source code in class DaprClientBuilder in sdk project:

  public DaprClientBuilder() {
        ......
    this.apiProtocol = Properties.API_PROTOCOL.get();
    this.methodInvocationApiProtocol = Properties.API_METHOD_INVOCATION_PROTOCOL.get();
    this.daprHttpBuilder = new DaprHttpBuilder();
  }

  private static final DaprApiProtocol DEFAULT_API_PROTOCOL = DaprApiProtocol.GRPC;
  private static final DaprApiProtocol DEFAULT_API_METHOD_INVOCATION_PROTOCOL = DaprApiProtocol.HTTP;

Since we split http and grpc in this proposal, the default behaviro should update to:

• In sdk-xxx-http projects, suggest to change the default behavior to use http client for all building blocks
• In sdk-xxx-grpc projects, suggest to change the default behavior to use grpc client for all building blocks

And we need to discuss and make decision that should we keep the default behavior of current dapr java sdk: use both grpc and http (only for service invoke).

keep class name and package name

For backward compatible, when we split the sdk-XXX project into sdk-XXX-http / sdk-XXX-grpc / sdk-XXX-common, we should keep curernt class name and pacakge name, especially for PUBLIC class.

When developers update the version of Dapr Java sdk to the new version with this proposal is included, they only need to change the name of the dependencies and don't need to change the code (at least they don't need to change too many code).

[mukundansundar]

This is an interesting proposal. I can see main benefit being reducing the no. of dependencies as well as reduction in conflicts. If I am understanding this correctly the client interfaces that we have will be moved to the sdk-common project so users might have to modify code to accommodate that switch initially.

And if they want to switch between protocols the main change would ideally be dependency alone and not any code right?

That is this is assuming all user facing class and interfaces will be sdk-common....

Additionally will the sdk-http jar that still depends on sdk-common then exclude the sdk-autogen dependency?

I am seeing that this will be a breaking change unless we strictly maintain the client interfaces and classes package path as it is now.

@skyao is my understanding of the proposal correct and please answer the questions asked above....

@artursouza thoughts? I think this is a change that will reduce the dependency conflicts and help users not import code that they do not use.

[skyao]

This is an interesting proposal. I can see main benefit being reducing the no. of dependencies as well as reduction in conflicts. If I am understanding this correctly the client interfaces that we have will be moved to the sdk-common project so users might have to modify code to accommodate that switch initially.

And if they want to switch between protocols the main change would ideally be dependency alone and not any code right?

That is this is assuming all user facing class and interfaces will be sdk-common....

Additionally will the sdk-http jar that still depends on sdk-common then exclude the sdk-autogen dependency?

I am seeing that this will be a breaking change unless we strictly maintain the client interfaces and classes package path as it is now.

@skyao is my understanding of the proposal correct and please answer the questions asked above....

@artursouza thoughts? I think this is a change that will reduce the dependency conflicts and help users not import code that they do not use.

All client interface will be in sdk-http or sdk-grpc, sdk-common will contain common code , not protocol code.

Developers will inport sdk-xxx-http or sdk-xxx-grpc into their pom.xml and not need to set exclusion.

Please see the pictures about the dependencies above.

In fact, here's a simple two-choice question to control the grpc and http related dependencies:

• [x] we don't split the dependencies, let the users do: either accept all of these dependencies or try to exclusion some of them. This is current design. In this way, our implementation is easy and leave the problem to users. (yes, we can add some documentation about this).
• [ ] we split the dependencies, make the users easy to choose: decide to use http or grpc and then import sdk-xxx-http or sdk-xxx-grpc. In this way, our implementation will be a little complicated with more small subprojects, but it's easy for the users.

BTW: one of Dapr's important benefits is to make applications lightweight because of sidecar model. From this perspective, the introduction of unwanted dependencies will reduce Dapr's core value.

[tanvigour]

I like the idea of splitting into http and grpc and keeping common features as a separate piece. Although, while contributing to these products here on, we will need to be extremely careful of separation of concerns. How do we plan to address redundancy caused by splitting into three different components and adding more code to these in future?

[skyao]

This proposal will be canceled if we agree to only use dapr grpc api in java sdk and Deprecating HTTP API.

see https://github.com/dapr/java-sdk/issues/794.