opensearch-project / data-prepper

OpenSearch Data Prepper is a component of the OpenSearch project that accepts, filters, transforms, enriches, and routes data at scale.
https://opensearch.org/docs/latest/clients/data-prepper/index/
Apache License 2.0
262 stars 202 forks source link

[RFC] Introducing Dependency Injection #519

Closed cmanning09 closed 2 years ago

cmanning09 commented 3 years ago

What kind of business use case are you trying to solve? What are your requirements?

Dependency Injection(DI) is the technique which objects receive their dependencies. Leveraging this technique and moving instantiation of dependencies out-side of the scope of an object give us a the following benefits:

What is the problem? What is preventing you from meeting the requirements?

Data Prepper (DP) does not utilize any Dependency Injection(DI) frameworks at this moment nor are we leveraging an existing framework which supports DI. Without a DI framework there are numerous code smells in DP. Currently, DP couples construction of dependent objects with implementation of other objects. DP also has cyclical dependencies and leverages static methods and instances to solve its own singleton management.

Other Requirements

What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?

We will adopt dependency injection framework to eliminate the current code smells in DP and help drive DP towards a more modular architecture.

What are remaining open questions?

cmanning09 commented 3 years ago

All three frameworks should work for our use case. While dagger is a compile time framework it does support assisted injection for runtime creation.

dlvenable commented 3 years ago

Overall, I'd like to see the following available:

  1. Data Prepper Core classes will have beans/instances available for dependency injection which are only part of Data Prepper Core. This likely would be the initial solution.
  2. Plugin authors can create their own sets of beans/instances to inject for their plugins so that they get the benefits for a DI framework.
  3. Both Core and Plugin can have a shared set of beans/instances which are available for use. This may not be necessary, but I'd like to be able to have it as an option.
  4. It also might be interesting for the plugin framework to add beans/instances to the Plugin set, rather than pass them in directly.

Spring DI has robust support for these needs. It generates application contexts at runtime (which is good for plugins). It supports application context inheritance which would work well for having a Common context which can be shared (item 3 above). And we can programmatically add beans which can help with item 4.

cmanning09 commented 2 years ago

Dagger won't be able to support core dependencies injected into plugins while Guice and Spring can. Given Spring's robust support, let's move forward with it. All plugin frameworks support JSR-330 and we should be able to easily migrate to one over the other if we identify issues (i.e start up performance impact).