search

nhl / link-move

A model-driven dynamically-configurable framework to acquire data from external sources and save it to your database.
Apache License 2.0
35 stars 15 forks source link

Improve design for URI connector #220

Closed vitalz closed 1 year ago

vitalz commented 1 year ago

By definition URI means different schemas (web url, classpath, file, ...) where each schema means its own connection type (and its processing / handling).
Any connection of these ones is related by a logical key StreamConnector.class for LinkMove engine what causes design problems.

For now URI connector is also assumed to be just a static value only: a String argument for java.net.URI constructor. There are additional features requested to be supported:

  1. authorization
  2. parametrization

For example, in most use cases web url needs api key authorization. It might either require parametrization supported like {teamId} parameter.

andrus commented 1 year ago

The new IConnectorFactory design introduced per #222 will allow multiple factories to be registered for a given connector type. LinkMove will first try to find a set of factories supporting the type. Then it would iterate through it, getting an Optional<C extends Connector> from each factory. The first factory returning a non-empty Optional wins. A specific Factory order is not guaranteed, so factories should be written to recognize explicit symbolic connector IDs.

The old approach of using ID as a URI is discouraged (and URIConnectorFactory is deprecated)