snowplow / snowbridge

For replicating streams across clouds, accounts and regions
Other
15 stars 7 forks source link

Consider making sources independent of sourceiface.Source #167

Open adatzer opened 2 years ago

adatzer commented 2 years ago

Currently the return values of functions that create a new source (in each source package) are (sourceiface.Source, error). Example: stdin source, etc.

However this is not necessary, since each source implements the Source interface, but does not use it. In addition, in GetSource we also check whether the component returned does indeed implement the Source interface. Instead we could potentially make those functions return each specific source type (like what targets do). For example:

// before
func configFunction(c *configuration) (sourceiface.Source, error)

// after (e.g. for kinesis source)
func configFunction(c *configuration) (*kinesisSource, error)

This would also allow us to mock a specific source more easily for the cases where the Source interface functionality is not needed/tested (for example here).

Not sure if this is a good call, just found myself thinking about it a couple of times and thought to trigger a discussion.

colmsnowplow commented 2 years ago

Yup this makes sense - there's no need for us to use the interface type here at all. Each source config function should just return its source. Good spot!