search

ScriptFUSION / Porter

:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.
GNU Lesser General Public License v3.0
611 stars 24 forks source link

CachingConnector is a poor user experience #61

Open Bilge opened 5 years ago

Bilge commented 5 years ago

Having to wrap a connector in CachingConnector just to use caching is not as easy to use as if the cache just worked with any connector. Moreover, cache + connector is a violation of SRP. The cache should be refactored as a separate entity, apart from connectors.

Bilge commented 5 years ago

There are a couple of problems with removing CachingConnector in lieu of storing the cache within Porter herself:

  1. The same cache would be used for all Porter providers, whereas CachingConnector permits different providers to use different caches very easily; possibly even for different resources, with some ingenuity.
  2. We want to cache the raw connection data, but Porter only receives record collections, which is not what we want to cache and cannot be cached easily (if at all) anyway.