Users should be able to load arbitrary connector classes - Githubissues

chronicle-app / chronicle-etl

📜 A CLI toolkit for extracting and working with your digital history

https://chronicle.app/

MIT License

149 stars 3 forks source link

Users should be able to load arbitrary connector classes #23

Closed hyfen closed 6 months ago

hyfen commented 2 years ago

Ways this could work:

using --extractor ./custom_extractor.rb as a flag
a known plugin directory in the config directory

We take that file, attempt to load it (and rescue from LoadError), and then set it as the connector in the JobDefinition.

This system would also need

basic validation system to ensure the right things are required and included

Limitations:

doesn't handle dependency management