denshoproject / ddr-cmdln

Command-line tools for automating the Densho Digital Repository's various processes.
Other
0 stars 2 forks source link

Treat ddr-defs as data not Python modules #54

Open gjost opened 6 years ago

gjost commented 6 years ago

Currently ddr-defs/repo_modules are a set of Python modules. This has worked well for us, allowing us to specify fields (as data in a list-of-dicts) and do some custom data handling on a per-field basis using the converters functions.

This doesn't support the following use cases:

This is because the definitions path is specified in DDR.config or (django.conf.settings) and imported whenever DDR.identifier is first imported. Definitions can only be imported once.

If we could separate the custom converters code from definitions, then we could (for example) import multiple definitions files, putting them into some data structure. We'd have default definitions (of course).

Definitions could be embedded in organization repos, or in collections, or even entities. Objects could point to definitions at a filesystem path (dangerous) or a URL.

The definitions structure would cache these pointers so they only have to be ingested once per application session.