Open jchamberlain opened 1 year ago
I don't see providers being unregistered when shutting down....
Good catch, @gdey. My impression was that providers lived only as pointers on each map, but I didn't realize each driver/provider type is keeping track of its instantiated providers. This causes a couple problems with my approach:
How important is it for the Driver/provider type to keep track of its instances? What if we instead kept a master list of all providers, and added a Cleanup()
method to Tiler
and MVTTiler
?
Actually, if we add a Cleanup()
method to each provider, we don't need a master list. We can grab the provider from the map(s) as we unload the maps.
Actually, if we add a
Cleanup()
method to each provider, we don't need a master list. We can grab the provider from the map(s) as we unload the maps.
Despite my previous comment, I ended up creating a global list of provider instances in provider/provider.go
. However, I prefix their names with a namespace (in the case of app config files, the file name) so uniqueness is required only per "app".
I have not forgotten this. My work has been focused on integrating our other systems with Tegola since Tegola is working well enough already, but I'll have some time later this month to get back to working on Tegola itself. I'll add the experimental guard to this PR as per @ARolek's request.
This is a first draft to demonstrate an approach to solving for #944 (Dynamic loading of maps). This is not meant to be ready for merge, but I've run it locally and it works well so far.This is now ready for review. Please give it a spin, and I'd love your feedback on the acceptability of the overall approach. Thanks!New Abstraction
The central new idea/concept/abstraction is the "App". (I'm sure there's a better term for this, but bear with me.) An App is a collection of providers and maps which share a lifecycle. They are loaded into a Tegola instance as a unit and, when no longer needed, unloaded from Tegola as a unit.
The base TOML configuration file may contain an initial App which will never be unloaded. (Which is to say, the
[[providers]]
and[[maps]]
section of the TOML file still work exactly as expected.)Configuring Tegola to load Apps dynamically
A new, optional section is added to the base config,
[app_config_source]
. It includes atype
plus any options required for that source type. The only option for the "file" type isdir
which sets a directory from which to load apps.Add any TOML file to that directory and its
[[providers]]
and[[maps]]
will be registered immediately. Delete the file, and they'll be removed.Key Implementation Details
The
config/source
package has been added providingConfigSource
andConfigWatcher
(implemented only byFileConfigSource
so far):The
initConfig
process of Tegola's root command has been augmented withinitAppConfigSource()
, which starts a goroutine to receive updates and deletions from theConfigWatcher
channels.To ensure that apps are loaded and unloaded as a unit,
Atlas
has been given two new methods,AddMaps()
andRemoveMaps()
. Name collisions are checked before adding any maps toAtlas
, so that we don't quit partway through having added some maps but not others.