search

oduwsdl / MemGator

A Memento Aggregator CLI and Server in Go
https://memgator.cs.odu.edu/api.html
MIT License
55 stars 11 forks source link

Add REST api for toggling various command line settings, especially timeouts and archive list #51

Open ikreymer opened 8 years ago

ikreymer commented 8 years ago

When running under load, it is sometimes desirable to be able to tweak certain settings, without having to restart memgator (and any dependent containers). It would be great if the settings that are provided on the command line, especially the timeout settings and the archive list, could be modified while memgator is running, perhaps by doing a PUT or POST to a special api endpoint..

Note that this is separate from customizing archivelist per query, in case the user is not doing that, it would still be great to change the archivelist dynamically..

This is related to and can be mitigated by #43 although sometimes it is desirable to remove an archive that is not yet failing (but perhaps overloading the remote servers)

ibnesayeed commented 8 years ago

Yes, this is a planned feature, though in a slightly different manner. Instead of providing options to make changes to every config option individually, the approach would be to allow reloading the configs from a config file #2 with the help of an administrative endpoint (somewhat similar to how Nginx configs allow reloading except it would be invoked from an API endpoint not the CLI). This way anyone can issue a reload config request, but the changes to the config file can only be made by those who have access to the server. Although, it would be very convenient to pass a reference (location or URI) of a config file in the request or pass the config file as the payload of a PATCH request to make such changes, but it would force us to have some sort of authentication in place otherwise anyone could mess the global configs of the running instance up. However, per-request config overrides don't necessarily need any authentication as they can't affect others.

It should be noted that the config file will have a way to specify the archive list inline or as a reference, so the implementation will allow changing the archive list as well.

ikreymer commented 8 years ago

A nginx-style config and reload option would work great for my use case as I can do it server side :+1:

Btw, Memgator has been running reliably under heavy considerable load for days, so thanks for making a great tool :)

ibnesayeed commented 8 years ago

Btw, Memgator has been running reliably under heavy considerable load for days, so thanks for making a great tool :)

I am really glad to hear that. Did you enable log and profile flags in your instances at oldweb.today and would you be willing to share them sometime in future when the service is settled after the current peak? I would like to include some statistical analysis in the announcement blog post when the 1.0 GA is released.

There is a proposal #44 to expose the profiling information as a stream to allow subscription for visualization and monitoring purposes.