Move configuration from AppModel.js to an external config file

[laurenwalker]

I propose we copy the configuration properties from the AppModel default attributes object to a simple JSON file - from here on referred to as metacatui-config.js. AppModel will still have default values for each attribute, but it will pull in the metacatui-config.js JSON and set() all the attributes onto itself. This means the properties in metacatui-config.js always override the default attributes in AppModel.

We will pull in the metacatui-config.js file via a <script> tag in index.html. This URL would need to be configured for each MetacatUI installation so the app knows where to retrieve it. It would have a default value of ./metacatui-config.js (current directory). If no metacatui-config.js file is found, the app will continue to load with all the default settings.

We should also merge the theme-specific settings currently located in config.js into metacatui-config.js. Currently, each config.js file contains a hash of HTML templates, CSS files, and Backbone models to override for each theme. Product owners can override any theme file with any other web-accessible file. Ideally, they will have a theme directory on the web that they maintain independently from the rest of the MetacatUI app.

When a new MetacatUI version is released with changes to the AppModel/metacatui-config.js property list, the default values in AppModel will be used unless product owners manually update their theme's metacatui-config.js properties (e.g. if they want to customize a new HTML template or turn off a new feature). Any new properties added to the AppModel/metacatui-config.js property list should be documented in the Git release notes.

We no longer will use index.html for configuring the Google Map API key, theme name, or metacat context. These will all be moved to metacatui-config.js. The theme name property can be removed entirely since it will no longer be necessary for locating theme-specific files.

Product owners will need to be careful about overriding HTML templates that contain necessary elements for the MetacatUI app to function. For example, in the new metadata editor, each editor section and subsection is identified by data- attributes on HTML elements (e.g. a theme can totally redesign the "People" section HTML but still needs the data-section="people" attribute on the container element so the View knows how to find it.) These kind of details need to be properly documented in MetacatUI theme documentation.

MetacatUI will still ship with the default, nceas, goa, knb, dataone, and arctic themes, each of which will have its own metacatui-config.js file, HTML templates, and CSS. We may also have development config files handy, such as themes/arctic/metacatui-dev-config.js for quickly deploying a MetacatUI with the Arctic theme pointing at dev.nceas.ucsb.edu and CN Stage 2.

An example metacatui-config.js file for a custom theme may look like:

{
"baseUrl": "https://my-repository.com",  //The base URL where metacat is installed
"google-maps-api-key": "XXX", //Google Maps API key. If missing or false, will turn off all map features
"metacatui.css": "https://my-repository.com/theme/css/metacatui-custom.css", //Overrides the theme CSS file.
'templates/defaultHeader.html' : 'https://my-repository.com/theme/templates/defaultHeader.html', //Overrides the theme header HTML template
'templates/footer.html' : 'https://my-repository.com/theme/templates/footer.html', //Overrides the theme footer HTML template
"nodeId" : "urn:node:myREPO", //Configures the member node ID
"annotator" : true   //Turns on the semantic annotator feature
}

Open to feedback!

[amoeba]

This sounds pretty good to me and your implementation sounds spot on.

What's are the motivating factor(s) for doing this?

[csjx]

This sounds great @laurenwalker. I like the idea of consolidating the configuration. I particularly like the terse but adequate inline documentation of each property in the config object so end-users can edit it directly without necessarily referencing the documentation. Maybe we can put each comment line above each property and provide good whitespace to make the JS file super-readable? Of course, we should also provide more detailed documentation to explain the theming system, but seems like another ticket.

One question: what do you think about splitting repository-specific themes out into separate repos like metacatui-knb and metacatui-arctic, etc.? We'd need to think about how these get overlayed and subsequently upgraded, but I'm thinking that we might want the theming to be more modular. If a group wanted to customize the UI, they'd install the base MetacatUI, and then install one or more themes that would be configured in metacatui-config.js. Thoughts @mbjones @amoeba @datadavev @gothub too?

[mbjones]

This sounds good to me as well. Is there a way to get the default to be outside of the main metacat-ui install dir, such that a new install/upgrade that overwrites that whole metacatui dir wouldn't overwrite the metacatui-config.js? For example, if it were located by default in /etc/metacatui/metacatui-config.js, then the app could first check if that file exists, and if not, then copy itself there from the defaults in AppConfig. Then, the next time the app was upgraded, the config file would still be in place. Some potential issues with my proposal:

• what if the web user can't access or write /etc/metacatui?
  • maybe this could be a cascade of locations? First /etc/metacatui, then ./metacatui-config.js, then somewhere in $HOME, then give up and use defaults from AppConfig?
• what if AppConfig defines some new properties? I guess they stay at their default values until they are added to the site metacatui-config.js. Or, I guess AppConfig could go through and add defaults for any missing properties each time it starts (without overwriting the properties that are set there).
• if this is used, how would one set up multiple metacatui instances on one machine pointing at different configs (they would all find and use the same /etc/metacatui default, so that would be an issue

Overall though, this would be a big improvement over the current search for configuration options. SO I'd support it with our without further refinement.

[laurenwalker]

Thanks for the feedback everyone.

@amoeba - The motivating factors for this are:

• Currently, app configuration is spread across 3 different files and isn't very intuitive for new developers or product owners. Consolidating to one config file seems more consistent with how other software is designed

• MetacatUI is having growing pains. It started out with just a couple themes and then got adopted by NCEAS, DataONE, Arctic... and now ESS-DIVE and Salmon Data Task Force. We need MetacatUI to be a shippable product that can be quickly installed with custom configurations and a custom theme by someone who has no beginning knowledge of how the app works. They also need to be able to update it with the latest release without worrying about overwriting their custom theme files. Ideally, streamlining all this will create a product that others can install and maintain with minimal contact with our team.

@csjx - Yep, I agree that the config file should be super readable with lots of whitespace and comments explaining each config property.

I thought about theme-specific repos, too. I'm open to the idea. It would reduce the size of the MetacatUI release .zip file and would further tidy up and modularize the code base. Any other reasons?

@mbjones - I'm not sure how having a default config file location outside of the base MetacatUI directory would work without knowing custom URLs or Apache configurations. The config file will be pulled in via an AJAX request at the time of app load so it needs to be accessible via HTTP. I think most product owners will need to customize the config file URL in index.html when they install and upgrade MetacatUI.

We could make the default location something like metacatui-theme/metacatui-config.js, where the metacatui-theme directory is non-existent until the user creates it in /var/www/. They might have MetacatUI installed at /var/www/metacatui and their theme at /var/www/metacatui-theme. Their Apache config for www.my-repo.com would serve up the metacatui/index.html page and www.my-repo.com/metacatui-theme would house the metacatui-config.js file and all other theme files.) This is similar to how we have KNB configured now. However, this does require some Apache config knowledge from the product owner.

[csjx]

@laurenwalker - Yes, it will tidy up the code, and I've always found it odd that we ship branding content for different groups in Metacat, and now in MetacatUI. Let's say MetacatUI took off a bit, and we had 20 themes. It starts to get unwieldy. So, like other themed software, it seems like we should allow people to download or fork one or more themes - just the ones they want to modify, and make it easy to overlay them. Still needs some thought, but in the long run, I think we'll be glad we modularized it.

[laurenwalker]

@csjx - Yep, that makes sense.

[amoeba]

Thanks for laying out those two points @laurenwalker. I think those are two great reasons.

I went looking for examples of how other kinds of applications handle in-place upgrades and basically found nothing super useful. But I did think of WordPress, which historically had you:

1. Download a new updated ZIP
2. Replace just sub-folders (i.e. not the whole thing) within the deployed web app dir. i.e. You kept wp-content in place and replaced wp-admin and wp-includewhole sale

This differs from the current MetacatUI upgrade procedure only at step 2 which is currently just replacing the whole directory. Could something like this work? A revised directory structure could look like:

components
config <-- don't replace on upgrade?
css
font
img
js
themes <-- don't replace on upgrade?

Maybe only custom themes would go in the themes dir which simplifies the upgrade instructions. If custom and default themes were mixed we'd have to tell the user to only update sub-folders of the themes directory when they're upgrading and not to replace their custom themes (if they have any!).

Of course, WordPress eventually switched to an automated update procedure and also discourages people from self-hosting WordPress so maybe copying their upgrade procedure is not ideal.