POC of making the Sponsors app less dependent on Fluent

[jmurty]

As part of our survey of GLAMkit features and their (inter-)dependencies we are looking into ways to reduce the tight coupling of GLAMkit components, and the overall dependencies of these components. See https://github.com/ixc/ixc-tech/issues/4

As part of this work we plan a proof of concept (POC) effort to restructure the Sponsors (glamkit_sponsors) app and related internal apps to separate out the key features of the app and those features tied into the Fluent/CMS system.

The goal is to find out how, and if, we can make the Sponsors app available in projects without slurping in the full weight of Fluent Contents/Pages + Django Polymorphic + MPTT + etc etc etc. To help inform how we might then go about doing the same thing for other more complex and tightly-coupled apps.

Specifically, we want to:

• leave only core features of the Sponsors app – models to capture and display information and logos for site sponsors/partners – in the main app location, as would be activated on a site with glamkit_sponsors in INSTALLED_APPS
• move the Fluent-specific features of the app into an app/module that can be installed separately (or not), something like glamkit_sponsors.fluent. This app would include any page types, plugins, templates, and other features with Fluent Contents/Pages dependencies
• make the new glamkit_sponsors.fluent app a portable app in its own right, so that any useful abstract/base classes can still be extended or replaced in downstream projects
• find any internal GLAMkit dependencies used by Sponsors which bring in Fluent etc dependencies when we don't want them, and restructure these internal dependencies to avoid doing so
• run the restructured Sponsors app in a POC project showing that you can install it in a lightweight way – without the Fluent features and dependencies – and have it actually work
• figure out if there is any way to do all of this in a way that could be applied to existing sites without major migration work, for example can we keep things working despite moving model locations and DB migrations etc etc.

Sponsors is a good candidate for this POC since it is by far the simplest app we have and isn't as widely used as the others.

The main problems I expect us to face are:

• how deep into GLAMkit's internal supporting libraries will the changes reach when we try to remove dependencies on Fluent? If it turns out we cannot do this for Sponsors without also restructuring half of GLAMkit we need to abort and rethink.
• how to maintain backwards compatibility when models and particularly their DB migrations are relocated? Is backwards-compatibility possible using standard migrations and clever techniques like those documented for making legacy apps "portable"? If not, is it at least possible using some manual DB update commands, or anything which avoids having to burn the DB to the ground and start over?

[markfinger]

james [3:10 PM] @mark It's okay, we mostly re-stated the same things as previously. From my perspective, we have some promising next steps:

• list the parts of GLAMkit we would like to use in non-GLAMkit projects
• try and use those parts in a non-GLAMkit project to see what breaks, and how difficult it would be to make it work

The mechanics of how we do this are up for debate: pip install all of GLAMkit and only install the apps you want, or copy/paste the apps you want and adjust freely to become project-specific forks, or...

But the general idea of trying it to see is I think a reasonable first step

And I think doing it this way, GLAMkit apps in a non-GLAMkit project, is likely to give faster and clearer results than trying to decouple things in GLAMkit in general without a very clear goal (edited)

mark [3:14 PM] Cool, sounds good.

tailee [3:16 PM] we might end up replicating much of the glamkit settings in a project just to get it to run at which point it's just another glamkit project (assuming it's a start from scratch project)... hopefully the glamkit settings are mostly grouped by purpose/app so it shouldn't be too hard to disable a bunch of stuff, or find all the settings you need to enable a particular thing...

[3:18] testing publishing/events/pages separately and in various combinations with each other are probably the ideal candidates for decoupling... though with publishing needed as a base model for anything publishable (events/pages) I suspect that getting them running without publishing will mean simply removing publishing completely rather than decoupling it and making it optional

mark [3:19 PM] I suspect we'll also need a lot more settings for FK and M2M relationships between parts of glamkit.

[3:20] If there's not already some convention for icekit settings, it might be worthwhile suggesting one. Eg: ICEKIT_ or simply dumping everything into a ICEKIT dict (edited)

tailee [3:22 PM] using settings to define FKs and M2Ms (swappable models) makes me nervous :smile:

mark [3:28 PM] What's the alternative for replacing parts of icekit?

[3:29] I have no qualms with copying the entirety of an app in and hacking on it, but that's another thing that you've indicated we should avoid. What are the alternatives? (edited)

tailee [3:44 PM] @mark I think the initial goal is to decide on which components from GLAMkit (as a library) are interesting and might be useful to use on their own in other projects (e.g. publishing, events, page tree, modular content), and then find out if those components work independently of each other and what will be required to make them work independently of each other... so we can then for example pip install django-icekit into any project and make use of its events app without also having to use publishing and the fluent page tree etc... I don't think the goal is (atm) to find another way to alter the way one piece GLAMkit works for a single project (e.g. by the current recommended way, forking that as a "portable app" into the project codebase, or a new way e.g. swappable models or abstract models only in GLAMkit and concrete always in the project)... a separate issue is looking at ways to make GLAMkit's concrete models/implementations flexible enough that we avoid the need to fork or change them for specific projects (e.g. perhaps with JSONField in Django 1.9+ removing the need to fork a whole app/plugin just to store some extra project specific data on a model)

mark [3:45 PM] Fair enough

tailee [3:56 PM] @mark ideally I think we are looking for ways that help us build new projects faster... one way might be in having GLAMkit as a library of features we can pick and choose from without needing to create a full blown everything enabled GLAMkit site (and that's what we're trying to gauge first above)... another might be to copy reference implementations (from GLAMkit) into the project and hack on it (which could make maintenance a nightmare if we have 10 slightly different publishing implementations to maintain), or use more abstract and fewer concrete models in GLAMkit and require projects to implement their own concrete models (which might still leave us with 10 slightly different publishing implementations, and a balancing act between upgrading the library's abstract models then having to create per-project migrations that match the GLAMkit model changes)...