search

davemacdo / composer-diversity

a frontend for the Composer Diversity Project
https://davemacdo.github.io/composer-diversity/
9 stars 2 forks source link

Moving data #32

Open davemacdo opened 6 years ago

davemacdo commented 6 years ago

Content management

Cleaning up the data is the first step to the plan of moving to a new storage location. The site is currently served on GH Pages. I would like to avoid having to run a server application for this project, but I'm open to it if it becomes necessary.

My proposal is to move the data into a Google Firestore database and develop a simple JavaScript CRUD interface for the research team. Eventually, we could add a form for user submissions as well. I'm open to a CMS as well. @captbaritone suggested Netlify, which would keep all the content in flat files on Github while still having a friendly backend. I'm a big fan of WordPress, but I would rather not deal with setting it up and managing it for this project.

Options

The main options I see are taking an existing CMS like Netlify or WordPress and customizing it for this project, or building a simple CMS-like backend from scratch. My intuition is that customizing an existing package would take about as much time as building a new one, but I am new to this. Thoughts?

@gtwright also mentioned an external search platform like Algolia. I think whichever way we go here will make it easier to add something like that down the road, but in the interest of handling the backend updating, I think I'd like to focus on these other options for now.

gtwright commented 6 years ago

Thanks for the summary. All of those factors make sense to me.

As far as how much setup time is required for Wordpress vs. custom, I'd say the big question is how simple the user interface can be for the research team—present and future. If you are confident they don't need much, then custom becomes an attractive option. One of the biggest advantages for Wordpress is that the interface will be familiar to a lot of folks, so no matter how many new people you add to the research team, training and managing them shouldn't suck up a ton of your time.

FWIW, I think it should be relatively quick to get the custom post type/fields/taxonomies in Wordpress to get this up and running. (I do that semi-regularly, so I'm happy to take that on) The bigger question would be importing the current dataset. I'm going to take a look into that myself and see how much of a challenge I think that will be.

(And as far as hosting/managing goes, I have a bulk plan with Flywheel and would be happy to donate a slot in that plan for this project, if you want to go that route.)

On the Algolia front, it does have Wordpress plugin, so it's easy to go that route. But Algolia is pretty straightforward to integrate in general, so I don't think the plugin in a massive advantage.

gtwright commented 6 years ago

I was curious, so I did a quick bit of reformatting on the giant JSON file and indexed it with Algolia, just to give an example. Here's the live demo and here's the repo.

davemacdo commented 6 years ago

Wow. The speed of the Algolia search is great! I think you've convinced me. I'll have to look into the costs associated with it.

A customized WordPress install for the data could be very useful. I've done a small bit of WP work before, and I know @ianring is a PHP/MySQL jockey. And if we wanted to make something even more custom down the road, we could just use the REST API for editing. And if we ever wanted to make a really big change, getting data out would be really easy as well.

Could these steps be taken separately, or would they need to happen at the same time? Would it make sense to have a separate repo for the server side?

gtwright commented 6 years ago

Cool. Glad you like the experiment. The free tier for Algolia is rather generous and will probably do the trick here. The only practical limit is that multi-user access doesn't come on the free tier. They do have an Open Source program that gives a free upgrade to the next tier, which does allow multiple users. Seems likely that this project would get approved. In the meantime, the free tier is solid.

I did a little bit of WordPress experimenting, too. I'll probably have something to show tonight. I'm at least convinced that getting the current dataset into WP won't be dealbreaker.

I'd agree with you on splitting out a server-side repo, especially if we're going the Wordpress route.

davemacdo commented 6 years ago

Sounds good. The more I think about it, the more I like the idea of using a WP backend with something like the current frontend. The user management will be very handy. The easier and friendlier we can make the editing workflow, the faster we can make the database better, and the frontend app more useful.

ianring commented 6 years ago

joining this discussion, catching up