search

wfth / wo

0 stars 0 forks source link

Populate the search database with real data #3

Closed aiwilliams closed 7 years ago

aiwilliams commented 7 years ago

We have a collector pulling things into a sqlite database. Move some of that into the database structures developed in #2.

isaacjwilliams commented 7 years ago

We've decided to create an Elixir mix app that will draw data from the Collector DB and use the Ecto library to insert it into the database of this Phoenix app, populating the regular tables and the search tables for sermons and sermon series.

We've decided to make this converter a separate Elixir app instead of expanding Collector or creating an all new Elixir app that replaces Collector for a few reasons:

  1. Intermediate data transformation (decoupling of Collector's schema and WO's schema)
  2. Use of Ecto + Mix (hopefully, using Ecto and Mix as libraries separate from Phoenix will help us understand them better)
  3. Use of Mechanize for site crawling (if we replace Collector with an Elixir app, we'll lose Mechanize)
isaacjwilliams commented 7 years ago

Converter should meet these requirements now. Now we can move on to actual searching!

aiwilliams commented 7 years ago

Yah!!!