search

dathere / datapusher-plus

A standalone web service that pushes data into the CKAN Datastore fast & reliably. It pushes real good!
GNU Affero General Public License v3.0
26 stars 18 forks source link

Issue when migrating from ckanext-xloader: column "job_key" of relation "jobs" does not exist #149

Open pdelboca opened 4 days ago

pdelboca commented 4 days ago

Describe the bug Database migration runs successfully but they do not create the required columns. Running datapusher-plus will raise an error: column "job_key" of relation "jobs" does not exist

To Reproduce Steps to reproduce the behavior:

  1. Have an instance with ckanext-xloader running
  2. Disable ckanext-xloader
  3. Install and enable datapusher-plus
  4. Run datapusher-plus database migrations (ckan datapusher init-db and ckan db upgrade -p datapusher_plus)
  5. Try to push data to the datastore

Expected behavior Running database migrations will successfully create the database tables or raise an error if conflicting tables already exist. (Maybe this is a error in CKAN)

Additional context So I have an instance with ckanext-xloader running and I'm migrating to datapusher-plus. Deactivating ckanext-xloader left the jobs table in the database. This is causing some inconsistent behaviour in CKAN since when I'm executing ckan datapusher init-db I have a Success confirmation message but the table is not created. I only have a leftover of the jobs table that ckanext-xloader created.

As I mention, this is probably an error in CKAN, but I wanted to create an issue here since it may happen to other users trying to migrate to datapusher-plus.

pdelboca commented 4 days ago

@tino097 @jqnatividad I'm wondering if this is something you would like to handle in a database migration (like you have for checking aps_jobs_id and logs.id) or if this should be handled by a migration on the project dealing with this.

tino097 commented 4 days ago

Thanks again @pdelboca. This is good question and we should look into asap.

Additionally this comment is suggesting me that we should drop old Datapusher from CKAN core and move the templates and other code into DP+/Xloader