Problems with Datapusher and Datastore

[LukusV]

Hi, I've added this to docker-compose.yml:

CKAN_DATASTORE_WRITE_URL : postgresql://ckan:ckan@db/datastore CKAN_DATASTORE_READ_URL: postgresql://datastore_ro:ckan@db/datastore CKAN_DATAPUSHER_URL: http://datapusher:8800 DS_RO_PASS: ckan

datapusher: container_name: datapusher restart: always image: clementmouchet/datapusher ports:

• "8800:8800"

And I've added datastore and datapusher plugins to ckan-entrypoint.sh. But datapusher and datastore don't work

Datapusher logs:

Job "push_to_datastore (trigger: RunTriggerNow, run = True, next run at: None)" raised an exception Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/apscheduler/scheduler.py", line 512, in _run_job retval = job.func(*job.args, job.kwargs) File "/usr/src/app/datapusher/jobs.py", line 222, in push_to_datastore resource = get_resource(resource_id, ckan_url, api_key) File "/usr/src/app/datapusher/jobs.py", line 178, in get_resource 'Authorization': api_key} File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 94, in post return request('post', url, data=data, json=json, kwargs) File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 49, in request return session.request(method=method, url=url, kwargs) File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 457, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 569, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 407, in send raise ConnectionError(err, request=request) ConnectionError: ('Connection aborted.', error(111, 'Connection refused'))

[piersoft]

update? news?

[nicola-inchingolo-aslbat]

Use the sfbelaine/datapusher docker image and add volumes in the docker-compose.yml:

volumes:
  ckan_it_datapusher:
  ckan_it_home:

services:
  ckan:
    ....
    ....
    restart: unless-stopped
    volumes:
      - ckan_it_home:/usr/lib/ckan/default

  datapusher:
    container_name: italia-ckan-it-datapusher
    image: sfbelaine/datapusher
    ports:
      - "0.0.0.0:8800:8800"
    restart: unless-stopped
    volumes:
      - ckan_it_datapusher:/opt/datapusher

Then after running docker-compose up -d replace these files in the ckan and datapusher volumes:

/var/lib/docker/volumes/ckan-it_ckan_it_home/_data/src/ckan/ckanext/datapusher/logic/action.py
/var/lib/docker/volumes/ckan-it_ckan_it_datapusher/_data/datapusher/jobs.py

with (see ###ASLBAT### comments)

https://github.com/nicola-inchingolo-aslbat/ckan/blob/ckan-2.6.8-datapusher-reverse-proxy/ckanext/datapusher/logic/action.py https://github.com/nicola-inchingolo-aslbat/datapusher/blob/ckan-2.6.8-datapusher-reverse-proxy/datapusher/jobs.py

Then configure in the ckan.ini:

ckan.datapusher.callback_url_base = http://ckan:5000
ckan.datapusher.url = http://datapusher:8800/
ckan.datastore.write_url = postgresql://ckan:ckan@db/datastore
ckan.datastore.read_url = postgresql://datastore_ro:datastore@db/datastore

and restart ckan and datapusher (docker-compose restart ckan datapusher)

see: italia/ckan#7 and italia/datapusher#1

See comments for SSL_VERIFY variable. To see log messages in datapusher use logging.error(msg)