Open valerikambal008 opened 6 years ago
metaodi
commented
6 years ago Did you follow the steps in the README in the section "Running the harvest jobs"?
There are a few things you can check:
Hope this helps.
valerikambal008
commented
6 years ago Stefan.
I tried all you wrote days ago. Now I got issue: I harvest gsa.gof/data.json and in production.ini
Json type is there uncommented, but somehow it is not seen in browser as harvest source type
(only CKAN ). Maybe it is matter of java script or webhlper.py ninja2 html part.
No idea why ithis is not there. Maybe you jnow?
Another option to try paster source type url command
Thank you for help.
Val
From: Stefan Oderbolz notifications@github.com Reply-To: ckan/ckanext-harvest reply@reply.github.com Date: Tuesday, May 15, 2018 at 3:46 PM To: ckan/ckanext-harvest ckanext-harvest@noreply.github.com Cc: valerikambal008 valerik@ekuber.com, Author author@noreply.github.com Subject: Re: [ckan/ckanext-harvest] cKan 2.8 Harvester (#322)
Did you follow the steps in the README in the section "Running the harvest jobs"?
There arema few things you can check: Is there any helpful output on the gather_consumer or fetch_consumer jobs? Are you using redis or RabbitMQ? Check their queues and/or logfiles. Are they started? Try to run a harvest job using "run_test" instead of "run" Hope this helps.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
metaodi
commented
6 years ago @valerikambal008 just to confirm: is your original problem solved or does the harvester still not run?
I don't fully understand your follow-up question: what do you mean with "Json type is there uncommented, but somehow it is not seen in browser as harvest source type". Did you add json to the list of plugins in your production.ini and don't see know a JSON harvest type in the harvester GUI?
Which harvester do you want to use to harvest JSON? ckanext-harvest on its own does not provide a harvester for JSON. So you need another extension for that (e.g. ckanext-dcat)
valerikambal008
commented
6 years ago Stefan.
YES, you right, needed json_harvest plugin, previous question was about formatting and confused me.
Help me with json_harvest github url. I tried one for public health .gov, run python setup.py got
Plugin not found exception, as I heard gsa.gov do not use dcan,, so I don’t know then what json_harvest I need.
Val
From: Stefan Oderbolz notifications@github.com Reply-To: ckan/ckanext-harvest reply@reply.github.com Date: Thursday, May 17, 2018 at 2:58 PM To: ckan/ckanext-harvest ckanext-harvest@noreply.github.com Cc: valerikambal008 valerik@ekuber.com, Mention mention@noreply.github.com Subject: Re: [ckan/ckanext-harvest] cKan 2.8 Harvester (#322)
@valerikambal008 just to confirm: is your original problem solved or does the harvester still not run?
I don't fully understand your follow-up question: what do you mean with "Json type is there uncommented, but somehow it is not seen in browser as harvest source type". Did you add json to the list of plugins in your production.ini and don't see know a JSON harvest type in the harvester GUI?
Which harvester do you want to use to harvest JSON? ckanext-harvest on its own does not provide a harvester for JSON. So you need another extension for that (e.g. ckanext-dcat)
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
Trying to run Harvester on cKan 2.8 and its not pulling any datasets, it keeps saying Running on GUI & we don't see any datasets even after running it for few hours. We have been looking at the various log files & outside of a few wsgi::error notices, nothing seems to have any true errors. Appreciate if any one can help us.