OpenDataServices / cove

CoVE is an web application to Convert, Validate and Explore data following certain open data standards - including 360Giving, Open Contracting Data Standard, IATI and the Beneficial Ownership Data Standard
http://cove.opendataservices.coop
Other
43 stars 11 forks source link

IATI CSV doesn't parse #815

Closed tobybatch closed 7 years ago

tobybatch commented 7 years ago

@Bjwebb when using the CLI interface to CoVE CSV doesn't get parsed. I'm on the master branch and the csv is here: http://ci.openag.org.uk/fixtures/basic_iati_unordered_valid.csv

root@058d61927f7e:/opt/cove# echo $DJANGO_SETTINGS_MODULE 
cove_iati.settings
root@058d61927f7e:/opt/cove# python manage.py upload /var/tmp/basic_iati_unordered_valid.csv 
/data/035bb379-c40a-4eec-818d-0258a1d5fe63
root@058d61927f7e:/opt/cove# ls media/035bb379-c40a-4eec-818d-0258a1d5fe63/
basic_iati_unordered_valid.csv

So no errors and just a CSV output, I was expecting XML.

Bjwebb commented 7 years ago

Ah, conversion only happens when that page gets loaded atm.

Short term fix, load the page e.g. using curl curl 'http://localhost:8000/data/035bb379-c40a-4eec-818d-0258a1d5fe63'

Longer term, we want a command that actually does the conversion, possibly based on the OCDS command line tool work https://github.com/OpenDataServices/cove/pull/770

tobybatch commented 7 years ago

@Bjwebb Hi Ben, how is progress on this?

Bjwebb commented 7 years ago

Sorry haven't had chance to look at this. Does the short term fix work for you?

tobybatch commented 7 years ago

I could use that but it would result in the total number tools in the tool box working as expected to one. I'm not running the web interface on cove at all, we start the docker, run the CLI import and exit the docker so I'd need to change the wrapping of the service.

If it's a ball ache then let me know and I'll replace it with a socket rather than CLI version.

Bjwebb commented 7 years ago

Makes sense. Our next IATI sprint starts next week - are you okay to wait another couple of weeks for this to be finished?

tobybatch commented 7 years ago

I still don't have a working geocoder so yes.

On 30 August 2017 at 10:50, Ben Webb notifications@github.com wrote:

Makes sense. Our next IATI sprint starts next week - are you okay to wait another couple of weeks for this to be finished?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OpenDataServices/cove/issues/815#issuecomment-325941885, or mute the thread https://github.com/notifications/unsubscribe-auth/AAV9b0-hL0VaJLC5007aUinrxqE8FmTuks5sdTB-gaJpZM4O3Uks .

-- Toby Batch Lead Developer, NeonTribe Internet user experience designers and programmers

Web: http://www.neontribe.co.uk Mobile: 07968505897 Google Chat: tobias@neontribe.co.uk Skype: toby.batch

edugomez commented 7 years ago

@tobybatch I'm currently working on a CLI for IATI similar to the one for OCDS (https://github.com/OpenDataServices/cove#command-line-interface). Just checking that an output folder with the conversion to XML plus a json file with validation and rule set errors would do for you.

Bjwebb commented 7 years ago

IATI CLI interface now exists - https://github.com/OpenDataServices/cove#command-line-interface - so I think this can be closed.