Add a csv export feature.

[Mike-Heneghan]

CSV

First as a management command (like digest emails).

We want the following to be filterable by published orgs/services/whatever, for public consumption.

Services

id, name, description, url, phone, email, aliss_url, permalink, last_edited, organisation_id, organisation_name, organisation_permalink, categories, service_areas, location_ids

Locations

id, formatted_address, name, locality, region, postal_code, country, latitude, longitude, organisation_name, organisation_id, organisation_permalink

Services at location

service_id, location_id, service_name, service_permalink, formatted_address, organisation_id

name this services_at_location.csv

Organisations

Default queryset should be: Service.objects.values('organisation_id').distinct(), also with option to include unpublished.

id, name, description, aliss_url, permalink, url, twitter, facebook, phone, email, last_edited, service_names, service_ids, service_permalinks

Notes

String wrap, and comma separate categories and service areas using slugs. For locations comma separate the ids and/or permalinks.

Be careful to escape quotation marks in descriptions. Try testing against quotation mark and comma characters

[Mike-Heneghan]

Django has a csv library for streaming csv from a view https://docs.djangoproject.com/en/2.1/howto/outputting-csv/.

Interesting although I don't think the is the way we want to implement this function.

[Mike-Heneghan]

Using an import/export library and adding it to the admin: https://www.youtube.com/watch?v=vYIuvSwLsMc

[Mike-Heneghan]

Information on CSV export: https://djangotricks.blogspot.com/2013/12/how-to-export-data-as-excel.html

[Mike-Heneghan]

Maybe best approach is to try exporting one model as a csv and work from there.

Further information form a blog post about generating csv files from the backend: http://blog.aeguana.com/2015/12/12/csv-export-data-for-django-model/

[Mike-Heneghan]

Open Referral docs https://openreferral.readthedocs.io/en/latest/

[Mike-Heneghan]

• [x] Make the column headings lowercase and spaces as underscores.

[Mike-Heneghan]

• [x] Build the aliss URL (with slug)?
• [x] Build the permalink using the id of the object as a URL.

Could create a new term to determine how a variable is handled. Could write methods which generate a permalink for the model record.

[Mike-Heneghan]

To Do List:

• [x] Add the other models to be exported.
• [x] ~Comma separate the fields with lists?~
• [x] Update the services_at_location.csv output.
• [x] Visually check how characters are handled in fields such as ""
• [x] Write some tests for the export_csv command. (wrote checks rather than tests)
• [x] ~Make it possible to filter the data.~

[Mike-Heneghan]

To do list:

• [x] Merge in most recently updated master.
• [x] Look at using prefetch for related objects?
• [ ] Make sure the csv export only writes unique organisations?
• [ ] Send pr and discuss.
• [ ] More rigorously check?

[Mike-Heneghan]

For making the results of the CSV more accurate we could use .distinct(field to compare against) for the organisations. Although how do we ensure the unique organisation returned is the one we want? We could potentially order_by something like last_edited or only include organisations which have greater than 0 services?

[Mike-Heneghan]

To resolve the issue with dummy data being committed created a new branch and copied the code across.

[Mike-Heneghan]

Merged into master