PyBackup module APIs

[tdipisa]

A set of methods need to be exposed by the PyBackup module to gain access and export the contents stored in the PostgreSQL database:

• [x] list of saved forms, with summaries about related records
• [x] list of saved records with field names and values, whose contents will be similar to those available through Fulcrum Data Shares.
• [x] The list of saved forms and records will be provided in JSON format as default but, if required by the caller, also the exports of saved records needs to be supported as CSV, GeoJSON, KML or ESRI Shapefiles

notes:

• Some checks are probably needed to return records in a specific format (for example geographical information may be missing)

• The methods offer the option to set filters by user id, form id and date ranges to narrow the result set.

[cezio]

I've added cli application that also uses pyfulcrum api, so i can test data more efficiently. It outputs values from api:

list of saved forms, with summaries about related records

Form: id=ec2fedff-2ba3-492c-a5da-63be7d026dda name=Copy of Wetland Classification records_count=3950
Form: id=dd7449cc-ef64-456f-9dc5-f2eca4afc07d name=test app records_count=4

list of saved records with field names and values, whose contents will be similar to those available through Fulcrum Data Shares.

Record: id=b3d2e4dc-545c-4ebd-a53a-a6be7bad0dfb status=fff form_id=dd7449cc-ef64-456f-9dc5-f2eca4afc07d
 field 0f50: deweeee
   value: Hdhj
 field c2aa: eeeee
   value: yes
 field aeb7: ewrewerw
   value: 8688
 field 4e91: qwqwq
   value: {'other_values': [], 'choice_values': ['cccc', 'ddd']}
 field ac03: Photos
  photo 05e61326-9046-4df4-9478-40e94b10414e: {'path': '/mnt/work/cezio/geosolutions/repos/pyfulcrum/lib/examples/_run/storage/dd7449cc-ef64-456f-9dc5-f2eca4afc07d/b3d2e4dc-545c-4ebd-a53a-a6be7bad0dfb/photo_original', 'url': None}

The list of saved forms and records will be provided in JSON format as default but, if required by the caller, also the exports of saved records needs to be supported as CSV, GeoJSON, KML or ESRI Shapefiles

I assume that spatial formats should return records that contain geolocation. How should shapefile be returned? It's build up from several files. Should I return archive containing them all?

[tdipisa]

@cezio an archive that contains all of them is fine.

[cezio]

For the record: Shapefile has serious limitation on field name and contents length, so some properties may have truncated names/values. This is known limitation, so we should advertise GeoJSON as more suitable format.

[cezio]

API is described in Readme, also there's small CLI app, that wraps API functionality into simple command line calls: https://github.com/cezio/pyfulcrum/tree/master/lib#pyfulcrum-cli