geosolutions-it / pyfulcrum

Python module for Fulcrum Webhooks and Fulcrum forms backup
https://waffle.io/geosolutions-it/pyfulcrum
1 stars 2 forks source link

PyBackup fetch phase #5

Closed tdipisa closed 5 years ago

tdipisa commented 6 years ago

Records that passed the filter and validation phase need to be processed in order to retrieve data according to the event payload:

cezio commented 6 years ago

Record fields and contents will be retrieved along with form and fields meta information. The latters will be retrieved from the Fulcrum API only the first time a record will reference it.

Mind that Fulcrum allows to update records, so we should not assume data we have fetched once are up-to-date all the time.

cezio commented 6 years ago

The organization API key must be defined in the module configuration, to make the endpoint service perform API requests on behalf of the organization account.

PyFulcrum lib per se doesn't handle configuration. It's up to caller code how configuration is stored/managed. PyFulcrum API (ApiManager class) expects configuration to be passed as parameters to constructor.

PyFulcrum CLI has switches to provide configuration variables: https://github.com/cezio/pyfulcrum/tree/master/lib#pyfulcrum-cli

PyFulcrum web app also expects configuration: https://github.com/cezio/pyfulcrum/blob/master/web/README.md#configuration

cezio commented 6 years ago

File contents for signatures, photos, audio and video field types will be obtained and downloaded through the Fulcrum API and stored on the configured storage backend (local block storage, NFS, S3).

At the moment storage is implemented on top of regular file system. NFS/S3 can be configured easily to have local file system wrappers (NFS share can be mounted and accessed as fs by default, S3 requires user-level fs wrapper, like s3fs). That means any additional storage strategy is up to deployment configuration.

Storage can also generate urls if it's served with http server. All it needs is web server that can access storage dir and base url under which statics will be served: https://github.com/cezio/pyfulcrum/blob/master/web/README.md#api-configuration