search

CRREL / GRiD-API

9 stars 3 forks source link

Long running tasks should provide callback option #18

Closed chambbj closed 7 years ago

chambbj commented 7 years ago

A new, optional parameter callback_url should be supported, that reports back to the caller when processing has completed (success or failure) for long running tasks such as export generation. Response should be a JSON-encoded export object, which already includes pk of the export, task_id, and status, along with full information of the exported dataset, thus obviating the need to reach back in through the API to gather such data, at least for successful exports. For failed exports, the user can ping the /task endpoint to retrieve any available task_traceback.

chambbj commented 7 years ago

Some models to consider:

AlexMountain commented 7 years ago

I've implemented django-rest-hooks for v3 of the API. Currently it supports subscribing to newdata notifications and export completion notifications, but at a global level instead of individual.

Is that sufficient or should we discuss callbacks at an individual export parameter as well?

If there are any other GRiD features that you would like callbacks for feel free to include them here.

chambbj commented 7 years ago

newdata and export completion were the obvious use cases.

Not sure what you mean by "global". Like every event global? Or every event related to my registered application? If the latter, I think that's fine. The application can filter out which AOI received newdata, which export completed, etc. If you mean truly global, that seems like overkill.

AlexMountain commented 7 years ago

Yeah sorry let me clarify. It only applies to your user. So if you register for a newdata.loaded event, you'll get a notification any time one of your AOIs intersects a new data load. Likewise if you register for export.complete, you'll be notified for all of your own exports that finish (success or failure).

AlexMountain commented 7 years ago

Webhook callbacks have been added to v3 via the hook endpoint. They support global events for the user or can be narrowed down to only trigger for specific AOIs for the user.