add configuration for it within the CKAN system, so that dpp would:
get notified when a new file is uploaded
run a fixed set tasks on each file
publish the results to the same dataset
This task is about collecting use cases and come up with a high level implementation plan for a potential Data Package Pipelines CKAN extension. The main idea behind it should be to make easier to integrate ETL pipelines into CKAN, but there might be the chance that is difficult to make it generic enough and that users are better served with plain pipelines using the proposed CKAN processors (see other issues)
OriHoch
commented
6 years ago
Description
From @akariv :
This task is about collecting use cases and come up with a high level implementation plan for a potential Data Package Pipelines CKAN extension. The main idea behind it should be to make easier to integrate ETL pipelines into CKAN, but there might be the chance that is difficult to make it generic enough and that users are better served with plain pipelines using the proposed CKAN processors (see other issues)
Tasks
Estimate
2