okfde / ckanext-offenedaten

CKAN extension for OffeneDaten.de (theme/UI & harvesters)
http://www.offenedaten.de
4 stars 3 forks source link

Data flow for crawled data #44

Open mattfullerton opened 9 years ago

mattfullerton commented 9 years ago

Currently reviewed and rejected data is in the DB as deleted, so that it should not be added again. But that would not work with current import scripts as new IDs are generated. A check needs to be done against URL. And then there is the question of whether to crawl directly into the intermediate DB, or to CSV (as now) or directly into CKAN.

I suspect we will never do this :-)