Implement records expiration for HDFS

telefonicaid / fiware-cygnus

A connector in charge of persisting context data sources into other third-party databases and storage systems, creating a historical view of the context

https://fiware-cygnus.rtfd.io/

GNU Affero General Public License v3.0

65 stars 105 forks source link

Implement records expiration for HDFS #1915

Open manucarrace opened 4 years ago

manucarrace commented 4 years ago

In the same line as in MySQL it should be possible to define an expiration policy for other sinks. Sinks to analize:

[x] Postgis: done in PR https://github.com/telefonicaid/fiware-cygnus/pull/1918
[x] Postgres: done in PR https://github.com/telefonicaid/fiware-cygnus/pull/1918
[x] CKAN: done by expirateRecordsCache in CKAN backend
[ ] HDFS
[x] STH: done in mongobackend when create collection
[x] Mongo: done in mongobackend when create collection

manucarrace commented 4 years ago

https://github.com/telefonicaid/fiware-cygnus/blob/5e63327cecadb2dfc650b83ca3c7a60ae5968bb9/doc/cygnus-ngsi/installation_and_administration_guide/backends_as_sth.md