search

telefonicaid / fiware-cygnus

A connector in charge of persisting context data sources into other third-party databases and storage systems, creating a historical view of the context
https://fiware-cygnus.rtfd.io/
GNU Affero General Public License v3.0
64 stars 105 forks source link

Script to remove old data in MySQL database #63

Closed fgalan closed 10 years ago

fgalan commented 10 years ago

As part of the resource folder, with some instructions on how to use it (e.g. how to install in cron).

frbattid commented 10 years ago

How useful would be all the databases and tables would be prefixed by "cygnus_"!

Working on some mechanism allowing to discard databases or tables not related to Cygnus... (there may exist on the same MySQL server many other non Cygnus stuff).

frbattid commented 10 years ago

It seems the easier way is to rely on the fact that any table (except for the ones related to Cygnus) has a "recvTime" column... At least, 99,99% of the times that will be true.

fgalan commented 10 years ago

Alternatively, the script could use a list of databases as parameter and consider that all the tables in these databases needs to be processed to remove old records.

frbattid commented 10 years ago

The database list does not like to me because such list may vary from time to time. I mean, it could be written in a file in order the script reads it. But, who is responsible for updating the list? If a human must manually check it in a regular fashion... that would imply to ignore the cleaning script and delete the data manually at the same time ;) On the contrary, if some kind of script is in charge of creating the list then, why not adding such script to ours? And how does that script looks like? Has it to iterate on the databases, looking for tables having the "recvTime" column and so on? That's just what we do :)

fgalan commented 10 years ago

Implemented in PR https://github.com/telefonicaid/fiware-connectors/pull/87