benchflow / data-transformers

Spark scripts utilised to transform data to the BenchFlow internal formats
Other
0 stars 0 forks source link

Enable the TTL to Alluxio Files #55

Open VincenzoFerme opened 8 years ago

VincenzoFerme commented 8 years ago

Since we use Alluxio, we want to have way to set the ttl for specific kind of information that we need for more time. This would we useful for checks like the one about the hosts, so that we can keep data about the hosts in Alluxio longer than other data. Or vice versa for some file we want a shorter TTL.

Currently waiting for the Python library implementing the Alluxio FileSystem API: https://alluxio.atlassian.net/browse/ALLUXIO-52?focusedCommentId=12113&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel