search

benchflow / analysers

Spark scripts utilised to analyse data and compute performance metrics
Other
0 stars 1 forks source link

Enable Alluxio for analysers #63

Open Cerfoglg opened 8 years ago

Cerfoglg commented 8 years ago

Make it so the scripts save data on Alluxio, and retrieves it from it if possible before trying with Cassandra.

The filename structure should be the Cassandra primary key.

Some parts were Alluxio would be useful, for example: https://github.com/benchflow/analysers/pull/75/files#r65813797, https://github.com/benchflow/analysers/pull/75/files#r65814542

A place where we used Alluxio and though about the name to store and retrieve the file: https://github.com/benchflow/data-transformers/pull/85/files

VincenzoFerme commented 8 years ago

@Cerfoglg i guess you are not gonna be able to add Alluxio for the scripts, so please:

  1. Document the idea we discussed about adding Alluxio as a comment in this issue and in your thesis.
  2. Poi in this issue the branch of your personal fork were you were working on adding Alluxio to data-transformers and analysers.
  3. Add whatever information can be useful for other people that will work on adding Alluxio to the scripts.