search

UTSASRG / Scaler

GNU General Public License v2.0
4 stars 0 forks source link

Improve experiment tool #74

Closed GammaPi closed 1 year ago

GammaPi commented 2 years ago

1. The current excel format is inconsistent with the paper latex format.

For example, in paper the figure looks like this: image

But my excel looks like this: image

The unaligned format makes it super difficult and error-prune to update experiment part.

The correct workflow is:

  1. Maintain a table in excel identical to the paper's format.
  2. Use Latex Table Generator to convert excel table to latex.
  3. Paste generate latex code to paper
  4. For any updates, we should update the excel rather than the paper and repeat step 1 to 3.

2. Current experiment environment and artifact integrity is not automatically checked.

I previously saved all experiment artifacts and moved Scaler repo to XtTech Cloud to avoid accidental data loss, data corruption. However, data integrity and testing environments are still unchecked to this date.

Besides, not all artifacts are saved and well-organized.

3. Current benchmark scripts

We should make more real applications testable by automated scripts.

Currently we can only automatically test mysql, memcached, redis, PARSEC.

The automated scripts for MySQL, Memcached and redis is good, but the compiled artifacts are not saved.

GammaPi commented 1 year ago

Task finished. We will use new benchmarktoolkit for future tests.