search

BEbillionaireUSD / Hades

12 stars 3 forks source link

Reading raw data - Incorrect paths? #2

Closed mmantyla closed 5 months ago

mmantyla commented 12 months ago

I am trying to read the raw data but I am having a difficulty. Looking at the preprocessing code at https://github.com/BEbillionaireUSD/Hades/blob/main/hades/preprocess/get_chunks.py#L60 it points to folder ./raw_data/wk_task_data/ from which is later in the file used for loading data. Here https://github.com/BEbillionaireUSD/Hades/blob/main/hades/preprocess/get_chunks.py#L31

However, such a path does not exist from the zip-file I downloaded from Zenodo. https://zenodo.org/records/7609780

Is there some action that should be run before this preprocessing?

BEbillionaireUSD commented 12 months ago

Thanks for your interest.

wk refers to the workload (i.e., svm, sort, etc.) task refers to the fault type.

I've merged all log files into the '._drain_labeled.csv' to streamline the log parsing. The processing code is outdated.

You can directly use "labeled_csv/_kpi_csv" and "labeled_csv/_log_csv". Or, for replication, run $ cd codes && python run.py --data ../data/chunk_10

BEbillionaireUSD commented 5 months ago

Hopefully my reply has answered your question.