Preprocessing: Combine all preprocessing steps in one method

[Lumik7]

In preprocessing.py create a method preprocess() which calls all required preprocessing steps. The method stub is already in preprocessing.py. This method should be called from make_dataset.py which first downloads the data to the data/raw folder and then applies the preprocessing steps and saves them to data/preprocessed.

[rmitsch]

@Lumik7 @MoBran Following changes in the latest commit:

• Existing functions are split in two classes (Preprocessor andDatasetDownloader) for clarity. If you prefer the original code structure, i can revert that before merging.
• All trips under 10 minutes long are dropped.
• All trips w/o annotations are dropped.
• Preprocessor.preprocess returns a dictionary with tokens as the first-level key. Each token points to another dictionary holding (1) a list of dataframes (same format as returned from the original get_data_per_token(...)) accessible with the key "dataframes" and (2) a list of resampled sensor data accessible with the key "resampled_sensor_data".

If there are no objections, I'll merge the current status into master and close this issue.

[Lumik7]

I agree, but could you update the notebook, so that it is compatible with the code changes?

[rmitsch]

Sorry, forgot about that. Updated now. Should I merge?

[Lumik7]

Thanks, yes I'm fine with it.

[Lumik7]

Normalize to same hertz rate should also be included in the Preprocessor.preprocessing() function @MoBran .

[Lumik7]

close this issue as normalization is included in preprocessing