search

vishalmhjn / pneuma_treatment

Treating noise and anomalies in the vehicle time-series data captured by drones
MIT License
5 stars 1 forks source link

Pneuma Data formatting problem #1

Open ahmedshahnisar opened 7 months ago

ahmedshahnisar commented 7 months ago

Hey Vishal, really great work. I ran into some weird issues with the code, I tried using the drone 7 data file download from Pneuma and put the .csv file into a new folder 'input data' in the data folder and ran the data_formatter.py. To generate the output file, I made a folder 'long' in in data folder and ran the code. Two files were generated. But the data was messed up, the data went into columns it wasn't supposed to. lat, lon came under v, a_x, and similar for other columns too.

I am attaching a screenshot of the same. Screenshot 2023-11-18 172326

Can you give any fix for it. Thanks

vishalmhjn commented 5 months ago

Thanks for pointing it out. Yes, this will happen, if the format of the input raw data has updated over time. To address this you will have to check how the items are ordered in the raw data and then make changes to the source code, specifically starting from the line below, so that the relevant items are added to corresponding lists. https://github.com/vishalmhjn/pneuma_treatment/blob/5d64535782f665d1022f0adaaa1a29d93761ba2c/src/data_formatter.py#L20