How to account for empty/null value readings for some of the sensors

[Amit12690]

What's the best way to preprocess the data if some of the sensor readings have null/invalid values ?

[guillaume-chevalier]

@Amit12690 Good question. Possible solutions:

• You may want to do interpolation
• Or else skip the sensor readings' time step and to add a "time delta" parameter as a feature.
• Another solution would be to just fill the NaN values with zeros (or any other default value suited to your need).
• Or else you may fill the NaN with zeros and even use heavy dropout-like masks during training in the inputs to prepare for the eventuality of having many empty values as a data augmentation.
• You could try other data augmentation and featurization techniques as well depending on your problem.

It depends on your data, project size, and how important it is to get very good accuracy. There is no "one size fits all" answer here.

[Amit12690]

Thanks for the detailed answer @guillaume-chevalier. I will explore them and check what works the best .