MesNet real-time estimation issues w.r.t ReplicationPad1d after Conv1D

[manymuch]

In MesNet, the ReplicationPad1d accoding to PyTorch.org is a double side padding for the input. The two padding block in the ConvNet yields the final output been "shift right" 8 pixel. Or in other worlds, the measurements_covs[i] actually derived from information of 8 future signals, which is about 80 ms in the future if IMU signal is measured in 100HZ. I believe it is not a big deal if we carefully fix the padding issue to reproduce similar results.

[ajay1606]

@manymuch I have tested with a Custom dataset in KITTI format and the results look pretty good. But as you mentioned in your comment, there are some parameters needed to tune to make it work in real-time well. Are able to test in real-time data? if so could you please share some suggestions?

Appreciate your response.

Regards Ajay

[scott81321]

@ajay1606 Don't abuse me and be grateful for your luck. There are initial parameters in main_kitti.py and YOU will have to find out, via testing, which ones have the most effect. The effect from each noise covariance parameter is not equal. The recommendation from Martin Brossard was to make the noise covariances large enough and then reduce them in size until you got the best results. There are also the parameters in utils_numpy_filter.py but they matter less. Some essential noise covariances are updated via the 'run' routine in utils_numpy_filter_py which makes a recursive call to the propagate routine and then the update routine. As to which parameters matter the most, which parameters have the most sensitive effect, YOU, NOT ME, NOR ANYONE ELSE. CAN FIGURE THAT OUT FOR YOU. The values for these covariances depend on the quality of the sensor and the vehicle/apparatus/robot and road/track conditions i.e. the particular environment you are working in. This is my last message from me to you on the subject. THIS IS YOUR PROBLEM. YOU SOLVE IT! LEAVE US ALONE!

[ajay1606]

@scott81321 I am extremely sorry if my comments were bothering you !! And nowhere am talking about noise covariance or accuracy parameter tuning. My question related to the real-time ability of the model! Which is somewhat similar to the issue posted by @manymuch.

My question related to this "In the 2 convolution network of this code, the receptive field is 21(not sure), which means when we compute the covariance matrix at time t, the data from t-100ms to t+100ms is used(suppose the IMU update frequency is 100Hz), it seems unreasonable for online usage. "

[scott81321]

@ajay1606 The real-time ability of the model isn't even something you should be asking me or anyone. You figure that out. The program is fairly fast IMHO for test cases. It depends on the size of the data and from what I gather, rather linearly. For test cases. the complexity is pretty good. I am not using CNN, so don't ask me questions about CNN. I don't even bother with the CNN. Personally and I mean no disrespect to the author of the code, I don't think the CNN machine learning training part of the code is actually that useful. Performance is an issue and improving performance is a matter of tweaking the noise covariances. I've had too many emails in the past soliciting guidance beyond the point of saturation. NOW PLEASE SOLVE YOUR PROBLEMS AND LEAVE ME ALONE.