search

hardbyte / python-can

The can package provides controller area network support for Python developers
https://python-can.readthedocs.io
GNU Lesser General Public License v3.0
1.31k stars 604 forks source link

_parse_container does not receive proper data to process #763

Closed cucdat closed 4 years ago

cucdat commented 4 years ago

Hi,

I am trying to use this tool to parse some blf files but there is one of them throwing me the Parsing Error. I am tried to fix it by updating the source as well as relog it with different CANalyzer Version but it cannot work so I post it here to ask for your support.

Issue Description: The data given to _parse_container does not start with the proper signature (LOBJ), as a consequence, the _parse_data raised the BLFParseError. Annotation

Environment:

If you have any idea to figure out the root cause of this, kindly let me know. I will appreciate it so much. I can share with you the blf for debugging via email in case you need it.

christiansandberg commented 4 years ago

It would be interesting to know which obj_type came before the exception. There is a problem with blf files in that different object types are padded differently. One person had issues with FlexRay data in the log.

christiansandberg commented 4 years ago

It’s also normal that containers do not start with LOBJ. Objects can be split up in two containers since containers usually have a fixed size.

cucdat commented 4 years ago

It is not the data of container actually, it is the data that will transfer to _parse_data. I will check what is the object type comes before this exception then get back to you later.

Usually, this tool works perfectly with other blf files, even up to hours, but only there are some files coming from specific PC/Vector Application have this issue.

cucdat commented 4 years ago

Hi,

Thanks for your hint! I tried to trace back to the root cause that made the unpacking exception. First, the object is just a normal CAN 2 Message object, so I do not think it will make the exception here

image

The real root cause seems to be the buffer size required for Struct.unpack_from is bigger than the buffer size that we defined. Actual Size: 131106 Required: 131120 image

By the way, the uncompressed when I unpacked the Container is 131072 = exact 128kb. image

Shall we change this buffer size a little bit bigger to handle this case and how should we change it? (Where should we put the fixed code to not affect other functions) It seems like we should check the buffer size and resize it to a proper one before unpacking https://docs.python.org/3.0/library/struct.html#struct.unpack_from

christiansandberg commented 4 years ago

I think I know what the problem is. I did some optimizations recently in develop where I skipped checking if the object is split up in the next container and relies on struct raising an error if that’s the case. With CanMessage2 however we only read the first part of it so it could succeed even if the rest of the object is missing.

So I think the latest released version should work for you. I’ll fix it in the coming days.

cucdat commented 4 years ago

That would be perfect. Just left me a line when you fixed it, I will be happy to be the first one to test it.

By the way, can you explain why don't we need the tail of the CAN2? Doesn't it contain data?

Thank you so much.

christiansandberg commented 4 years ago

The rest of the data is not anything we use in our message class.

duracell013 commented 4 years ago

Hello,

I've only been using this module (version 3.3.2) for a day, but I'm also facing some BLFParseError with some files due to header issues. I've modified the blf.py to print the header here.

In the console output, it seems like the file position is incorrect at the start of the iteration. In the exemple below, I think the position should be one byte earlier as we can clearly see the end of "LOBJ".

image

In another example, the position is "late" and I only see the end of "LOBJ".

For info, I'm using genuine BLF files generated by CANanlyzer. Please, let me know if you need an example file.

Thank you for your help! Stéphan

cucdat commented 4 years ago

Hi Duracell, Use the dev branch might fix this issue. I used to encounter this with 3.3.2 but not anymore with the dev branch.

christiansandberg commented 4 years ago

This makes me slightly more confused. Any example log files would be great! You can email me at christiansandberg at me dot com if you don’t want to share them publicly.

christiansandberg commented 4 years ago

@duracell013, I just realized a bug fix for CAN-FD messages has not been released yet so you can try develop instead.

duracell013 commented 4 years ago

@christiansandberg I've emailed you a sample file. I have tested the develop branch without success (though I'm not 100% sure the module installation from the ZIP file was correct).

Thank you for your help!

christiansandberg commented 4 years ago

I've made a fix that seems to work.

pip install https://github.com/hardbyte/python-can/archive/bugfix/763-fix-blf-reader.zip
duracell013 commented 4 years ago

@christiansandberg Thank you so much for looking into it so quickly! That seems to work perfectly!

cucdat commented 4 years ago

Sorry for my late feedback.

I have tested this on the blf files that I had problems with and it worked without any Parsing issues. Also tested on other working files to check if it is compatible or not and found no issues.

The solution seems to work perfectly.

Thank you a lot for your support @christiansandberg!