mitchdetailed / Log-CANverter

Turn CANbus Log files into CSV files for analysis using a supporting DBC file
GNU General Public License v3.0
21 stars 6 forks source link

The expected format of reading .LOG files #1

Open AFBecan opened 2 years ago

AFBecan commented 2 years ago

Dear Mitch,

This, I am writing, is more of a support to run the file in a specific fashion.

I have been analyzing your code as I wish to decode a raw data of CAN messages. I am a jr. data engineer who is still learning Python, and especially the more complex RegExs. Is the format of expected rows and validated lines in a .LOG file determined by the try...except segment between lines 122 and 186? Or does it begin with reading the first line with the .compile method with RegEx in Line 120?

I would appreciate your feedback, AFB

mitchdetailed commented 2 years ago

I'm not sure exactly which lines in the code you're referencing by number, but I'll use try/except where I'm testing if conditions and will 'pass' the information if it doesn't have the appropriate structure defined. it's simply a way to 'skip' the line and move onto the next step in the process

AFBecan commented 2 years ago

The lines I speak of, start with these lines. Since in my compiler, they are realigned.

linePattern` = `re.compile(r"\((\d+.\d+)\)\s+...#([0-9A-F]+)"

and the other range is from

        for row in tqdm(inputfile, desc="Lines", total=numlines, unit=" Lines"):
            try:
                tokens = linePattern.search(row).groups()
...
            except:
                print("invalidated line observed: '%s'" % (row[:-1]))

Because of a specific datalog format from my csv files, which I convert to .log in order to work for your code script, I only receive invalidated lines, regardless of how much I optimize the headers and the individual columns. By the end of the code run, I hoped to at least decode the CAN messages, but I so far only received the exception print message for every line. I also identified the delimiter, which was whitespace and then the others as I modified in the code.

This was what my last post was about.