Error importing J1939 Sym file

[MikeAyles]

Hi, I'm struggling to get a J1939 Sym file to convert. I am getting the error below. It looks like to me that the SYM file has characters that are being interpreted as comments.

I can email a copy of the sym file if you would like to replicate this error?

PS C:\Users\XXXXXXXXXX\XXXXXXXXXX> canconvert .\J1939.sym test.dbc
INFO - formats - arxml is not supported
INFO - formats - kcd is not supported
INFO - formats - fibex is not supported
Traceback (most recent call last):
  File "c:\python27\lib\runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "c:\python27\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python27\Scripts\canconvert.exe\__main__.py", line 9, in <module>
  File "c:\python27\lib\site-packages\canmatrix\convert.py", line 294, in main
    convert(infile, outfileName, **cmdlineOptions.__dict__)
  File "c:\python27\lib\site-packages\canmatrix\convert.py", line 41, in convert
    dbs = canmatrix.formats.loadp(infile, **options)
  File "c:\python27\lib\site-packages\canmatrix\formats.py", line 54, in loadp
    return load(fileObject, importType, **options)
  File "c:\python27\lib\site-packages\canmatrix\formats.py", line 66, in load
    dbs[key] = moduleInstance.load(fileObject, **options)
  File "c:\python27\lib\site-packages\canmatrix\sym.py", line 312, in load
    split = shlex.split(tempArray[1])
  File "c:\python27\lib\shlex.py", line 279, in split
    return list(lex)
  File "c:\python27\lib\shlex.py", line 269, in next
    token = self.get_token()
  File "c:\python27\lib\shlex.py", line 96, in get_token
    raw = self.read_token()
  File "c:\python27\lib\shlex.py", line 172, in read_token
    raise ValueError, "No closing quotation"
ValueError: No closing quotation

[ebroecker]

shure - you can send your sym to me. This will accelerate the fix...

@altendky Do you currently do any updates on sym? Just to prevent collisions

[altendky]

@broecker no, I'm not currently in the middle of anything. Let me know if you want me to take a look.

[altendky]

@MikeAyles go ahead and copy me when you send it as well (or just share here if it's not too secret). Both our emails are on our profiles.

[altendky]

The first piece was a straightforward parsing improvement. Then we hit:

[XFER]
ID=18CAFEFEh    
Type=J1939PG
Multipacket=True
Var=PGNOfRequestedInformation unsigned 0,24 /ln:"Parameter Group Number of Requested Information (XFER)" /spn:2552
Var=LengthOfDataForTheReportedPGN unsigned 24,8 /ln:"Length Of Data For The Reported PGN (XFER)" /spn:2553
Var=ControllerApplicationIdentity unsigned 32,32 /ln:"Controller Application Identity (CAI)" /spn:2554
Var=TransferData raw 64,14216 /ln:"Transfer Data" /spn:2555

and get:

/home/altendky/misc/experiments/MikeAyles/venv/bin/python /home/altendky/misc/experiments/MikeAyles/venv/bin/canconvert Peak.sym test.dbc
INFO - formats - arxml is not supported
INFO - canmatrix - bitstruct could not be imported // No signal de/encoding possible // try pip install bitstruct
INFO - formats - kcd is not supported
INFO - formats - fibex is not supported
INFO - formats - xls is not supported
INFO - formats - xlsx is not supported
INFO - formats - yaml is not supported
Variable type 'string' found and skipped
Variable type 'string' found and skipped
Variable type 'string' found and skipped
Variable type 'string' found and skipped
Traceback (most recent call last):
  File "/home/altendky/misc/experiments/MikeAyles/venv/bin/canconvert", line 11, in <module>
    load_entry_point('canmatrix', 'console_scripts', 'canconvert')()
  File "/home/altendky/misc/experiments/MikeAyles/canmatrix/canmatrix/convert.py", line 329, in main
    convert(infile, outfileName, **cmdlineOptions.__dict__)
  File "/home/altendky/misc/experiments/MikeAyles/canmatrix/canmatrix/convert.py", line 41, in convert
    dbs = canmatrix.formats.loadp(infile, **options)
  File "/home/altendky/misc/experiments/MikeAyles/canmatrix/canmatrix/formats.py", line 63, in loadp
    return load(fileObject, importType, key, flatImport, **options)
  File "/home/altendky/misc/experiments/MikeAyles/canmatrix/canmatrix/formats.py", line 75, in load
    dbs[key] = moduleInstance.load(fileObject, **options)
  File "/home/altendky/misc/experiments/MikeAyles/canmatrix/canmatrix/sym.py", line 391, in load
    tempArray[1]))
ValueError: Unknown type 'raw' found

@ebroecker Is there any multipacket or raw (byte sequence?) data type concept elsewhere in canmatrix?

@MikeAyles I use the .sym support from canmatrix but my files are only J1939 in the sense that we picked non-conflicting identifiers. I haven't used any multipacket frames or the raw datatype. Perhaps if you provide some info on what you are trying to do we can consider how much and how best to support this. Or at least how sufficiently to support it. :] I'll note that we are almost certainly not parsing the SPN presently.

[ebroecker]

I have no idea about J1939 - because I'm a can-automotive-car-only guy :-( I don't think there is any multipacket/raw support anywhere - but I'll try to learn something about it to understand a bit more. Maybe someone can point me to some documentation where I can learn what we would need in canmatrix to support J1939 better.

Indeed something about J1939 is already in canmatrix : https://github.com/ebroecker/canmatrix/blob/development/canmatrix/canmatrix.py#L1057-L1080 and in https://github.com/ebroecker/canmatrix/blob/development/canmatrix/join.py

can a multipacket/raw 'signal' defined anyhow in dbc?

[ebroecker]

can we do anything here? Maybe at least ignore such lines?

[ebroecker]

@altendky is it ok to close this issue? I'm not sure...

[altendky]

Sure, it can be reopened if details are provided.