PacketParser only picks up a 100% currupt packages

[jnslsrwy]

The parser only reads out currupt packages. Mostly the same package with NO_ACK. I tried downgrading to the same script from release 0.1.2 and parsing works again with average currption of 1%. In both cases I tested the script without making any changes to it on the Wemos D1. What could cause this issue?

[0xCAFEDECAF]

hmm ☹️ not good. Obviously, I tested a lot before releasing; on my side the average corruption is very low (< 0.01%). Are you perhaps running on 160MHz CPU speed? There might be a small issue there. Otherwise, can you run the 'VanBusDump.ino' sketch and dump some logging output here?

[jnslsrwy]

Sorry for my late reply. I'm running on 80MHz speed so that could be ruled out. Furthermore, as you asked I tried 'VanBusDump.ino' with SN65HVD230 transciver. You'll find the log enclosed.

I also tried using the transciever with the parser script and the chip keeps crashing and restarting. With the resistor circuit it only picked up currupt packages as described above.

Logfile: buslog.txt

[0xCAFEDECAF]

Really strange... Although the packets seem corrupt, the vast majority has CRC_OK. E.g. have a look at this packet from your log: Raw: #1695 ( 1/15) 7(12) 0E 824 W-0 00-00-00-00-00-00-00:54-98 NO_ACK OK 5498 CRC_OK

All zeros, but a correct CRC.

On my vehicle the IDEN 824 packets look like this: Raw: #0000 ( 1/15) 7(12) 0E 824 W-0 FF-FF-FF-FF-00-00-00:5D-68 NO_ACK OK 5D68 CRC_OK

At first sight that might also seem corrupt, but I'm pretty sure this is fine.

So in fact your packets might seem to be corrupt, but may in fact be fine.

Similarly, comparing this packet from your log: Raw: #1694 (15/15) 27(32) 0E 564 RA0 85-00-00-00-00-00-00-00-00-00-18-00-00-1C-00-00-FF-FF-00-00-FF-FF-FF-FF-00-9B-85:C6-E2 ACK OK C6E2 CRC_OK

with an example from mine: Raw: #0132 (13/15) 27(32) 0E 564 RA0 84-00-00-00-00-00-00-88-7A-FE-00-4A-34-1A-2F-2E-00-3C-06-0E-00-42-FF-FF-02-EE-84:7D-5A ACK OK 7D5A CRC_OK

I guess packets often have repeated "00" values and it is ok.

[jnslsrwy]

Today I tried the parser from your latest release v2.0.1 and the most recent changes from master branch and both gave me an inital parse result again after resetting the chip but then it won't parse any more packets. Enclosed you'll find the serial log.

PacketParserSerial.txt

Do you have any clues why it is like this?

[0xCAFEDECAF]

No idea. Strikes me that only 15 packets are received, which happens to be the RX queue size. What happens if you change line 3747 of PacketParser.ino to:

VanBusRx.Setup(RX_PIN, 100)

so that the RX queue size is 100?

And what do you mean by "resetting the chip"?

[jnslsrwy]

Doing the changes you proposed makes the parser recieve 100 packages. So it seems like it has something to do with the Rx queue size.

By resetting I mean pressing the reset button to cause it to reboot.

[0xCAFEDECAF]

To be honest, I am baffled. The latest version is the one I am using almost daily for my https://github.com/0xCAFEDECAF/VanLiveConnect project.

So it seems that in your case the RX queue is piling up, packets are not being freed when:

    VanBusRx.Receive(pkt, &isQueueOverrun);

is called in line 3770 of PacketParser.ino.

TVanPacketRxQueue::Receive calls AdvanceTail(); in line 232 of VanBusRx.cpp, which in turn is found at line 285 of VanBusRx.h:

void AdvanceTail()
{
    if (++tail == end) tail = pool;  // roll over if needed
} // _AdvanceTail

How could that go wrong?

Are you sure you installed the newest version? Check also by downloading the .zip file and comparing the unpacked contents with that in your ...\Arduino\libraries\VanBus directory.

[0xCAFEDECAF]

Also, I would expect lines like this:

QUEUE OVERRUN!

because the isQueueOverrun variable will become true if no packets are freed from the RX queue (line 3772 of PacketParser.ino).

[jnslsrwy]

Ok so I downloaded the .zip file from the link you provided, extracted the lib folder to my desktop and did a CRC-64 checksum with 7zip on it only to find its the same from my copy of the libary in ...\Arduino\libraries\VanBus. So I am pretty sure I got the exact same copy you are working with.

My checksums were both times 08B8941E95C85D6D for data & 3D5AB89F8C250B96 for data and names..

[jnslsrwy]

note: bypassing the queue overrun check by removing &isQueueOverrun from the VanBusRx.Receive() call in line 3770 of PacketParser.ino seems to fix the stop of parsing without any further issues for me

[0xCAFEDECAF]

Ok, that's good news :-) So it's working now as expected?

I just checked again the latest version of the PacketParser.ino example sketch on my vehicle. All works as expected. So I hope we will discover which aspect is different between your setup and mine.

By the way, which model do you have? From the VIN number it seems you have a 307?

[jnslsrwy]

Yes now it parses packets besides the queue check without further limitations. You're right my car is a 307 SW with manual transmission. BTW: I noticed that the rate of corrupt recieved packets is significantly lower than at the last working release (v0.1.2).

ATM I'm using the same transceiver as shown in your wiring diagram without a level shifter. Although I also tried everything with the voltage divider circuit with the same results as with the transceiver.

[0xCAFEDECAF]

Yes, the latest version (0.2.1) has improved the packet reception quite a bit.

The ESP8266 platform seems less suitable for the microsecond timing that is needed to time the bits on the VAN bus. ESP8266 is single core. With its various sources of high-priority (or even non-maskable) interrupts, it will now and then delay the invocation of the pin-level interrupt handler. This results in a wrong number of bits read. For example, Wi-Fi is a notorious source of non-maskable interrupts on the ESP8266. Disabling Wi-Fi substantially improves the accuracy of the microsecond timing.

I am considering to port the library also to the ESP32 platform as that might have better facilities to perform accurate microsecond timing, like the dual core CPU.

[jnslsrwy]

Furthermore during latest testing using the SN65HVD230 transciver I had some 'glitchy' behaviour occuring. For example a passenger airbag fault message or the gauges on the speedometer falling to zero while driving. In that case the segment display of the odometer also shows illegible characters.

All of this happens without the TX pin of the transciever beeing connected to the Wemos. Could this be the queue overrun causing unexpected behaviour or might the transciever be faulty?

[0xCAFEDECAF]

In my opinion, this can only be caused by a faulty transceiver. I did many test drives with the SN65HVD230 (only the RX pin connected) and never witnessed this behaviour. Anything happening on the RX pin should never have any effect on the bus. The datasheet says: "These devices [...] include many protection features providing device and CAN network robustness."

During my tests, I have the TX pin disconnected. Maybe your TX pin is picking up noise, and should be pulled down to GND?

The datasheet also mentions:

10.4.3 Standby Mode (Listen Only Mode) of the HVD230 If a logic high(> 0.75VCC) is applied to RS(pin 8) in Figure 30 and Figure 32, the circuit of the SN65HVD230 enters a low-current, listen only standby mode, during which the driver is switched off and the receiver remains active. In this listen only state, the transceiver is completely passive to the bus.

Maybe that is a solution to your issue?

[morcibacsi]

I had several faulty SN65HVD230 devices. On average from a stock of 5 only 1-2 was working.... I ditched them very fast and replaced them with MCP2551 and TJA1050/1040 family. They are much more reliable. They do require a voltage level shifter, but BSS138 based ones are quite good. If you are making a custom board it doesn't take much space if you build it from SMD components. This type: https://learn.sparkfun.com/tutorials/bi-directional-logic-level-converter-hookup-guide/all

[0xCAFEDECAF]

Thanks Peter, good to know. Personally I tested my 2 SN65HVD230 devices without any issues, but of course "2" is not a representative sample size :-)

[0xCAFEDECAF]

Long time no hear; I guess this ticket can be closed.

[Antonzlo]

I'll reopen this. Nodemcu, 4.7kOhm resistors, tried 80 and 160 MHz. Out of 10000 packets only 50ish are crc fixed. Rest is error. https://pastebin.com/zQ7bc99w If you need all packets even with errors: https://pastebin.com/G6NTKSrt

[0xCAFEDECAF]

hmmm... that's a lot...

Typically my test setup shows about 1 in 100,000 unfixed CRC-error packets.

I always use the schematics with the tranceiver ICs. The last time I used the schematic with the resistor divider is a few years ago. Maybe I'll remove that schematic alltogether.

I've never tested with a NodeMCU, only with Wemos D1 mini (ESP8266 based). And currently I am testing also with a LilyGO TTGO T7 Mini32 (ESP32 based). I see no reason why a NodeMCU would not work.

I will try to reproduce this issue. If that succeeds, I will try to find a fix for it.

In the meantime, keep on experimenting. I'm eager to read the results!

[Antonzlo]

The only thing I can think of is I used 2 2.2k resistors as I don't have 4.7, but I don,t think 4.4 is such a big difference

[Vincent0690]

The only thing I can think of is I used 2 2.2k resistors as I don't have 4.7, but I don,t think 4.4 is such a big difference

x2 2.2k ohm? It's supposed to be 470ohm not 4.7kohm

[Antonzlo]

The only thing I can think of is I used 2 2.2k resistors as I don't have 4.7, but I don,t think 4.4 is such a big difference

x2 2.2k ohm? It's supposed to be 470ohm not 4.7kohm

The simplest schematic is not to use a transceiver at all, but connect the VAN DATA line to GrouND using two 4.7 kOhm resistors. Connect the GPIO pin of your ESP8266/ESP32 board to the 1:2 voltage divider that is thus formed by the two resistors. This is only for receiving packets, not for transmitting. Results may vary.

In readme it's 4.7k. I can try 470 but I believe it's too low an can cause problems on the bus?

[Vincent0690]

The only thing I can think of is I used 2 2.2k resistors as I don't have 4.7, but I don,t think 4.4 is such a big difference

x2 2.2k ohm? It's supposed to be 470ohm not 4.7kohm

The simplest schematic is not to use a transceiver at all, but connect the VAN DATA line to GrouND using two 4.7 kOhm resistors. Connect the GPIO pin of your ESP8266/ESP32 board to the 1:2 voltage divider that is thus formed by the two resistors. This is only for receiving packets, not for transmitting. Results may vary.

In readme it's 4.7k. I can try 470 but I believe it's too low an can cause problems on the bus?

Oh sorry I thought you were using a SN65 or MCP. I only tried once to directly connect bus to the ESP, it was looking almost the same as you, too unstable. Im using a SN65HVD230 transceiver now.

[Vincent0690]

I did my first test with a SN65HVD230 transceiver. With no diode and resistor. Is it my log normal? Almost everything is corrupted, is it just because of the clamping or something else is wrong? I'm powering my ESP8266 (ESP12F) via USB.

https://pastebin.com/rJQDp1eq

[0xCAFEDECAF]

This looks like a bit timing issue.

For example, take the following packet from your pastebin:

Raw: #0002 ( 3/15) 16(21) 0E E65 1 (W-1) AD-BD-66-65-AC-3C-2A-5D-48-94-94-98-88-99-1D-31:35-26 NO_ACK ERROR_NBITS 04BE CRC_ERROR uBit=92

I'm pretty sure that this is a VIN packet; see also: http://pinterpeti.hu/psavanbus/PSA-VAN.html#E24 (Press Ctrl-F to search for 'E24' on that page).

Those VIN packets always start with:

E24 W00  56-46-33
          V  F  3

I know because PSA-Peugeot manufacturer code is "VF3".

Now, AD-BD-66 may not look like 56-46-33, but it is the same if stuffing in some bits here and there.

It seems to me that in your case, many 2-bit sequences (e.g. 00 or 11) are seen incorrectly as 1-bit sequence (0 resp. 1).

You might want to experiment a bit with different bit timing values in function nBitsTakingIntoAccountJitter

To get detailed bit timing dumps, uncomment this line:

//#define VAN_RX_ISR_DEBUGGING

in VanBusRx.h

Note that you get a lot of output on your serial port if you uncomment this line. Having this #define active is suitable for capturing only a handful of packets before the RX queue overruns.

[Antonzlo]

LilyGO TTGO T7 Mini32 (ESP32 based)

Does it work better? I can use esp32-cam that I have, but my transceiver didn't arrive yet

[morcibacsi]

LilyGO TTGO T7 Mini32 (ESP32 based)

Does it work better? I can use esp32-cam that I have, but my transceiver didn't arrive yet

@Antonzlo For the esp32 you can use my library from here: https://github.com/morcibacsi/esp32_rmt_van_rx

[0xCAFEDECAF]

LilyGO TTGO T7 Mini32 (ESP32 based)

Does it work better? I can use esp32-cam that I have, but my transceiver didn't arrive yet

I don't know, but for me it works 😊

[Antonzlo]

I have tried adjusting timing, without much success. It was around 6% repaired at one point but then dropped to 0. I have noticed that packets of stalk button are consistent, only difference is iden seems to be something 0xEE where is changing. Here is button on and off

19:14:17.561 -> Raw: #2012 ( 3/15) 24(29) 0E 9EE E (RA0) EE-EA-EE-EE-EE-E1-11-1D-DE-ED-1A-EE-EE-EE-EE-EE-A6-EF-F9-90-00-0F-F5-67:CB-98 NO_ACK ERROR_NBITS 5496 CRC_ERROR
19:14:17.561 ->                                           ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  g 
19:14:17.778 -> Raw: #2018 ( 9/15) 24(29) 0E 2EE E (RA0) EE-EA-EE-EE-EE-E1-11-1D-EE-ED-1A-EE-EE-EE-EE-EE-A6-FF-F9-90-00-0F-F5-67:95-6C NO_ACK ERROR_NBITS 9BF0 CRC_ERROR

[0xCAFEDECAF]

Did you already try uncommenting this line:

#define VAN_RX_ISR_DEBUGGING

in VanBusRx.h ?

[Antonzlo]

Did you already try uncommenting this line:

#define VAN_RX_ISR_DEBUGGING

in VanBusRx.h ?

Yes, but it didn't make a difference? No additional info showed up. I'll pastebin my code later today

[Vincent0690]

For me it was working, I adjusted a bit and here's the result: https://pastebin.com/MK9BPmxz Pretty good, ig.

Raw: #0040 (11/15) 17(22) 0E E24 8 (W-0) 56-46-33-32-43-4B-46-55-41-34-34-37-33-35-32-37-37:B5-B8 NO_ACK OK B5A4 CRC_ERROR V F 3 2 C K F U A 4 4 7 3 5 2 7 7

Something a bit weird tho is that all E24 packets are declared as corrupted, but they look fine. Any idea why? Also timings settings has to be adjusted for every board? I mean, my actual settings would work fine if i replace my esp8266 with a similar one?

[0xCAFEDECAF]

Yes, but it didn't make a difference? No additional info showed up. I'll pastebin my code later today

Did you run VanBusDump.ino? It will dump bit timings for packets that have CRC ERROR.

[0xCAFEDECAF]

Also timings settings has to be adjusted for every board?

Bit timings are measured by counting CPU cycles. That should be the same for every board, or not?

[Antonzlo]

Yes, but it didn't make a difference? No additional info showed up. I'll pastebin my code later today

Did you run VanBusDump.ino? It will dump bit timings for packets that have CRC ERROR.

Oh, sorry. I tried packetParser, I thought the magic was inside the lib. I'll try tomorrow probably. How often is VIN transmitted? Is it like once per power on or once a minute?

[Vincent0690]

Also timings settings has to be adjusted for every board?

Bit timings are measured by counting CPU cycles. That should be the same for every board, or not?

Well it should be. But maybe hardware manufacturers/differences can be applied in that? Why I had to change it, we both have an ESP8266?

[0xCAFEDECAF]

Well it should be. But maybe hardware manufacturers/differences can be applied in that? Why I had to change it, we both have an ESP8266?

I have no idea...

[0xCAFEDECAF]

How often is VIN transmitted? Is it like once per power on or once a minute?

I had a search into my old session logs, but I could not find evidence of this packet being repeatedly transmitted at a fixed interval. It may be that it is triggered by some event, e.g. contact key turned, or head unit switched on. But I am not sure.

[morcibacsi]

How often is VIN transmitted?

Every 1000 ms by the BSI.

[Antonzlo]

How often is VIN transmitted? Is it like once per power on or once a minute?

I had a search into my old session logs, but I could not find evidence of this packet being repeatedly transmitted at a fixed interval. It may be that it is triggered by some event, e.g. contact key turned, or head unit switched on. But I am not sure.

Well my radio is aftermarket 😀 I'll try the dumper though.

[0xCAFEDECAF]

How often is VIN transmitted?

Every 1000 ms by the BSI.

Ah ok. Just realized that my session logs were written after filtering out the duplicate packets, so those will not show evidence of any repetition.

[0xCAFEDECAF]

I have been looking at some pictures and photos I took long ago when I used the resistor divider schematic. It seems that, at that time, I connected the top resistor to pin 2 of the radio (ISO) connector, not pin 3 as shown in the schematic.

So maybe the shown schematic is wrong. I will test and follow-up.

[Antonzlo]

Did you already try uncommenting this line:

#define VAN_RX_ISR_DEBUGGING

in VanBusRx.h ?

https://pastebin.com/x3EAcaAg

[0xCAFEDECAF]

Hi, I seemed to be able to reproduce your issue.

When using the schematic as shown (resistor divider connected to pin 3 of the radio ISO connector), I get output like this with the VanBusDump.ino sketch:

[...]
Raw: #0223 (14/15)  0( 5) 0E 626 1 (W-1) D1-2E NO_ACK OK 5734 CRC_ERROR
Raw: #0224 (15/15) 11(16) 0E 656 2 (R-0) ED-E6-ED-EE-9D-EE-92-92-91-91-ED:29-6E NO_ACK ERROR_NBITS 9B96 CRC_ERROR
Raw: #0225 ( 1/15)  2( 7) 0E 526 6 (RA0) EE-11:D1-1E NO_ACK OK D990 CRC_ERROR
Raw: #0226 ( 2/15)  2( 7) 0E 526 6 (RA0) EE-11:D1-1E NO_ACK OK D990 CRC_ERROR
Raw: #0227 ( 3/15)  0( 2) 0E NO_ACK ERROR_NBITS 0000 CRC_ERROR uBit=8
Raw: #0228 ( 4/15)  0( 5) 0E 626 1 (W-1) D1-2E NO_ACK OK 5734 CRC_ERROR
received pkts: 229, corrupt: 229 (100.000%), repaired: 1 (0%) [SB:0, DCB:0, TCB:0, DSB:1], UCB:0, overall: 228 (99.563%), avg1bit: 672, maxQueued: 2/15
Raw: #0229 ( 5/15) 11(16) 0E 656 2 (R-0) ED-E6-ED-EE-9D-EE-92-92-91-91-ED:29-6E NO_ACK ERROR_NBITS 9B96 CRC_ERROR
Raw: #0230 ( 6/15)  2( 7) 0E 526 6 (RA0) EE-11:D1-1E NO_ACK OK D990 CRC_ERROR
[...]

I changed the #define VAN_BIT_INVERTED_WIRING to 0 in VanBusRx.h, line 37, like this:

#define VAN_BIT_INVERTED_WIRING 0

After compiling and uploading the same sketch, all seems to run fine:

[...]
Raw: #0473 ( 9/15)  0( 5) 0E 7CE F (RA1) 21-14 NO_ACK ERROR_NBITS 2114 CRC_OK uBit=40
Raw: #0474 (10/15)  0( 5) 0E 4EC F (RA1) 97-68 NO_ACK OK 9768 CRC_OK
Raw: #0475 (11/15) 11(16) 0E 4D4 E (RA0) 81-0C-01-00-31-00-3E-3E-3F-3F-81:6B-C8 ACK OK 6BC8 CRC_OK
Raw: #0476 (12/15)  2( 7) 0E 5E4 C (WA0) 00-FF:1F-F8 NO_ACK OK 1FF8 CRC_OK
Raw: #0477 (13/15)  0( 5) 0E 7CE F (RA1) 21-14 NO_ACK ERROR_NBITS 2114 CRC_OK uBit=40
Raw: #0478 (14/15)  2( 7) 0E 5E4 C (WA0) 00-FF:1F-F8 NO_ACK OK 1FF8 CRC_OK
Raw: #0479 (15/15)  0( 5) 0E 4EC F (RA1) 97-68 NO_ACK OK 9768 CRC_OK
Raw: #0480 ( 1/15) 11(16) 0E 4D4 E (RA0) 81-0C-01-00-31-00-3E-3E-3F-3F-81:6B-C8 ACK OK 6BC8 CRC_OK
received pkts: 481, corrupt: 0 (0.000%), repaired: 0 (---%) [SB:0, DCB:0, TCB:0, DSB:0], UCB:0, overall: 0 (0.000%), avg1bit: 668, maxQueued: 2/15
Raw: #0481 ( 2/15)  2( 7) 0E 5E4 C (WA0) 00-FF:1F-F8 NO_ACK OK 1FF8 CRC_OK
Raw: #0482 ( 3/15)  0( 5) 0E 7CE F (RA1) 21-14 NO_ACK ERROR_NBITS 2114 CRC_OK uBit=40
[...]

I also tried to connect the resistor divider to pin 2 of the radio (ISO) connector (instead of pin 3 as shown). With that setup, results are opposite: #define VAN_BIT_INVERTED_WIRING 0 gives 100% corrupt packets whereas #define VAN_BIT_INVERTED_WIRING 1 gives 0% corrupt packets.

So I guess you can either set #define VAN_BIT_INVERTED_WIRING 0 in VanBusRx.h, or connect the resistor divider to pin 2 of the radio (ISO) connector.

I will update the schematic as shown in the README.md .

[Antonzlo]

Works now, thanks! I have better luck with 160mHz and powering Vin from 3.3v

[0xCAFEDECAF]

Nice! 😄 Indeed I always use 160 MHz CPU frequency. I will update the README.md file to mention that.

[Antonzlo]

Probably wrong place, but I flashed VanLiveConnect and I get around 30% CRC errors while having practically 0 with PacketParser. I understand the load and everything, but the most "bad" thing is website not loading. I had that while testing and it was kinda intermittent. The question is can I disable modules like mdns and remote ir? Or will it break everything?

[0xCAFEDECAF]

Wrong place, idd. Closing this ticket.

Pretty sure your new issue is not load related. I have 0% CRC errors when running VanLiveConnect .