sinara-hw / sinara

Sayma AMC/RTM issue tracker
Other
42 stars 7 forks source link

Kasli SI/Remote EEMs #543

Closed hartytp closed 6 years ago

hartytp commented 6 years ago

I thought there was already an issue for this, but I can't find it now :(

hartytp commented 6 years ago

First up, a 50cm IDC supplied by technosystem going directly from EEM0 to EEM1:

INFO:worker(631,si_tst.py):print:Input delay tap 0(0.000 ns), 313660000 errors in 313660000 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 1(0.161 ns), 303173922 errors in 303173922 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 2(0.323 ns), 0 errors in 303124912 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 3(0.484 ns), 0 errors in 303068296 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 4(0.645 ns), 0 errors in 303108264 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 5(0.806 ns), 0 errors in 303074088 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 6(0.968 ns), 0 errors in 309747082 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 7(1.129 ns), 0 errors in 303127706 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 8(1.290 ns), 0 errors in 303161214 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 9(1.452 ns), 0 errors in 305045974 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 10(1.613 ns), 0 errors in 304028032 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 11(1.774 ns), 0 errors in 303122296 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 12(1.935 ns), 0 errors in 304083158 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 13(2.097 ns), 0 errors in 304080592 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 14(2.258 ns), 0 errors in 309810428 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 15(2.419 ns), 0 errors in 303154850 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 16(2.581 ns), 0 errors in 304081054 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 17(2.742 ns), 0 errors in 303077760 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 18(2.903 ns), 0 errors in 303118028 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 19(3.065 ns), 0 errors in 303134876 bits (0.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 20(3.226 ns), 121711163 errors in 308838898 bits (3.94E-01 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 21(3.387 ns), 302859312 errors in 303124748 bits (9.99E-01 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 22(3.548 ns), 303106386 errors in 303106386 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 23(3.710 ns), 303106032 errors in 303106032 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 24(3.871 ns), 303088200 errors in 303088200 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 25(4.032 ns), 303113788 errors in 303113788 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 26(4.194 ns), 303162456 errors in 303162456 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 27(4.355 ns), 310738054 errors in 310738054 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 28(4.516 ns), 303121604 errors in 303121604 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 29(4.677 ns), 303107956 errors in 303107956 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 30(4.839 ns), 303145980 errors in 303145980 bits (1.00E+00 error rate)
INFO:worker(631,si_tst.py):print:Input delay tap 31(5.000 ns), 303119462 errors in 303119462 bits (1.00E+00 error rate)

This looks really good, we have a 3ns clear window with no errors in 3e8 bits transmitted.

jordens commented 6 years ago

Ok. The code is lacking a lot of clock domain crossings. And it is ignoring ISI. Both will bite you sooner or later. But yeah. It depends on what you are going to measure now.and the conclusions you are drawing.

hartytp commented 6 years ago

Still get a decent eye with a 1m SCSI cable, but it's down to sub ns by the time I go through a 10m cable.

hartytp commented 6 years ago

The code is lacking a lot of clock domain crossings.

Which ones? The crucial one is the CDC on the looped back data input. But, I'm using the idealy to ensure that the data inputs meet setup/hold on the rtio clock, so no cdc is needed.

There are also no CDCs on most of the CSRs, but I'm only reading from them when their values are static, so again CDCs aren't needed.

And it is ignoring ISI.

What do you mean here? Do you mean because I'm using a counter instead of a PRBS? Or, do you mean because I only have traffic in one direction in the cabling?

Anyway, this isn't supposed to be a definitive test of everything possible. But, I think it does give a good indication of what kind of things are likely to work and what won't.

jordens commented 6 years ago

No. rio and the kernel CPU are different domains. The CSRs need CDCs (or be in the kernel CPU CD). Have a look at how we transfer the rtio_counter from rio to the kernel (and what happens when we failed to do so for a replica). ISI because the patterns are fixed. You are somewhat resolving that with different patterns on different IOs but there are not long runs and there are no asymmetric patterns AFAICT.

hartytp commented 6 years ago

The CSRs need CDCs (or be in the kernel CPU CD).

ACK. In general, yes. I know the code isn't documented, but if you look at the experiment I'm running, there is only 1 CSR that actually needs a CDC and that's the output_en CSR, which has a CDC. All the other CSRs are only read out when they are static, and so don't need a CDC.

ISI because the patterns are fixed. You are somewhat resolving that with different patterns on different IOs but there are not long runs and there are no asymmetric patterns AFAICT.

ACK. A PRBS would be better, but IIRC there is a decent density of transitions on the various lines with a 7-bit counter. This should show up any really major SI issues, which is all I'm trying to do here. Beyond that it's hard to test like this, since the results are likely to be so dependent on the quality of the SCSI cable, the exact use case etc.

Out of curiosity, is there any plan to move the PRBS generator from the JESD stack to somewhere like misoc as a general-purpose core?

jordens commented 6 years ago

ACK. I overlooked that you are stopping the logic. But it's a bit more complicated. out_en_ret only has the single FF in the IDDR. That's usually not enough. Sure. This will show the major issues. But ultimately you'll want to make sure that there no relevant SI issues which is a different bar to clear.

If the JESD PRBS can be made generic then it can move to misoc IMHO. But there is already a LFSR in misoc. And there is XORSHIFT in redpid. And redpid has another LFSR.

hartytp commented 6 years ago

out_en_ret only has the single FF in the IDDR. That's usually not enough.

Yes, but don't forget there is an IDELAY2E on that input, which I'm using to ensure it meets S/H, so there should be no metastability issues.

If the JESD PRBS can be made generic then it can move to misoc IMHO. But there is already a LFSR in misoc. And there is XORSHIFT in redpid. And redpid has another LFSR.

ACK.

hartytp commented 6 years ago

Anyway, if I wrote this again from scratch I'd do it differently; it was partially a learning exercise.

hartytp commented 6 years ago

hmmm...there was a factor of 2 error in my tap width calculation. So, the eye above is actually only about 1.5ns.

Here is the eye for 125MHz DDR with the following configuration: Kasli EEM0 -> 30cm IDC -> VHDCI_carrier -> 10m SCSI cable -> VHDCI_CARRIER -> 30cm IDC -> Kasli EEM1

INFO:worker(642,si_tst.py):print:Input delay tap 0(0.000 ns), 303167696 errors in 3.03E+08 bits (1.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 1(0.081 ns), 303129578 errors in 3.03E+08 bits (1.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 2(0.161 ns), 314551530 errors in 3.15E+08 bits (1.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 3(0.242 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 4(0.323 ns), 0 errors in 3.05E+08 bits (0.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 5(0.403 ns), 9932 errors in 3.03E+08 bits (3.28E-05 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 6(0.484 ns), 7643119 errors in 3.03E+08 bits (2.52E-02 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 7(0.565 ns), 23031930 errors in 3.04E+08 bits (7.58E-02 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 8(0.645 ns), 23685326 errors in 3.03E+08 bits (7.81E-02 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 9(0.726 ns), 23760042 errors in 3.04E+08 bits (7.81E-02 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 10(0.806 ns), 26463892 errors in 3.04E+08 bits (8.70E-02 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 11(0.887 ns), 28496883 errors in 3.04E+08 bits (9.37E-02 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 12(0.968 ns), 28412457 errors in 3.03E+08 bits (9.37E-02 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 13(1.048 ns), 28506996 errors in 3.04E+08 bits (9.37E-02 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 14(1.129 ns), 28423169 errors in 3.03E+08 bits (9.38E-02 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 15(1.210 ns), 29058780 errors in 3.10E+08 bits (9.38E-02 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 16(1.290 ns), 37690671 errors in 3.03E+08 bits (1.24E-01 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 17(1.371 ns), 38009287 errors in 3.04E+08 bits (1.25E-01 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 18(1.452 ns), 38015686 errors in 3.04E+08 bits (1.25E-01 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 19(1.532 ns), 37885180 errors in 3.03E+08 bits (1.25E-01 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 20(1.613 ns), 38590497 errors in 3.08E+08 bits (1.25E-01 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 21(1.694 ns), 77312361 errors in 3.03E+08 bits (2.55E-01 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 22(1.774 ns), 198012955 errors in 3.04E+08 bits (6.51E-01 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 23(1.855 ns), 293010332 errors in 3.04E+08 bits (9.64E-01 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 24(1.935 ns), 304073956 errors in 3.04E+08 bits (1.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 25(2.016 ns), 303172588 errors in 3.03E+08 bits (1.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 26(2.097 ns), 304039816 errors in 3.04E+08 bits (1.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 27(2.177 ns), 310745574 errors in 3.11E+08 bits (1.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 28(2.258 ns), 304088886 errors in 3.04E+08 bits (1.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 29(2.339 ns), 304029800 errors in 3.04E+08 bits (1.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 30(2.419 ns), 304077312 errors in 3.04E+08 bits (1.00E+00 error rate)
INFO:worker(642,si_tst.py):print:Input delay tap 31(2.500 ns), 304059326 errors in 3.04E+08 bits (1.00E+00 error rate)

The eye where the error rate is < 1e-8 is only 2 taps, so about 150ps. I'd interpret that as meaning that 125MHz DDR down a 10m SCSI won't work with Kasli driving (might be better with beefier MLVDS drivers).

gkasprow commented 6 years ago

When there is such demand, we can add tiny IDC-IDC adapter with high current LVDS drivers. The direction would need to be controlled by the I2C.

hartytp commented 6 years ago

Oops...the design I posted didn't have an ODDR on the synchronization line (out_en) which was messing up the timing!

Fixed that and retook the above data with a 10m SCSI cable. I had to swap the IDCs for shorter (about 10cm) ones to move the data into the range of the IDELAY.

Eye scan is:

INFO:worker(648,si_tst.py):print:Input delay tap 0(0.000 ns), 303129860 errors in 3.03E+08 bits (1.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 1(0.081 ns), 303128240 errors in 3.03E+08 bits (1.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 2(0.161 ns), 303059640 errors in 3.03E+08 bits (1.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 3(0.242 ns), 303121512 errors in 3.03E+08 bits (1.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 4(0.323 ns), 308800598 errors in 3.09E+08 bits (1.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 5(0.403 ns), 303091334 errors in 3.03E+08 bits (1.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 6(0.484 ns), 303134100 errors in 3.03E+08 bits (1.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 7(0.565 ns), 303085993 errors in 3.03E+08 bits (1.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 8(0.645 ns), 296658727 errors in 3.10E+08 bits (9.58E-01 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 9(0.726 ns), 153647128 errors in 3.03E+08 bits (5.07E-01 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 10(0.806 ns), 102354457 errors in 3.04E+08 bits (3.37E-01 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 11(0.887 ns), 28371558 errors in 3.03E+08 bits (9.36E-02 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 12(0.968 ns), 7 errors in 3.03E+08 bits (2.31E-08 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 13(1.048 ns), 0 errors in 3.05E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 14(1.129 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 15(1.210 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 16(1.290 ns), 0 errors in 3.05E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 17(1.371 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 18(1.452 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 19(1.532 ns), 0 errors in 3.14E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 20(1.613 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 21(1.694 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 22(1.774 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 23(1.855 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 24(1.935 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 25(2.016 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 26(2.097 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 27(2.177 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 28(2.258 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 29(2.339 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 30(2.419 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(648,si_tst.py):print:Input delay tap 31(2.500 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)

So, that actually looks fine.

Okay, provisionally, 125MHz DDR with 10m SCSI cables can be made to work in at least some circumstances if one's careful...

New design pushed, experiment is https://hastebin.com/jopededibe.py

hartytp commented 6 years ago

30cm IDC between EEM0 and EEM1 with the new design:

INFO:worker(654,si_tst.py):print:Input delay tap 0(0.000 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 1(0.081 ns), 0 errors in 3.10E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 2(0.161 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 3(0.242 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 4(0.323 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 5(0.403 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 6(0.484 ns), 0 errors in 3.09E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 7(0.565 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 8(0.645 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 9(0.726 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 10(0.806 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 11(0.887 ns), 0 errors in 3.02E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 12(0.968 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 13(1.048 ns), 0 errors in 3.11E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 14(1.129 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 15(1.210 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 16(1.290 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 17(1.371 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 18(1.452 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 19(1.532 ns), 0 errors in 3.09E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 20(1.613 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 21(1.694 ns), 0 errors in 3.09E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 22(1.774 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 23(1.855 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 24(1.935 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 25(2.016 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 26(2.097 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 27(2.177 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 28(2.258 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 29(2.339 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 30(2.419 ns), 0 errors in 3.03E+08 bits (0.00E+00 error rate)
INFO:worker(654,si_tst.py):print:Input delay tap 31(2.500 ns), 0 errors in 3.04E+08 bits (0.00E+00 error rate)
hartytp commented 6 years ago

Okay, that's all I plan to do for now, so closing.

tl;dr SI seems very good with 30cm IDCs. With 10m SCSI cables, we get an eye that's something like 1.5ns-2ns wide.

gkasprow commented 6 years ago

This issue was moved to sinara-hw/Kasli#14