Error "key 'packets_per_frame' not found" when analysing a stream

[olivier-ll]

Hello

I tried the online version ( http://list.ebu.io ) to analyse a PCAP of a 2110-20 1080i50 video. After the import, the stream is detected as "unknown". I select "Video" with the profile "1080i50", and then "Analyse Stream" => after a few minutes, I have an error popup with "something went wrong"

I made the same test with the docker image of v1.4, and I have the same error, but now I can see the error message: list_server_1 | [2019-05-20 12:53:57.608] [console] [info] Based on EBU LIST v1.4.0 list_server_1 | [2019-05-20 12:53:57.611] [console] [error] exception: [json.exception.out_of_range.403] key 'packets_per_frame' not found list_server_1 | 2019-05-20T12:53:57.613Z [pcap-full-analysis] error: exception: Error: Command failed: "/app/bin//st2110_extractor" aaf85720-7afc-11e9-9c54-27f63f1cae82 "/home//5ce2756a5ac26100236ab9a8/aaf85720-7afc-11e9-9c54-27f63f1cae82/1558356123026_aaf85721-7afc-11e9-9c54-27f63f1cae82.pcap" "/home//5ce2756a5ac26100236ab9a8/aaf85720-7afc-11e9-9c54-27f63f1cae82" -influx_url http://influxdb:8086 -mongo_url mongodb://mongo:27017 -s "f06e4130-74de-40da-8e47-8c055871d271"

The problem is that the "packets per frame" field is not editable in the GUI, so I can't put a value of it ?

[pedro-alves-ferreira]

Hello

Thanks for the feedback. Can you please send me the pcap file so I can find out what is going wrong? not being able to detect th stream usually means something is really wrong with the file.

Pedro

[olivier-ll]

Here is a "small" (126 MB) capture of the ST2110-20 stream (1080i50, yuv422 10 bit) : https://1fichier.com/?a8dvuhxevjih0v4yfntt

It's only one second long, if you need a longer file I can upload a bigger file.

Thanks for your help

[pedro-alves-ferreira]

Thanks. That should be more than enough.

[pedro-alves-ferreira]

The problem is that you have a lot of dropped packets. If you open the file in Wireshark and filter for rtp.marker == 1, you will see that the number of captured packets per frame varies a lot. (see picture).

Anyway, I forced the number of packets per frame (2160) and the read schedule and LIST correctly reported a substantial number of dropped packets (see picture).

I will file a bug so that we find a way of reporting to the user that the file has dropped packets.

Which tool did you use to capture the stream?

[olivier-ll]

Thanks for the answer

I used a plain "tcpdump" capture through command line on a Mellanox 10Gb card. I assumed tcpdump was enough for a simple capture like that (1.1 Gb/sec), but seems like I was wrong !

I know there are several other tools to make pcap captures. What is the recommended way of making a capture for pi-list ?

[pedro-alves-ferreira]

We have our own capture tool that uses hardware acceleration, but it's not open source. For relatively low bit rates, tcpdump should be enough. When you run it, it will show you a report on the number of packets dropped. Something like:

100000 packets captured 100000 packets received by filter 0 packets dropped by kernel

Can you confirm you are losing packets in tcpdump?

[olivier-ll]

I don't know why, but yes, packets were lost, and tcpdump reported no packets dropped. I rebooted the host, and now it is ok : captures really have no packets dropped, and I can now import the PCAP without error !

I tried 2 different cards (Intel X710 vs Mellanox Connect X-5), and 2 different pcap tools : tcpdump and n2disk.

=> the results are similar between tcpdump and n2disk => on Mellanox, the timings seem completely off :

=> on Intel, the timings seem better, but not narrow compliant

The source is supposed to be narrow compliant, and the source and capture host are synchronized by PTP. Can I make something more to make sure the timings of Intel capture are accurate ? Or maybe my source is not really narrow compliant ?

[pedro-alves-ferreira]

Are you using nanosecond accuracy? Not doing that will cause this or similar results.

tcpdump -i --time-stamp-precision=nano -w

This should fix C buffer analysis.

For accurate Vrx measurement, both the source and the capture device must be PTP locked and you must make sure that the pcap file is saved with the right timestamps. -j adapter_unsynced will do that.

tcpdump -i --time-stamp-precision=nano -j adapter_unsynced -w

In order for this to work, you will have to have ptp4l or a similar PTP daemon working. This requires a bit more configuration.

De: olivier-ll notifications@github.com Enviada: 21 May 2019 14:06 Para: ebu/pi-list pi-list@noreply.github.com Cc: Pedro Ferreira pedro@bisect.pt; Comment comment@noreply.github.com Assunto: Re: [ebu/pi-list] Error "key 'packets_per_frame' not found" when analysing a stream (#41)

I don't know why, but yes, packets were lost, and tcpdump reported no packets dropped. I rebooted the host, and now it is ok : captures really have no packets dropped, and I can now import the PCAP without error !

I tried 2 different cards (Intel X710 vs Mellanox Connect X-5), and 2 different pcap tools : tcpdump and n2disk.

[captures]https://user-images.githubusercontent.com/50832405/58097805-7d7ce400-7bd8-11e9-8e93-0229bf0c6ed2.PNG

=> the results are similar between tcpdump and n2disk => on Mellanox, the timings seem completely off :

[mellanox]https://user-images.githubusercontent.com/50832405/58097804-7d7ce400-7bd8-11e9-94ff-f204baf5a360.PNG

=> on Intel, the timings seem better, but not narrow compliant

[intel2]https://user-images.githubusercontent.com/50832405/58098012-f714d200-7bd8-11e9-9841-86d654a9d01d.PNG

[intel]https://user-images.githubusercontent.com/50832405/58097806-7d7ce400-7bd8-11e9-81dc-4d3811e073c3.PNG

The source is supposed to be narrow compliant, and the source and capture host are synchronized by PTP. Can I make something more to make sure the timings of Intel capture are accurate ? Or maybe my source is not really narrow compliant ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ebu/pi-list/issues/41?email_source=notifications&email_token=AEZI3QCVZAPAPCWMT4LXQATPWPXTZA5CNFSM4HOBXUB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV325BQ#issuecomment-494382726, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEZI3QB6R6VDAVDFMXQIJALPWPXTZANCNFSM4HOBXUBQ.

[olivier-ll]

I have ptp4l and phc2sys running.

I tried " --time-stamp-precision=nano " option :

• no effect on Intel
• but great effect on Mellanox !

But now the Mellanox has bad Vrx ! We'll make some more test on our PTP setup to see if we can fix that.

Thanks for all your help !

[pedro-alves-ferreira]

I'm glad to know it's working now. Let us know if you need more help.