srp fails to capture ICMPv6EchoReply if EchoRequest has data

[desbma]

I send a simple ICMPv6EchoRequest to a remote device, and want to capture the response (a good old IPv6 "ping").

• With a ICMPv6EchoRequest without data, scapy.all.srp captures the response as expected
• With a ICMPv6EchoRequest with data, scapy.all.srp fails to capture a response
• Using scapy.all.sniff response is captured as expected (I use this workaround for now)
• Wireshark sees the request and response fine in all cases
• ping6 command line utility works fine too

Here is a full test case to reproduce (ping from LOCAL_IP to REMOTE_IP), test_captureWithSniff succeeds and test_captureWithSrp fails:

#!/usr/bin/env python3

import os
import threading
import unittest

import scapy.all

REMOTE_MAC = "00:01:02:03:04:05"  # change for your config
REMOTE_IP = "2015::2"
LOCAL_ITF = "eth5"
LOCAL_IP = "2015::1"

class SniffThread(threading.Thread):

  def __init__(self, *args, **kwargs):
    self.args = args
    self.kwargs = kwargs
    self.received = []
    super().__init__()

  def run(self):
    self.received.extend(scapy.all.sniff(*self.args, **self.kwargs))

class TestPingScapySrpIssue(unittest.TestCase):

  def test_captureWithSniff(self):
    for ping_data in (None, os.urandom(8)):
      # build eth frame
      eth = scapy.all.Ether(src=scapy.all.get_if_hwaddr(LOCAL_ITF),
                            dst=REMOTE_MAC)
      ipv6 = scapy.all.IPv6(src=LOCAL_IP,
                            dst=REMOTE_IP)
      icmp = scapy.all.ICMPv6EchoRequest()
      frame = eth / ipv6 / icmp
      if ping_data is not None:
        frame = frame / ping_data

      # send it and get response
      sniff_thread = SniffThread(timeout=1,
                                 iface=LOCAL_ITF,
                                 lfilter=lambda x: x.haslayer("ICMPv6EchoReply"))
      sniff_thread.start()
      scapy.all.sendp(frame, iface=LOCAL_ITF, verbose=False)
      sniff_thread.join()

      # build expected response
      expected_eth = scapy.all.Ether(src=REMOTE_MAC,
                                     dst=scapy.all.get_if_hwaddr(LOCAL_ITF))
      expected_ipv6 = scapy.all.IPv6(src=REMOTE_IP,
                                     dst=LOCAL_IP)
      expected_icmp = scapy.all.ICMPv6EchoReply()
      expected_frame = expected_eth / expected_ipv6 / expected_icmp
      if ping_data is not None:
        expected_frame = expected_frame / ping_data

      # check response
      self.assertEqual(len(sniff_thread.received), 1)
      self.assertEqual(bytes(sniff_thread.received[0]), bytes(expected_frame))

  def test_captureWithSrp(self):
    for ping_data in (None, os.urandom(8)):
      # build eth frame
      eth = scapy.all.Ether(src=scapy.all.get_if_hwaddr(LOCAL_ITF),
                            dst=REMOTE_MAC)
      ipv6 = scapy.all.IPv6(src=LOCAL_IP,
                            dst=REMOTE_IP)
      icmp = scapy.all.ICMPv6EchoRequest()
      ping_data = os.urandom(8)
      frame = eth / ipv6 / icmp
      if ping_data is not None:
        frame = frame / ping_data

      # send it and get response
      answered, unanswered = scapy.all.srp(frame,
                                           iface=LOCAL_ITF,
                                           timeout=1,
                                           verbose=False)

      # build expected response
      expected_eth = scapy.all.Ether(src=REMOTE_MAC,
                                     dst=scapy.all.get_if_hwaddr(LOCAL_ITF))
      expected_ipv6 = scapy.all.IPv6(src=REMOTE_IP,
                                     dst=LOCAL_IP)
      expected_icmp = scapy.all.ICMPv6EchoReply()
      expected_frame = expected_eth / expected_ipv6 / expected_icmp
      if ping_data is not None:
        expected_frame = expected_frame / ping_data

      # check response
      self.assertEqual(len(answered), 1)  # fails here if ping_data != None
      self.assertEqual(bytes(answered[0]), bytes(expected_frame))

if __name__ == "__main__":
    unittest.main()

Context :

• Ubuntu 14.04
• Python 3.4
• Scapy v0.13 from this repo.

[phaethon]

Can you provide the actual ICMPv6EchoReply bytes received and expected? Such as printed out with scapy.all.hexstr. I don't have two IPV6 hosts in my test environment to repeat your test case as is.

There are test cases in regression.uts like

= ICMPv6EchoReply - Dissection with specific values
a=ICMPv6EchoReply(b'\x80\xff\x11\x11""33thisissomestring')
a.type == 128 and a.code == 0xff and a.cksum == 0x1111 and a.id == 0x2222 and a.seq == 0x3333 and a.data == b"thisissomestring"

which test basic functionality of ICMPv6EchoReply with data. It would be nice to know what exactly differs in your test case.

[desbma]

I have updated/improved my test case to reproduce: https://gist.github.com/desbma/f68e118d5fce578f3159

Here is the test output, with frame data displayed for the test_captureWithSniff test that works (using sniff instead of srp), I have removed MAC addresses:

WARNING: No route found for IPv6 destination :: (no default route?)
WARNING: Please, report issues to https://github.com/phaethon/scapy
reply received xx xx xx xx xx xx xx xx xx xx xx xx 86 dd 60 00 00 00 00 08 3a 40 20 15 00 00 00 00 00 00 00 00 00 00 00 00 00 02 20 15 00 00 00 00 00 00 00 00 00 00 00 00 00 01 81 00 3e 90 00 00 00 00
reply expected xx xx xx xx xx xx xx xx xx xx xx xx 86 dd 60 00 00 00 00 08 3a 40 20 15 00 00 00 00 00 00 00 00 00 00 00 00 00 02 20 15 00 00 00 00 00 00 00 00 00 00 00 00 00 01 81 00 3e 90 00 00 00 00
reply received xx xx xx xx xx xx xx xx xx xx xx xx 86 dd 60 00 00 00 00 10 3a 40 20 15 00 00 00 00 00 00 00 00 00 00 00 00 00 02 20 15 00 00 00 00 00 00 00 00 00 00 00 00 00 01 81 00 63 62 00 00 00 00 c8 c7 b6 b9 e4 23 77 80
reply expected xx xx xx xx xx xx xx xx xx xx xx xx 86 dd 60 00 00 00 00 10 3a 40 20 15 00 00 00 00 00 00 00 00 00 00 00 00 00 02 20 15 00 00 00 00 00 00 00 00 00 00 00 00 00 01 81 00 63 62 00 00 00 00 c8 c7 b6 b9 e4 23 77 80
./usr/local/lib/python3.4/dist-packages/scapy/sendrecv.py:362: ResourceWarning: unclosed file <_io.BufferedReader name=6>
  a,b=sndrcv(s ,x,*args,**kargs)

======================================================================
FAIL: test_captureWithSrp (__main__.TestPingScapySrpIssue) (ping_data_size=8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "icmpv6/issue_scapy1.py", line 111, in test_captureWithSrp
    self.assertEqual(len(answered), 1)  # fails here if ping_data_size != 0
AssertionError: 0 != 1

----------------------------------------------------------------------
Ran 2 tests in 3.098s

FAILED (failures=1)

Actually I noticed that even sniff randomly fails to capture some packets (captured fine with Wireshark) on my system. The ping issue however is systematic.

EDIT: I have solved the sniff random capture reliability issues mentioned above, it was a bug in my code (SniffThread would be started but not yet capturing when traffic was sent). Still this ping issue is still present.