FreeBSD 7.1 Stable. AMD64 Memory leak

[GoogleCodeExporter]

What steps will reproduce the problem?
Find below. 

What is the expected output? What do you see instead?
Works great on i386.  Very big memory leak on 64bit. 

What version of the product are you using? On what operating system?
FreeBSD 7.1 amd64 release. flow-tools 0.68.4.1

Please provide any additional information below.
All the BSD(amd64) people will appreciate the patch. 

Thanks.

Original issue reported on code.google.com by inkon...@gmail.com on 9 Feb 2009 at 9:41

[GoogleCodeExporter]

I also am experiencing this issue. Running the flow-tools-ng port, which says 
it is
0.68.4.1

I have to restart my flow-capture process every couple of hours or it quickly 
starts
consuming all the ram and swap in the machine and leads to operating system
failure... and this machine has 8GB of RAM and 16GB of swap space...

What can I do to help track down where the problem is?

Original comment by masas...@gmail.com on 13 Mar 2009 at 3:31

[GoogleCodeExporter]

Hello,

I have same problem on FreeBSD 7.0 amd64 release. Is anybody fix this problem ?

Original comment by 0k.a.b...@gmail.com on 13 May 2009 at 8:23

[GoogleCodeExporter]

Any news?

Original comment by Oleg.Gaw...@gmail.com on 6 Aug 2009 at 7:32

[GoogleCodeExporter]

Here comes a very ugly, but effective patch... Use it at your own risk...
*** flow-capture.c.orig Mon Jan 05 17:56:50 2005
--- flow-capture.c      Fri Aug 14 19:42:41 2009
***************                                 
*** 23,29 ****                                  
   * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
   * SUCH DAMAGE.                                                          
   *                                                                       
!  *      $Id: flow-capture.c,v 1.78 2004/01/05 17:56:50 maf Exp $         
   */                                                                      

  #include "ftconfig.h"                                                    
--- 23,29 ----                                                             
   * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
   * SUCH DAMAGE.                                                          
   *                                                                       
!  *      $Id: flow-capture.c,v 1.79 2009/08/14 19:42:00 zvladov Exp $     
   */                                                                      

  #include "ftconfig.h"                                                    
***************                                                            
*** 855,861 ****                                                           
        /* got destination IP back? */                                     
        if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&                  
            (ftnet.msgip.hdr.cmsg_type == IP_RECVDSTADDR)) {               
!           ftnet.loc_addr.sin_addr.s_addr = ftnet.msgip.ip.s_addr;        
        } else {                                                           
          ftnet.loc_addr.sin_addr.s_addr = 0;                              
        }                                                                  
--- 855,861 ----                                                           
        /* got destination IP back? */                                     
        if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&                  
            (ftnet.msgip.hdr.cmsg_type == IP_RECVDSTADDR)) {               
! /*          ftnet.loc_addr.sin_addr.s_addr = ftnet.msgip.ip.s_addr; */   
        } else {                                                           
          ftnet.loc_addr.sin_addr.s_addr = 0;                              
        }                                                                  
***************                                                            
*** 863,869 ****                                                           
  #ifdef IP_PKTINFO                                                        
        if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&                  
            (ftnet.msgip.hdr.cmsg_type == IP_PKTINFO)) {                   
!           ftnet.loc_addr.sin_addr.s_addr = 
ftnet.msgip.pktinfo.ipi_addr.s_addr;
        } else {                                                                 
          ftnet.loc_addr.sin_addr.s_addr = 0;                                    
        }                                                                        
--- 863,869 ----                                                                

  #ifdef IP_PKTINFO                                                              
        if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&                        
            (ftnet.msgip.hdr.cmsg_type == IP_PKTINFO)) {                         
! /*          ftnet.loc_addr.sin_addr.s_addr = 
ftnet.msgip.pktinfo.ipi_addr.s_addr; */
        } else {                                                                      
          ftnet.loc_addr.sin_addr.s_addr = 0;                                         
        }                                                                             

Original comment by zvla...@gmail.com on 14 Aug 2009 at 4:50

[GoogleCodeExporter]

Ah, and yes, it is not a memory leak... but some problems with IP headers 
decoding
on amd64 (for now)...

Original comment by zvla...@gmail.com on 14 Aug 2009 at 4:51

[GoogleCodeExporter]

Mark R. mark at inetu.net 
Tue Oct 7 14:53:09 EDT 2008 

Previous message: [Flow-tools] ftpdu_seq_check lost flows 
Next message: [Flow-tools] limit for number of filter elements 
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] 

--------------------------------------------------------------------------------

Paul,

Upon further investigation, it appears as if there is an alignment issue in 
'struct msgip' inside of 'struct ftnet'.  I've looked at some other code, and 
there appear to be a handful of useful CMSG_* macros to help deal with this. 
RFC 2292 details them.

The attached patch works cleanly on my system and fixes the issue at hand. 
Hopefully it doesn't break other platforms.  It's there for anybody who is 
interested.

Thanks,
Mark

On Tue, 7 Oct 2008, Mark R. wrote:

> 
> Paul,
> 
> This is where I'm picking up the garbage data:
>
>   ftnet.loc_addr.sin_addr.s_addr = ftnet.msgip.ip.s_addr
> 
> The kernel is GENERIC plus IPFW.  Nothing terribly oddball.  I'm wondering if 
> this is a variable type issue somewhere.  Running a 32bit build on amd64 and 
> the same box doesn't exhibit this issue.
> 
> If you care to look into this, I can provide access to the box in question. 
> If not, I'll might eventually figure it out.
> 
> I'm not at the point of caring about a memory leak.  When I get there, I can 
> run it through valgrind.
> 
> Thanks,
> Mark
> 
> On Tue, 7 Oct 2008, Paul P Komkoff Jr wrote:
> 
>> Replying to Mark R.:
>>> Are there any known issues with flow-tools on 64-bit platforms?  I'm
>>> trying to run 0.68 on FreeBSD 7.0/amd64 and running into some odd
>>> behavior with flow-capture and flow-fanout.
>> 
>> There is at least one known memory leak that only happens on 64-bit
>> FreeBSD, that I still haven't fixed (mainly because it requires to set up
>> FreeBSD somewhere).
>> 
>> This one that you telling here is new. And it's serious, I wonder why
>> nobody else seeing it. Maybe your kernel is broken?
>> Or maybe it's ipv4-over-ipv6 issue?
>> I don't have access to freebsd system to verify how recvmsg works
>> there, sorry.
>> 
>> -- 
>> Paul P 'Stingray' Komkoff Jr // http://stingr.net/key <- my pgp key
>> This message represents the official view of the voices in my head
>> 
> _______________________________________________
> Flow-tools mailing list
> flow-tools at splintered.net
> http://mailman.splintered.net/mailman/listinfo/flow-tools
-------------- next part --------------
--- lib/ftlib.h-orig    2008-03-09 13:09:01.000000000 +0000
+++ lib/ftlib.h 2008-10-07 17:41:11.000000000 +0000
@@ -459,11 +459,16 @@
   int fd;                         /* fd receiving flows on */
   struct mymsghdr msg;            /* recvmsg data */
   struct {
-    struct cmsghdr hdr;
 #ifdef IP_RECVDSTADDR
+#ifdef CMSG_DATA
+    char cbuf[CMSG_SPACE(sizeof(struct sockaddr_storage))];
+#else
+    struct cmsghdr hdr;
     struct in_addr ip;
+#endif /* CMSG_DATA */
 #else
 #ifdef IP_PKTINFO
+    struct cmsghdr hdr;
     struct in_pktinfo pktinfo;
 #endif /* else */
 #endif /* IP RECVDSTADDR */
--- src/flow-fanout.c-orig  2008-01-27 20:48:55.000000000 +0000
+++ src/flow-fanout.c   2008-10-07 17:42:33.000000000 +0000
@@ -124,6 +124,11 @@
   int i, n, detach, one, ret, offset, hdr_len;
   int npeers, tx_delay;
   int stat_interval, stat_next, src_ip_spoof;
+#ifdef IP_RECVDSTADDR
+#ifdef CMSG_DATA
+  struct cmsghdr *cmsg;
+#endif
+#endif

   time_startup = time((time_t)0L);

@@ -625,12 +630,24 @@

 #ifdef IP_RECVDSTADDR
       /* got destination IP back? */
+#ifdef CMSG_DATA
+      for (cmsg = CMSG_FIRSTHDR(&ftnet.msg); cmsg != NULL;
+          cmsg = CMSG_NXTHDR(&ftnet.msg, cmsg)) {
+              if (cmsg->cmsg_level == IPPROTO_IP &&
+                  cmsg->cmsg_type == IP_RECVDSTADDR) {
+                      memcpy(&ftnet.loc_addr.sin_addr.s_addr,
+                          CMSG_DATA(cmsg), sizeof(struct in_addr));
+                      break;
+              }
+      }
+#else      
       if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&
           (ftnet.msgip.hdr.cmsg_type == IP_RECVDSTADDR)) {
           ftnet.loc_addr.sin_addr.s_addr = ftnet.msgip.ip.s_addr;
       } else {
         ftnet.loc_addr.sin_addr.s_addr = 0;
       }
+#endif /* CMSG_DATA */
 #else
 #ifdef IP_PKTINFO
       if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&
--- src/flow-receive.c-orig 2008-01-27 20:48:55.000000000 +0000
+++ src/flow-receive.c  2008-10-07 17:42:35.000000000 +0000
@@ -93,6 +93,11 @@
   char fmt_src_ip[32], fmt_dst_ip[32], fmt_dst_port[32];
   char xl_rec[FT_IO_MAXREC], *out_rec;
   int stat_interval, stat_next;
+#ifdef IP_RECVDSTADDR
+#ifdef CMSG_DATA
+  struct cmsghdr *cmsg;
+#endif
+#endif

   time_startup = time((time_t)0L);

@@ -488,12 +493,24 @@

 #ifdef IP_RECVDSTADDR
       /* got destination IP back? */
+#ifdef CMSG_DATA
+      for (cmsg = CMSG_FIRSTHDR(&ftnet.msg); cmsg != NULL;
+          cmsg = CMSG_NXTHDR(&ftnet.msg, cmsg)) {
+              if (cmsg->cmsg_level == IPPROTO_IP &&
+                  cmsg->cmsg_type == IP_RECVDSTADDR) {
+                      memcpy(&ftnet.loc_addr.sin_addr.s_addr,
+                          CMSG_DATA(cmsg), sizeof(struct in_addr));
+                      break;
+              }
+      }
+#else      
       if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&
           (ftnet.msgip.hdr.cmsg_type == IP_RECVDSTADDR)) {
           ftnet.loc_addr.sin_addr.s_addr = ftnet.msgip.ip.s_addr;
       } else {
         ftnet.loc_addr.sin_addr.s_addr = 0;
       }
+#endif /* CMSG_DATA */
 #else
 #ifdef IP_PKTINFO
       if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&
--- src/flow-capture.c-orig 2008-01-27 20:48:55.000000000 +0000
+++ src/flow-capture.c  2008-10-07 17:42:30.000000000 +0000
@@ -170,6 +170,11 @@
   int stat_interval, stat_next, child_status;
   int v_flag;
   int preserve_umask;
+#ifdef IP_RECVDSTADDR
+#ifdef CMSG_DATA
+  struct cmsghdr *cmsg;
+#endif
+#endif

   time_startup = time((time_t)0L);

@@ -621,7 +626,12 @@
   ftnet.msg.msg_name = &ftnet.rem_addr;
   ftnet.msg.msg_namelen = sizeof ftnet.rem_addr;
   ftnet.msg.msg_control = &ftnet.msgip;
+  
+#ifdef CMSG_DATA
+  ftnet.msg.msg_controllen = CMSG_LEN(sizeof(struct sockaddr_storage));
+#else
   ftnet.msg.msg_controllen = sizeof ftnet.msgip;
+#endif

   while (1) {

@@ -853,12 +863,24 @@

 #ifdef IP_RECVDSTADDR
       /* got destination IP back? */
+#ifdef CMSG_DATA
+      for (cmsg = CMSG_FIRSTHDR(&ftnet.msg); cmsg != NULL;
+          cmsg = CMSG_NXTHDR(&ftnet.msg, cmsg)) {
+              if (cmsg->cmsg_level == IPPROTO_IP &&
+                  cmsg->cmsg_type == IP_RECVDSTADDR) {
+                      memcpy(&ftnet.loc_addr.sin_addr.s_addr,
+                          CMSG_DATA(cmsg), sizeof(struct in_addr));
+                      break;
+              }
+      }
+#else      
       if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&
           (ftnet.msgip.hdr.cmsg_type == IP_RECVDSTADDR)) {
           ftnet.loc_addr.sin_addr.s_addr = ftnet.msgip.ip.s_addr;
       } else {
         ftnet.loc_addr.sin_addr.s_addr = 0;
       }
+#endif /* CMSG_DATA */
 #else
 #ifdef IP_PKTINFO
       if ((ftnet.msgip.hdr.cmsg_level == IPPROTO_IP) &&

Original comment by zvla...@gmail.com on 15 Aug 2009 at 6:50

[GoogleCodeExporter]

Thank You very much.

Original comment by inkon...@gmail.com on 1 Sep 2009 at 8:14

[GoogleCodeExporter]

Tested it on 7.2 amd64. All works perfectly, thanks.
I've make a PR to FreeBSD ports tree to include this.

Original comment by Oleg.Gaw...@gmail.com on 14 Sep 2009 at 12:14

[GoogleCodeExporter]

Hello Oleg Gawriloff,

I've mailed to ports@freebsd.org about the issue, almost 1 month ago, and
here is what I've got, not even e-mailed back to my private e-mail...:

--------------------------------------------------------------------------------
---
Re: Flow-tools and Flow-tools-ng - Flow-Capture - supposed to be memory leak - 
patch

Volodymyr Kostyrko
Mon, 17 Aug 2009 13:35:03 -0700

Zvezdelin Vladov wrote:

    Dear Sir/Madam,

    Please, publish in the official ports build patch system, the patch
    bellow for the problem
    that manifest itself only on the amd64 platform, and it is one and the
    same for both the
    flow-tools and flow-tools-ng - i.e. flow-capture eats all of the RAM
    and SWAP of the
    machine, until killed by the kernel.

I'm sorry, but as a maintainer of net-mgmt/flow-tools-ng I'm not going to 
accept this
patch. It goes far beyond than simple build fix, it changes some internal 
structures
in weird way (from my point of view) and I can't predict how this would work.

You have already posted the patch on the project issue tracker and this is 
indeed a
correct way. I'm watching it and will bring the final fix or new version to the 
ports
ASAP.

--
Sphinx of black quartz judge my vow.

_______________________________________________
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Original comment by zvla...@gmail.com on 16 Sep 2009 at 9:17

[GoogleCodeExporter]

I've just sent a pr for port update, latest version includes mentioned patch.

Original comment by c.kw...@gmail.com on 28 Sep 2009 at 8:23

[GoogleCodeExporter]

Original comment by therap...@gmail.com on 24 Feb 2010 at 1:02

• Changed state: Accepted
• Added labels: Priority-High
• Removed labels: Priority-Medium