store rebuild - Githubissues

GoogleCodeExporter commented 9 years ago

sometimes :

2009/08/21 09:47:02| Starting Squid Cache version LUSCA_HEAD for
x86_64-unknown-linux-gnu...
2009/08/21 09:47:02| Process ID 7349
....
2009/08/21 09:47:02| Swap maxSize 133120000 + 524288 KB, estimated 10280329
objects
2009/08/21 09:47:02| Target number of buckets: 514016
2009/08/21 09:47:02| Using 524288 Store buckets
2009/08/21 09:47:02| Max Mem  size: 524288 KB
2009/08/21 09:47:02| Max Swap size: 133120000 KB
2009/08/21 09:47:02| Local cache digest enabled; rebuild/rewrite every
3600/3600 sec
2009/08/21 09:47:02| Store logging disabled
2009/08/21 09:47:02| Rebuilding storage in /squid/cache01 (CLEAN)
2009/08/21 09:47:02| Using Least Load store dir selection
2009/08/21 09:47:02| Set Current Directory to /squid
2009/08/21 09:47:02| Loaded Icons.
2009/08/21 09:47:02| Accepting transparently proxied HTTP connections at
0.0.0.0, port 3128, FD 13.
2009/08/21 09:47:02| Accepting transparently proxied and tproxy'ied HTTP
connections at 0.0.0.0, port 3129, FD 14.
2009/08/21 09:47:02| Accepting ICP messages at 0.0.0.0, port 3130, FD 15.
2009/08/21 09:47:02| Accepting SNMP messages on port 3401, FD 16.
2009/08/21 09:47:02| WCCP Disabled.
2009/08/21 09:47:02| Accepting WCCPv2 messages on port 2048, FD 17.
2009/08/21 09:47:02| Initialising all WCCPv2 lists
2009/08/21 09:47:02| Ready to serve requests.
2009/08/21 09:47:03| WARNING: 1 invalid swap log entries found
2009/08/21 09:47:03| WARNING: 10 invalid swap log entries found
2009/08/21 09:47:03| WARNING: 100 invalid swap log entries found
2009/08/21 09:47:03| WARNING: 10000 invalid swap log entries found
2009/08/21 09:47:03| WARNING: 100000 invalid swap log entries found
2009/08/21 09:47:05| storeAufsRebuildHelperRead: /squid/cache01: read
returned 0; error/eof?
2009/08/21 09:47:05| Done scanning /squid/cache01 (4481129 entries)
2009/08/21 09:47:05| Finished rebuilding storage from disk.
2009/08/21 09:47:05|   4481129 Entries scanned
2009/08/21 09:47:05|   4481129 Invalid entries.
2009/08/21 09:47:05|         0 With invalid flags.
2009/08/21 09:47:05|         0 Objects loaded.
2009/08/21 09:47:05|         0 Objects expired.
2009/08/21 09:47:05|         0 Objects cancelled.
2009/08/21 09:47:05|         0 Duplicate URLs purged.
2009/08/21 09:47:05|         0 Swapfile clashes avoided.
2009/08/21 09:47:05|   Took 2.4 seconds (   0.0 objects/sec).
2009/08/21 09:47:05| Beginning Validation Procedure
2009/08/21 09:47:05|   Completed Validation Procedure
2009/08/21 09:47:05|   Validated 0 Entries
2009/08/21 09:47:05|   store_swap_size = 0k
2009/08/21 09:47:05| storeLateRelease: released 0 objects
2009/08/21 09:47:57| Preparing for shutdown after 0 requests
2009/08/21 09:47:57| Waiting 30 seconds for active connections to finish
2009/08/21 09:47:57| FD 13 Closing HTTP connection
2009/08/21 09:47:57| FD 14 Closing HTTP connection
2009/08/21 09:47:57| FD 17 Closing WCCP socket
2009/08/21 09:48:28| Shutting down...
2009/08/21 09:48:28| FD 15 Closing ICP connection
2009/08/21 09:48:28| FD 16 Closing SNMP socket
2009/08/21 09:48:28| aioSync: flushing pending I/O operations
2009/08/21 09:48:28| aioSync: done
2009/08/21 09:48:28| Closing unlinkd pipe on FD 10
2009/08/21 09:48:28| storeDirWriteCleanLogs: Starting...
2009/08/21 09:48:28|   Finished.  Wrote 0 entries.
2009/08/21 09:48:28|   Took 0.0 seconds (   0.0 entries/sec).

Original issue reported on code.google.com by ajcor...@gmail.com on 21 Aug 2009 at 12:50

GoogleCodeExporter commented 9 years ago

i think that it´s because tools that make rebuild storage are broken... 

ufs_log_build[19199]: segfault at 1 ip 000000316c062134 sp 00007fffc7a671f0 
error 4
in libc-2.5.so[316c000000+14c000]
ufs_log_build[19302]: segfault at 1 ip 000000316c062134 sp 00007fffc16a1e20 
error 4
in libc-2.5.so[316c000000+14c000]
ufs_log_build[19319]: segfault at 1 ip 000000316c062134 sp 00007fff0ac3d3c0 
error 4
in libc-2.5.so[316c000000+14c000]
ufs_log_build[19531]: segfault at 1 ip 000000316c062134 sp 00007fff62db9550 
error 4
in libc-2.5.so[316c000000+14c000]
ufs_log_build[19595]: segfault at 1 ip 000000316c062134 sp 00007fff0f3acb40 
error 4
in libc-2.5.so[316c000000+14c000]

Original comment by ajcor...@gmail.com on 21 Aug 2009 at 5:54

Added labels: ****
Removed labels: ****

GoogleCodeExporter commented 9 years ago

[root@looney util]# uname -a
Linux looney.onda.net.br 2.6.29.4-onda #1 SMP Thu May 28 03:24:12 BRT 2009 
x86_64
x86_64 x86_64 GNU/Linux

[root@looney util]# squid -v
Squid Cache: Version LUSCA_HEAD
configure options: '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr'
'--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc'
'--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec'
'--sharedstatedir=/usr/com' '--mandir=/usr/share/man' 
'--infodir=/usr/share/info'
'--exec_prefix=/usr' '--bindir=/usr/sbin' '--libexecdir=/usr/lib64/squid'
'--localstatedir=/var' '--datadir=/usr/share' '--sysconfdir=/etc/squid'
'--enable-epoll' '--enable-snmp' '--enable-removal-policies=heap,lru'
'--enable-storeio=aufs,coss,null' '--enable-ssl' '--with-openssl=/usr/kerberos'
'--disable-delay-pools' '--with-pthreads' '--disable-useragent-log'
'--enable-referer-log' '--disable-dependency-tracking'
'--enable-cachemgr-hostname=localhost' '--enable-cache-digests'
'--disable-ident-lookups' '--enable-wccpv2' '--with-maxfd=32768'
'--enable-xmalloc-statistics' '--enable-async-io' 
'--enable-follow-x-forwarded-for'
'--enable-large-cache-files' '--with-large-files' '--enable-linux-tproxy4'
'--enable-linux-netfilter'

Original comment by ajcor...@gmail.com on 22 Aug 2009 at 5:28

Added labels: ****
Removed labels: ****

GoogleCodeExporter commented 9 years ago

Patch to solve Segmentation Fault and i think this can resolve rebuild errors
(because helper not dying more).

Index: tools/ufs_log_cat.c
===================================================================
--- tools/ufs_log_cat.c (revision 14289)
+++ tools/ufs_log_cat.c (working copy)
@@ -140,7 +140,7 @@
  }

  /* begin echo'ing the log info */
- storeSwapLogPrintHeader(1);  /* to stdout */
+ //storeSwapLogPrintHeader(1); /* to stdout */

  /* Now - loop over until eof or error */
  while (! feof(fp)) {
Index: tools/ufs_log_build.c
===================================================================
--- tools/ufs_log_build.c   (revision 14289)
+++ tools/ufs_log_build.c   (working copy)
@@ -254,11 +254,10 @@
  printf("Usage: %s <store path> <l1> <l2> <path to swapfile>\n", argv[0]);
  exit(1);
     }
-
     store_ufs_init(&store_ufs_info, argv[1], atoi(argv[2]), atoi(argv[3]), argv[4]);

     /* Output swap header to stdout */
-    (void) storeSwapLogPrintHeader(1);
+    //(void) storeSwapLogPrintHeader(1);

     read_dir(&store_ufs_info);
     store_ufs_done(&store_ufs_info);

Original comment by ajcor...@gmail.com on 22 Aug 2009 at 5:29

Added labels: ****
Removed labels: ****

GoogleCodeExporter commented 9 years ago

i fixed the relevant helpers to pass the correct parameters in. Please try now!

Original comment by adrian.c...@gmail.com on 11 Sep 2009 at 9:14

Added labels: Version-Head
Removed labels: Version-1.0

GoogleCodeExporter commented 9 years ago

Fixed a while ago.

Original comment by adrian.c...@gmail.com on 19 Jan 2010 at 9:03

Changed state: Fixed
Added labels: ****
Removed labels: ****

inevity / lusca-cache

store rebuild #53