Near Hits High ratio

[GoogleCodeExporter]

Hi Guys 
I use Lusca Head latest and i have the following symtoms ...

Very High ratio of near hits, why as the websites takes ages to open
whereas if i go direct without the cache the internet is very much more
better ....

here's the configuration file .
incoming_rate 10
pipeline_prefetch on
cache_mem 1024 MB
ipcache_size 8196
maximum_object_size 600 MB
refresh_all_ims off
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
#acl denyme dst 10.1.1.2
#http_access deny denyme
acl all src 0.0.0.0/0
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src 213.254.232.0/25
acl localnet src 212.30.32.0/19
acl SSL_ports port 443
acl Safe_ports port 80      # http
acl Safe_ports port 21      # ftp
acl Safe_ports port 443     # https
acl Safe_ports port 70      # gopher
acl Safe_ports port 210     # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280     # http-mgmt
acl Safe_ports port 488     # gss-http
acl Safe_ports port 591     # filemaker
acl Safe_ports port 777     # multiling http
acl CONNECT method CONNECT
http_access allow manager all
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access deny all
http_port 3128 tproxy  
#transparent
http_port 3129 transparent
hierarchy_stoplist cgi-bin ?
refresh_pattern ^ftp:       1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern .       0   20% 4320
coredump_dir none
#/var/spool/squid/cache
cache_store_log none
cache_dir aufs /srv/cache 300000 64 256
visible_hostname c1.comnet.net.lb
access_log none
#/var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
wccp2_router  212.30.38.1
wccp2_rebuild_wait off
#wccp_version 4
#wccp2_forwarding_method 3
#wccp2_return_method 3
wccp2_service standard 0
#wccp2_assignment_method hash
#tcp_outgoing_address 213.254.232.98
via on 
forwarded_for on
cache_effective_user squid

and here is the squidclient mgr:info 
Squid Object Cache: Version LUSCA_HEAD
Start Time: Thu, 02 Apr 2009 15:31:26 GMT
Current Time:   Fri, 03 Apr 2009 07:20:33 GMT
Connection information for squid:
    Number of clients accessing cache:  23
    Number of HTTP requests received:   1316301
    Number of ICP messages received:    0
    Number of ICP messages sent:    0
    Number of queued ICP replies:   0
    Request failure ratio:   0.00
    Average HTTP requests per minute since start:   1386.9
    Average ICP messages per minute since start:    0.0
    Select loop called: 19373542 times, 2.939 ms avg
Cache information for squid:
    Request Hit Ratios: 5min: 14.5%, 60min: 16.6%
    Byte Hit Ratios:    5min: 27.1%, 60min: 20.6%
    Request Memory Hit Ratios:  5min: 17.3%, 60min: 16.5%
    Request Disk Hit Ratios:    5min: 32.4%, 60min: 38.0%
    Storage Swap size:  19093324 KB
    Storage Mem size:   878988 KB
    Mean Object Size:   16.42 KB
    Requests given to unlinkd:  0
Median Service Times (seconds)  5 min    60 min:
    HTTP Requests (All):   0.32154  0.32154
    Cache Misses:          0.33943  0.33943
    Cache Hits:            0.00000  0.00000
    Near Hits:             8.22659  3.66001
    Not-Modified Replies:  0.00000  0.00000
    DNS Lookups:           0.00000  0.00000
    ICP Queries:           0.00000  0.00000

notice the very high value for Near Hits 

and here is the compilation command 
 squid -v
Squid Cache: Version LUSCA_HEAD
configure options:  '--prefix=/usr' '--exec_prefix=/usr'
'--bindir=/usr/sbin' '--sbindir=/usr/sbin' '--libexecdir=/usr/lib/squid'
'--sysconfdir=/etc/squid' '--localstatedir=/var/spool/squid'
'--datadir=/usr/share/squid' '--enable-async-io' '--with-pthreads'
'--enable-storeio=ufs,aufs,diskd' '--enable-arp-acl'
'--enable-removal-policies=lru,heap' '--enable-snmp' '--enable-delay-pools'
'--disable-htcp' '--enable-cache-digests' '--enable-referer-log'
'--enable-follow-x-forwarded-for' '--with-large-files' '--with-maxfd=65536'
'CFLAGS=-Wall -g -O3 -DNUMTHREADS=60 -pipe -fomit-frame-pointer
-funroll-loops -ffast-math -fno-exceptions'
'LDFLAGS=-Wl,-Bsymbolic-functions' 'CPPFLAGS=' '--enable-truncate'
'--enable-linux-netfilter' '--enable-linux-tproxy4'

yet also i have to say 
tproxy4 is not functioning still .....

much regards

Original issue reported on code.google.com by degreane@gmail.com on 3 Apr 2009 at 7:26

[GoogleCodeExporter]

also i have to add that i see lots of 
2009/04/03 10:37:25| httpReadReply: FD 214: skipping 1xx response!
2009/04/03 10:37:25| httpReadReply: FD 214: skipping 1xx response!
2009/04/03 10:37:27| httpReadReply: FD 111: skipping 1xx response!
2009/04/03 10:37:27| httpReadReply: FD 413: skipping 1xx response!
2009/04/03 10:37:28| httpReadReply: FD 413: skipping 1xx response!
2009/04/03 10:37:28| httpReadReply: FD 111: skipping 1xx response!
2009/04/03 10:37:28| httpReadReply: FD 111: skipping 1xx response!
2009/04/03 10:37:30| httpReadReply: FD 111: skipping 1xx response!
2009/04/03 10:37:35| httpReadReply: FD 229: skipping 1xx response!
2009/04/03 10:37:35| httpReadReply: FD 267: skipping 1xx response!
2009/04/03 10:37:35| httpReadReply: FD 267: skipping 1xx response!
2009/04/03 10:37:35| httpReadReply: FD 229: skipping 1xx response!
2009/04/03 10:37:39| httpReadReply: FD 513: skipping 1xx response!
2009/04/03 10:37:39| httpReadReply: FD 389: skipping 1xx response!
2009/04/03 10:37:39| httpReadReply: FD 389: skipping 1xx response!
2009/04/03 10:37:39| httpReadReply: FD 513: skipping 1xx response!
2009/04/03 10:37:40| httpReadReply: FD 111: skipping 1xx response!
2009/04/03 10:37:42| httpReadReply: FD 386: skipping 1xx response!
2009/04/03 10:37:43| httpReadReply: FD 386: skipping 1xx response!
2009/04/03 10:37:44| httpReadReply: FD 322: skipping 1xx response!
2009/04/03 10:37:44| httpReadReply: FD 563: skipping 1xx response!
2009/04/03 10:37:44| httpReadReply: FD 322: skipping 1xx response!
2009/04/03 10:37:45| httpReadReply: FD 563: skipping 1xx response!
2009/04/03 10:37:46| httpReadReply: FD 244: skipping 1xx response!
2009/04/03 10:37:46| httpReadReply: FD 300: skipping 1xx response!
2009/04/03 10:37:47| httpReadReply: FD 244: skipping 1xx response!

why ? and what are these ????
besides i had to disable-htcp in order to get rid of the ctx-level 0 stuff 

regards

Original comment by degreane@gmail.com on 3 Apr 2009 at 7:45

[GoogleCodeExporter]

Try applying this diff:

Index: http.c
===================================================================
--- http.c      (revision 13904)
+++ http.c      (working copy)
@@ -1125,7 +1125,7 @@

        /* Did we get a successful parse but 1xx? Try again */
         if (reply->sline.status >= 100 && reply->sline.status < 200) {
-               debug(1, 1) ("httpReadReply: FD %d: skipping 1xx response!\n", 
fd);
+               debug(1, 2) ("httpReadReply: FD %d: skipping 1xx response!\n", 
fd);
                httpReplyReset(reply);
                httpState->reply_hdr_state = 0;
                po += done;             /* Skip the reply in the incoming buffer */

I wonder if the 1xx skipping code is properly working. I need to find a site 
which is triggering that particular 
error. Hm..

Original comment by adrian.c...@gmail.com on 3 Apr 2009 at 7:57

[GoogleCodeExporter]

ok but this doesn't actually fix the error ...mmmmmm 
what about the high ratio of near hits ????

regards

Original comment by degreane@gmail.com on 3 Apr 2009 at 8:15

[GoogleCodeExporter]

and still the dns stat doesn't show ??? why ????
regards

Original comment by degreane@gmail.com on 3 Apr 2009 at 8:23

[GoogleCodeExporter]

re dns stats - http://code.google.com/p/lusca-cache/issues/detail?id=8

re - near hits? no idea. Will look at that next.

re - "error" - does it quieten the logging in cache.log ? Excessive cache.log 
logging will affect throughput quite 
significantly..

Original comment by adrian.c...@gmail.com on 3 Apr 2009 at 11:32

[GoogleCodeExporter]

Try this:

Index: http.c
===================================================================
--- http.c      (revision 13911)
+++ http.c      (working copy)
@@ -1125,7 +1125,10 @@

        /* Did we get a successful parse but 1xx? Try again */
         if (reply->sline.status >= 100 && reply->sline.status < 200) {
+               Ctx ctx = ctx_enter(entry->mem_obj->url);
                debug(1, 1) ("httpReadReply: FD %d: skipping 1xx response!\n", fd);
+               debug(1, 1) ("httpReadReply: FD %d: reply 
was:\n===\n%.*s\n====\n", fd, done, 
buf_buf(httpState->read_buf) + po);
+               ctx_exit(ctx);
                httpReplyReset(reply);
                httpState->reply_hdr_state = 0;
                po += done;             /* Skip the reply in the incoming buffer */

I specifically want to see what URLs you're fetching from so I can try and 
reproduce it locally!

Original comment by adrian.c...@gmail.com on 5 Apr 2009 at 8:38