search

zendesk / ultragrep

the grep that greps the hardest.
Apache License 2.0
29 stars 4 forks source link

do not read from dead workers #11

Closed grosser closed 11 years ago

grosser commented 11 years ago

Reduces the number of threads and do not read from dead workers, since we already waitall and then add new pipes

@osheroff @vanchi-zendesk

osheroff commented 11 years ago

This pr just has a syntactical change: miss a commit?

grosser commented 11 years ago

It moves the children_pipes inside the block -> no longer keeps old pipes from last iteration around

osheroff commented 11 years ago

ah. yeah, you're right. have you tested in prod?

grosser commented 11 years ago

Just tried, getting this:

*** [err :: logs03.pod1] make
*** [err :: logs03.pod1] gcc -I/opt/local/include -g -c ug_guts.c
*** [err :: logs03.pod1] gcc -I/opt/local/include -g -c rails_req.c
*** [err :: logs03.pod1] gcc -I/opt/local/include -g -c work_req.c
*** [err :: logs03.pod1] gcc -I/opt/local/include -g -c request.c
*** [err :: logs03.pod1] gcc -I/opt/local/include -g -o ug_guts request.o
rails_req.o work_req.o ug_guts.o -lpcre
*** [err :: logs03.pod1] gcc -I/opt/local/include -g -c ug_build_index.c
*** [err :: logs03.pod1] ug_build_index.c: In function ‘main’:
*** [err :: logs03.pod1] ug_build_index.c:45:9: warning: passing argument 1
of ‘work_req_matcher’ from incompatible pointer type [enabled by default]
*** [err :: logs03.pod1] work_req.h:5:16: note: expected ‘on_req’ but
argument is of type ‘void (*)(struct request_t *, struct
build_idx_context_t *)’
*** [err :: logs03.pod1] ug_build_index.c:49:9: warning: passing argument 1
of ‘rails_req_matcher’ from incompatible pointer type [enabled by default]
*** [err :: logs03.pod1] rails_req.h:5:16: note: expected ‘on_req’ but
argument is of type ‘void (*)(struct request_t *, struct
build_idx_context_t *)’
*** [err :: logs03.pod1] gcc -I/opt/local/include -g -c ug_index.c
*** [err :: logs03.pod1] gcc -g -c zran.c
*** [err :: logs03.pod1] gcc -I/opt/local/include -g -lz -o ug_build_index
request.o rails_req.o work_req.o ug_index.o ug_build_index.o zran.o -lpcre
*** [err :: logs03.pod1] zran.o: In function `build_gz_index':
*** [err :: logs03.pod1]
/usr/local/rvm/gems/ruby-1.9.3-p429/gems/ultragrep-0.0.0/ext/ultragrep/zran.c:67:
undefined reference to `inflateInit2_'
*** [err :: logs03.pod1]
/usr/local/rvm/gems/ruby-1.9.3-p429/gems/ultragrep-0.0.0/ext/ultragrep/zran.c:100:
undefined reference to `inflate'
*** [err :: logs03.pod1]
/usr/local/rvm/gems/ruby-1.9.3-p429/gems/ultragrep-0.0.0/ext/ultragrep/zran.c:171:
undefined reference to `inflateEnd'
*** [err :: logs03.pod1]
/usr/local/rvm/gems/ruby-1.9.3-p429/gems/ultragrep-0.0.0/ext/ultragrep/zran.c:176:
undefined reference to `inflateEnd'
*** [err :: logs03.pod1] collect2: ld returned 1 exit status
*** [err :: logs03.pod1] make: *** [ug_build_index] Error 1

On Thu, Jun 6, 2013 at 9:58 AM, osheroff notifications@github.com wrote:

ah. yeah, you're right. have you tested in prod?

— Reply to this email directly or view it on GitHubhttps://github.com/zendesk/ultragrep/pull/11#issuecomment-19058992 .

grosser commented 11 years ago

seems to work