start: Unknown job: kafka

[drewbroadley]

After installing on Ubuntu 10.04-LTS, I receive this message when trying to start kafka.

Install of dpkg output:

drew@broa01vm1:~$ dpkg -i kafka_0.7.2_all.deb dpkg: requested operation requires superuser privilege drew@broa01vm1:~$ sudo dpkg -i kafka_0.7.2_all.deb Selecting previously deselected package kafka. (Reading database ... 129256 files and directories currently installed.) Unpacking kafka (from kafka_0.7.2_all.deb) ... Setting up kafka (0.7.2) ... Processing triggers for ureadahead ... ureadahead will be reprofiled on next reboot

[drewbroadley]

Zookeeper and Kakfa can be started fine without init.d script:

$ /usr/lib/kafka/bin/zookeeper-server-start.sh /etc/kafka/zookeeper.properties & $ /usr/lib/kafka/bin/kafka-server-start.sh /etc/kafka/server.properties &

[ottomata]

Hm, did you paste the wrong output? You pasted the dpkg -i output, which doesn't say anything about starting kafka.

[drewbroadley]

:That is all the output there is, nothing in any system log.

Here's the strace of running /etc/init.d/kafka start if it will help:

execve("/etc/init.d/kafka", ["/etc/init.d/kafka", "start"], [/* 20 vars */]) = 0 brk(0) = 0x235f000 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f290106f000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=40641, ...}) = 0 mmap(NULL, 40641, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f2901065000 close(3) = 0 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) open("/lib/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\355\1\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1584520, ...}) = 0 mmap(NULL, 3692584, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2900acc000 mprotect(0x7f2900c49000, 2093056, PROT_NONE) = 0 mmap(0x7f2900e48000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17c000) = 0x7f2900e48000 mmap(0x7f2900e4d000, 18472, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2900e4d000 close(3) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2901064000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2901063000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2901062000 arch_prctl(ARCH_SET_FS, 0x7f2901063700) = 0 mprotect(0x7f2900e48000, 16384, PROT_READ) = 0 mprotect(0x617000, 4096, PROT_READ) = 0 mprotect(0x7f2901071000, 4096, PROT_READ) = 0 munmap(0x7f2901065000, 40641) = 0 getpid() = 16937 rt_sigaction(SIGCHLD, {SIG_DFL, [CHLD], SA_RESTORER|SA_RESTART, 0x7f2900affba0}, {SIG_DFL, [], 0}, 8) = 0 geteuid() = 0 brk(0) = 0x235f000 brk(0x2380000) = 0x2380000 getppid() = 16936 stat("/home/drew", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 open("/etc/init.d/kafka", O_RDONLY) = 3 fcntl(3, F_DUPFD, 10) = 10 close(3) = 0 fcntl(10, F_SETFD, FD_CLOEXEC) = 0 rt_sigaction(SIGINT, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGINT, {0x408189, ~[RTMIN RT_1], SA_RESTORER, 0x7f2900affba0}, NULL, 8) = 0 rt_sigaction(SIGQUIT, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGQUIT, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER, 0x7f2900affba0}, NULL, 8) = 0 rt_sigaction(SIGTERM, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGTERM, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER, 0x7f2900affba0}, NULL, 8) = 0 read(10, "#!/bin/sh -e\n# upstart-job\n#\n# S"..., 8192) = 1830 pipe([3, 4]) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f29010639d0) = 16938 close(4) = 0 read(3, "kafka\n", 128) = 6 read(3, "", 128) = 0 close(3) = 0 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16938 --- SIGCHLD (Child exited) @ 0 (0) --- write(1, "Rather than invoking init script"..., 74) = 74 write(1, "utility, e.g. service kafka star"..., 34) = 34 write(1, "\n", 1) = 1 write(1, "Since the script you are attempt"..., 71) = 71 write(1, "Upstart job, you may also use th"..., 69) = 69 pipe([3, 4]) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f29010639d0) = 16939 close(4) = 0 read(3, "", 128) = 0 --- SIGCHLD (Child exited) @ 0 (0) --- close(3) = 0 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16939 stat("/usr/local/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/usr/local/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/usr/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/usr/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/sbin/start", {st_mode=S_IFREG|0755, st_size=114056, ...}) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f29010639d0) = 16942 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 16942 --- SIGCHLD (Child exited) @ 0 (0) --- exit_group(1)

On Tue, Jan 29, 2013 at 3:43 AM, Andrew Otto notifications@github.comwrote:

Hm, did you paste the wrong output? You pasted the dpkg -i output, which doesn't say anything about starting kafka.

— Reply to this email directly or view it on GitHubhttps://github.com/wmf-analytics/kafka-debian/issues/1#issuecomment-12784538.

[ottomata]

Hm,

If you are using the init/upstart script to start kafka, you should be Try starting it this way:

sudo service kafka start

/etc/init.d/kafka is just a symlink to upstart. /etc/init/kafka.conf is the real service script. Do you have a system with upstart installed?

On Jan 28, 2013, at 9:19 PM, Drew Broadley notifications@github.com wrote:

:That is all the output there is, nothing in any system log.

Here's the strace of running /etc/init.d/kafka start if it will help:

execve("/etc/init.d/kafka", ["/etc/init.d/kafka", "start"], [/* 20 vars */]) = 0 brk(0) = 0x235f000 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f290106f000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=40641, ...}) = 0 mmap(NULL, 40641, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f2901065000 close(3) = 0 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) open("/lib/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\355\1\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1584520, ...}) = 0 mmap(NULL, 3692584, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2900acc000 mprotect(0x7f2900c49000, 2093056, PROT_NONE) = 0 mmap(0x7f2900e48000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17c000) = 0x7f2900e48000 mmap(0x7f2900e4d000, 18472, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2900e4d000 close(3) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2901064000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2901063000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2901062000 arch_prctl(ARCH_SET_FS, 0x7f2901063700) = 0 mprotect(0x7f2900e48000, 16384, PROT_READ) = 0 mprotect(0x617000, 4096, PROT_READ) = 0 mprotect(0x7f2901071000, 4096, PROT_READ) = 0 munmap(0x7f2901065000, 40641) = 0 getpid() = 16937 rt_sigaction(SIGCHLD, {SIG_DFL, [CHLD], SA_RESTORER|SA_RESTART, 0x7f2900affba0}, {SIG_DFL, [], 0}, 8) = 0 geteuid() = 0 brk(0) = 0x235f000 brk(0x2380000) = 0x2380000 getppid() = 16936 stat("/home/drew", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 open("/etc/init.d/kafka", O_RDONLY) = 3 fcntl(3, F_DUPFD, 10) = 10 close(3) = 0 fcntl(10, F_SETFD, FD_CLOEXEC) = 0 rt_sigaction(SIGINT, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGINT, {0x408189, ~[RTMIN RT_1], SA_RESTORER, 0x7f2900affba0}, NULL, 8) = 0 rt_sigaction(SIGQUIT, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGQUIT, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER, 0x7f2900affba0}, NULL, 8) = 0 rt_sigaction(SIGTERM, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGTERM, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER, 0x7f2900affba0}, NULL, 8) = 0 read(10, "#!/bin/sh -e\n# upstart-job\n#\n# S"..., 8192) = 1830 pipe([3, 4]) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f29010639d0) = 16938 close(4) = 0 read(3, "kafka\n", 128) = 6 read(3, "", 128) = 0 close(3) = 0 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16938 --- SIGCHLD (Child exited) @ 0 (0) --- write(1, "Rather than invoking init script"..., 74) = 74 write(1, "utility, e.g. service kafka star"..., 34) = 34 write(1, "\n", 1) = 1 write(1, "Since the script you are attempt"..., 71) = 71 write(1, "Upstart job, you may also use th"..., 69) = 69 pipe([3, 4]) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f29010639d0) = 16939 close(4) = 0 read(3, "", 128) = 0 --- SIGCHLD (Child exited) @ 0 (0) --- close(3) = 0 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16939 stat("/usr/local/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/usr/local/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/usr/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/usr/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/sbin/start", {st_mode=S_IFREG|0755, st_size=114056, ...}) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f29010639d0) = 16942 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 16942 --- SIGCHLD (Child exited) @ 0 (0) --- exit_group(1)

On Tue, Jan 29, 2013 at 3:43 AM, Andrew Otto notifications@github.comwrote:

Hm, did you paste the wrong output? You pasted the dpkg -i output, which doesn't say anything about starting kafka.

— Reply to this email directly or view it on GitHubhttps://github.com/wmf-analytics/kafka-debian/issues/1#issuecomment-12784538.

— Reply to this email directly or view it on GitHub.

[drewbroadley]

I gave both a go, same output:

/etc/init.d/kafka start

start: Unknown job: kafka

sudo service kafka start

start: Unknown job: kafka

upstart is installed:

apt-get install upstart

Reading package lists... Done Building dependency tree Reading state information... Done upstart is already the newest version.

Let me know what else I can do to help.

On Wed, Jan 30, 2013 at 3:33 AM, Andrew Otto notifications@github.comwrote:

Hm,

If you are using the init/upstart script to start kafka, you should be Try starting it this way:

sudo service kafka start

/etc/init.d/kafka is just a symlink to upstart. /etc/init/kafka.conf is the real service script. Do you have a system with upstart installed?

On Jan 28, 2013, at 9:19 PM, Drew Broadley notifications@github.com wrote:

:That is all the output there is, nothing in any system log.

Here's the strace of running /etc/init.d/kafka start if it will help:

execve("/etc/init.d/kafka", ["/etc/init.d/kafka", "start"], [/* 20 vars */]) = 0 brk(0) = 0x235f000 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)

mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)

0x7f290106f000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=40641, ...}) = 0 mmap(NULL, 40641, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f2901065000 close(3) = 0 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) open("/lib/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\355\1\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1584520, ...}) = 0 mmap(NULL, 3692584, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2900acc000 mprotect(0x7f2900c49000, 2093056, PROT_NONE) = 0 mmap(0x7f2900e48000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17c000) = 0x7f2900e48000 mmap(0x7f2900e4d000, 18472, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2900e4d000 close(3) = 0

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)

0x7f2901064000

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)

0x7f2901063000

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)

0x7f2901062000 arch_prctl(ARCH_SET_FS, 0x7f2901063700) = 0 mprotect(0x7f2900e48000, 16384, PROT_READ) = 0 mprotect(0x617000, 4096, PROT_READ) = 0 mprotect(0x7f2901071000, 4096, PROT_READ) = 0 munmap(0x7f2901065000, 40641) = 0 getpid() = 16937 rt_sigaction(SIGCHLD, {SIG_DFL, [CHLD], SA_RESTORER|SA_RESTART, 0x7f2900affba0}, {SIG_DFL, [], 0}, 8) = 0 geteuid() = 0 brk(0) = 0x235f000 brk(0x2380000) = 0x2380000 getppid() = 16936 stat("/home/drew", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 open("/etc/init.d/kafka", O_RDONLY) = 3 fcntl(3, F_DUPFD, 10) = 10 close(3) = 0 fcntl(10, F_SETFD, FD_CLOEXEC) = 0 rt_sigaction(SIGINT, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGINT, {0x408189, ~[RTMIN RT_1], SA_RESTORER, 0x7f2900affba0}, NULL, 8) = 0 rt_sigaction(SIGQUIT, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGQUIT, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER, 0x7f2900affba0}, NULL, 8) = 0 rt_sigaction(SIGTERM, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigaction(SIGTERM, {SIG_DFL, ~[RTMIN RT_1], SA_RESTORER, 0x7f2900affba0}, NULL, 8) = 0 read(10, "#!/bin/sh -e\n# upstart-job\n#\n# S"..., 8192) = 1830 pipe([3, 4]) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f29010639d0) = 16938 close(4) = 0 read(3, "kafka\n", 128) = 6 read(3, "", 128) = 0 close(3) = 0 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16938 --- SIGCHLD (Child exited) @ 0 (0) --- write(1, "Rather than invoking init script"..., 74) = 74 write(1, "utility, e.g. service kafka star"..., 34) = 34 write(1, "\n", 1) = 1 write(1, "Since the script you are attempt"..., 71) = 71 write(1, "Upstart job, you may also use th"..., 69) = 69 pipe([3, 4]) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f29010639d0) = 16939 close(4) = 0 read(3, "", 128) = 0 --- SIGCHLD (Child exited) @ 0 (0) --- close(3) = 0 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16939 stat("/usr/local/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/usr/local/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/usr/sbin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/usr/bin/start", 0x7fffb88ff070) = -1 ENOENT (No such file or directory) stat("/sbin/start", {st_mode=S_IFREG|0755, st_size=114056, ...}) = 0 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f29010639d0) = 16942 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 16942 --- SIGCHLD (Child exited) @ 0 (0) --- exit_group(1)

On Tue, Jan 29, 2013 at 3:43 AM, Andrew Otto notifications@github.comwrote:

Hm, did you paste the wrong output? You pasted the dpkg -i output, which doesn't say anything about starting kafka.

— Reply to this email directly or view it on GitHub< https://github.com/wmf-analytics/kafka-debian/issues/1#issuecomment-12784538>.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/wmf-analytics/kafka-debian/issues/1#issuecomment-12836978.

[vide]

It happens here the same on Debian Squeeze with upstart 0.6.6 installed

start kafka

start: Unknown job: kafka

(which is what the legacy init script is executing)

[dastra]

I had the same trouble - so forked this project and wrote a standard init.d script for it. See https://github.com/dastra/kafka-debian

[vide]

Oh dastra, I was just about to do the same :)

[ottomata]

Cool! Sorry guys, I've been super busy with other stuff and haven't been able to replicate this myself. init.d script may be the way to go.

[vide]

Don't worry, anyway even if it worked, upstart for Debian makes you change a critical system component which is not a good idea in a server.

[drdee]

What's the status of this? Otto, do we need to fix this or can we close it?

[ottomata]

I suppose it is still an issue, but this repository can probably be removed, as we will be hosting it here:

https://github.com/wikimedia/operations-debs-kafka