Closed Guimli closed 8 years ago
Thank you so much @Guimli for this PR. I'll have a look and merge it if all is fine.
$ sudo make dkms
Installing Douane Linux kernel module version 0.8.2...
Creating symlink /var/lib/dkms/douane/0.8.2/source ->
/usr/src/douane-0.8.2
DKMS: add completed.
Kernel preparation unnecessary for this kernel. Skipping...
Building module:
cleaning build area....
make KERNELRELEASE=4.4.0-22-generic -C /lib/modules/4.4.0-22-generic/build M=/var/lib/dkms/douane/0.8.2/build.....(bad exit status: 2)
Error! Bad return status for module build on kernel: 4.4.0-22-generic (x86_64)
Consult /var/lib/dkms/douane/0.8.2/build/make.log for more information.
make: *** [dkms] Error 10
$ cat /var/lib/dkms/douane/0.8.2/build/make.log
DKMS make.log for douane-0.8.2 for kernel 4.4.0-22-generic (x86_64)
Mon May 9 08:04:06 EDT 2016
make[1]: Entering directory `/usr/src/linux-headers-4.4.0-22-generic'
LD /var/lib/dkms/douane/0.8.2/build/built-in.o
CC [M] /var/lib/dkms/douane/0.8.2/build/douane.o
/var/lib/dkms/douane/0.8.2/build/douane.c:1115:3: warning: initialization from incompatible pointer type [enabled by default]
.hook = netfiler_packet_hook,
^
/var/lib/dkms/douane/0.8.2/build/douane.c:1115:3: warning: (near initialization for ‘nfho_outgoing.hook’) [enabled by default]
/var/lib/dkms/douane/0.8.2/build/douane.c:1119:3: error: unknown field ‘owner’ specified in initializer
.owner = THIS_MODULE
^
/var/lib/dkms/douane/0.8.2/build/douane.c:1120:1: warning: excess elements in struct initializer [enabled by default]
};
^
/var/lib/dkms/douane/0.8.2/build/douane.c:1120:1: warning: (near initialization for ‘nfho_outgoing’) [enabled by default]
make[2]: *** [/var/lib/dkms/douane/0.8.2/build/douane.o] Error 1
make[1]: *** [_module_/var/lib/dkms/douane/0.8.2/build] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-4.4.0-22-generic'
I removed the .owner line in the structure and it compiled dkms. This works fine for kernel 3.13. However, for linux mint 17.3 64bit kernel 4.4.0.22 is locks up instantly after turning on the configurator. Sorry to say we are back at square one. I see you tested linux mint 17.2 (ubuntu 14.04) but only with kernel 3.13. Please try to get it working with kernel 4. In my configurator (when running kernel 3.13) I do not see any rules. Also there is a warning about dbus.
P.S. I am using LPFW (Leopard Flower Firewall), but it is not active right now. The rules for it go bad a lot and block everything. Other than that it works. I just have to delete the rule, launch the app, and recreate the rule.
I think you use the wrong git source to test this PullRequest. Uninstall the dkms module : sudo make cleandkms Delete your douane-dkms local source directory and retry with this Git : git clone https://github.com/Guimli/douane-dkms.git
I will try again. Any issue with me trying this in vmware? The old Douane corrupted my whole install. Took 3 reformats to get back to normal. I have a frowny face now.
On 05/09/2016 09:49 AM, Nicolas Weill wrote:
I think you use the wrong git source to test this PullRequest. Uninstall the dkms module : sudo make cleandkms Delete your douane-dkms local source directory and retry with this Git : git clone https://github.com/Guimli/douane-dkms.git
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/Douane/douane-dkms/pull/16#issuecomment-217869345
ok,
I did a sudo make cleandkms I deleted teh entire douane-dkms folder I used git to clone the below folder i did a sudo make dkms I ran the configurator.
Nothing is being blocked. The crashing has stopped. I did previously create a bunch of rules. Maybe they need to be deleted.
On 05/09/2016 09:49 AM, Nicolas Weill wrote:
I think you use the wrong git source to test this PullRequest. Uninstall the dkms module : sudo make cleandkms Delete your douane-dkms local source directory and retry with this Git : git clone https://github.com/Guimli/douane-dkms.git
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/Douane/douane-dkms/pull/16#issuecomment-217869345
found the issue...please hold on
On 05/09/2016 09:49 AM, Nicolas Weill wrote:
I think you use the wrong git source to test this PullRequest. Uninstall the dkms module : sudo make cleandkms Delete your douane-dkms local source directory and retry with this Git : git clone https://github.com/Guimli/douane-dkms.git
— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/Douane/douane-dkms/pull/16#issuecomment-217869345
After douane borked my system (i used the old code) I reinstalled. I failed to notice that I had not reinstalled all of the dpendencies. After fixing that there is no crashing and it is working fine with kernel 4. Seems to be working better than LPFW. However, in the configurator there are no rules. Where are the rules?
Never mind. I found the refresh button. I see my rules now.
I think you have replaced LPFW officially. I will use Douane for a week and if all is fine I will include your product in version #2 of my ebook on linux mint.
Speak of the devil. Right after the last note my whole system locked up bigger than shit. Is there some sort of log file that will tell me what crashed?
Here is some of /var/log/douane.log:
10/05/2016 08:29:05 | daemon | DEBUG | [rules_manager.cpp::RulesManager:30]: An allowing rule found for /usr/lib/firefox/firefox
10/05/2016 08:29:06 | daemon | DEBUG | [netlink_message_handler.cpp::NetlinkMessageHandler:7]: [] TCP 192.168.1.130:58220 -> 54.186.241.180:443 with a size of 52 for process /usr/lib/firefox
/firefox
10/05/2016 08:29:06 | daemon | DEBUG | [rules_manager.cpp::RulesManager:30]: An allowing rule found for /usr/lib/firefox/firefox
10/05/2016 08:29:07 | daemon | DEBUG | [netlink_message_handler.cpp::NetlinkMessageHandler:7]: [^FE?^F??z?ki??U??@^F] TCP 192.168.1.130:58220 -> 54.186.241.180:443 with a size of 52 for proc
ess /usr/lib/firefox/firefox
10/05/2016 08:29:07 | daemon | DEBUG | [rules_manager.cpp::RulesManager:30]: An allowing rule found for /usr/lib/firefox/firefox
10/05/2016 08:29:07 | daemon | DEBUG | [netlink_message_handler.cpp::NetlinkMessageHandler:7]: [?3^Bo-?_?^N^O
??b^_?^F] TCP 192.168.1.130:58220 -> 54.186.241.180:443 with a size of 52 for process /usr/lib/firefox/firefox
10/05/2016 08:29:07 | daemon | DEBUG | [rules_manager.cpp::RulesManager:30]: An allowing rule found for /usr/lib/firefox/firefox
10/05/2016 08:29:08 | daemon | DEBUG | [netlink_message_handler.cpp::NetlinkMessageHandler:7]: [!ESC?g????>?A?q?^]^^^F] TCP 192.168.1.130:54588 -> 54.149.8.1:443 with a size of 52 for proces
s /usr/lib/firefox/firefox
...
10/05/2016 08:29:12 | daemon | DEBUG | [rules_manager.cpp::RulesManager:30]: An allowing rule found for /usr/lib/firefox/firefox
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
After a certain point of handling the firefox rule it somehow becomes overloaded and starts dumping junk characters into the log file.
Thank you @bash64 for your messages (I had a smile :)).
In order to avoid crashing your system, and especially while testing special branches of the DKMS modules, you should use K-Dump (Here is a blog article for Ubuntu: http://blog.zedroot.org/debug-your-linux-kernel-module/). It will handle the kernel panic, boot a new kernel and move your session, saving your machine, and also saving a crash file (readable with the crash
command line tool) which would help us to debug.
Just to be sure, here are the instructions in order to test this branch:
The following steps are required in case you already have Douane installed
make cleandkms
as rootNow clone the @Guimli's repository and rebuild the kernel module:
~/Douane/
if you following the compilation instructions) open a terminalgit clone git@github.com:Guimli/douane-dkms.git douane-dkms-guimli
cd douane-dkms-guimli
make dkms
as rootIn the case you haven't followed those instructions, it means you've used the original code which is broken.
Let us know.
Ok, The "activity->devise_name" is not initilized with >= kernel 4.1.0 in douane-dkms. I will work on this problem. But for now, I do not know how to retrieve the name of the interface with the new implementation of nf_hook ...
The code for the blog article from Ubuntu does not compile. No way to make KEDR.
Oops... sorry @bash64, wrong blog article. Here is the right one: http://blog.zedroot.org/linux-kernel-debuging-using-kdump-and-crash/
kdump is installed. /proc/cmdline show proper crash info. forcing a crash , however, does not result in a reboot into the crash kernel. The system just sits there locked up. kdump=1 set in the config file. not sure what is wrong.
ok, a sudo reboot is captured by kdump. a sudo coldreboot is needed to bypass kdump. so it is installed and works on a reboot. it however fails to catch the write to sysrq. i am wondering if it will catch douane crashing for me?
You shouldn't run Douane until you got the hot reboot working.
It was working fine for me when I wrote that blog article and I was using Ubuntu. Today I'm trying to make it working with Debian (I have switched) but I got a similar issue than yours. We need to find a way to this hot reboot working before to continue with Douane.
I have reproduce the crash on Ubuntu 16.04. But I don't know what to do with this information :
crash 7.1.4 Copyright (C) 2002-2015 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.6 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu"...
KERNEL: /usr/lib/debug/boot/vmlinux-4.4.0-22-generic
DUMPFILE: /var/crash/201605121610/dump.201605121610 [PARTIAL DUMP]
CPUS: 4
DATE: Thu May 12 16:10:00 2016
UPTIME: 00:02:13
LOAD AVERAGE: 0.92, 0.38, 0.14 TASKS: 483 NODENAME: douane RELEASE: 4.4.0-22-generic VERSION: #39-Ubuntu SMP Thu May 5 16:53:32 UTC 2016 MACHINE: x86_64 (3399 Mhz) MEMORY: 2 GB PANIC: "BUG: unable to handle kernel NULL pointer dereference at 0000000000000017" PID: 0 COMMAND: "swapper/2" TASK: ffff88007c8e8c80 (1 of 4) [THREAD_INFO: ffff88007c8f4000] CPU: 2 STATE: TASK_RUNNING (PANIC)
crash> bt PID: 0 TASK: ffff88007c8e8c80 CPU: 2 COMMAND: "swapper/2"
[exception RIP: netfiler_packet_hook+381]
RIP: ffffffffc016498d RSP: ffff88007fd03620 RFLAGS: 00010286
RAX: ffffffffffffffff RBX: ffff88007b65f4f4 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 000000000000000c RDI: 0000000000000000
RBP: ffff88007fd037a0 R8: 000000000000ffff R9: ffff88007b65f508
R10: ffff88007b65f504 R11: ffffffff81cccb00 R12: ffff880076428000
R13: ffff88007b873200 R14: ffff88007b65f508 R15: ffff88007b873200
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
---
[exception RIP: unknown or invalid address]
RIP: 0000000000000000 RSP: 0000000000000000 RFLAGS: 00000000
RAX: ffffffff81f35140 RBX: ffff88007c8f4000 RCX: ffff88007b37d800
RDX: 00000000ffff5d08 RSI: 0000000000000000 RDI: ffff88007fd0dca0
RBP: 0000000000000087 R8: ffff88007c8f7e90 R9: 0000000000000002
R10: 0000000000000000 R11: 0000000000000000 R12: ffff88007fd0ed40
R13: 000000000000001c R14: 00000000ffff5d09 R15: 00000000ffff5d08
ORIG_RAX: 0000000000000000 CS: 0000 SS: ffffffffffffff5e
bt: WARNING: possibly bogus exception frame
crash>
Ok, I found the problem. skb->sk properties is not initialized and can cause Kernel Panic if state = TCP_TIME_WAITor TCP_NEW_SYN_RECV. @bash64 , can you test with my last commit ?
I want to try to get kdump working first. I will try kdump again.
On 05/15/2016 06:46 PM, Nicolas Weill wrote:
Ok, I found the problem. skb->sk properties is not initialized and can cause Kernel Panic if state = TCP_TIME_WAITor TCP_NEW_SYN_RECV. @bash64 https://github.com/bash64 , can you test with my last commit ?
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Douane/douane-dkms/pull/16#issuecomment-219315895
kdump seems to be working now. Let you all know tomorrow.
On 05/15/2016 06:46 PM, Nicolas Weill wrote:
Ok, I found the problem. skb->sk properties is not initialized and can cause Kernel Panic if state = TCP_TIME_WAITor TCP_NEW_SYN_RECV. @bash64 https://github.com/bash64 , can you test with my last commit ?
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Douane/douane-dkms/pull/16#issuecomment-219315895
Question. Nothing runs on boot. I launch the configurator to get the daemon to run. I have to manually launch the douane-dialog also.
On 05/15/2016 06:46 PM, Nicolas Weill wrote:
Ok, I found the problem. skb->sk properties is not initialized and can cause Kernel Panic if state = TCP_TIME_WAITor TCP_NEW_SYN_RECV. @bash64 https://github.com/bash64 , can you test with my last commit ?
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Douane/douane-dkms/pull/16#issuecomment-219315895
Marking time: It is 12:21pm EST. Waiting for a crash.
Indeed, everything has to be started manually. Once the main problems will be solved, I'll try to integrate the automatic start douane-daemon and douane-dialog
ok, I set douane-daemon to run out of rc.local. I set douane-dialog to run on login. So far so good. I have run a lot of apps and the rules are working. No crashing so far. Will post an update tonight.
kdump still not working when a crash happens or when triggered with sysreq
It would be nice to be able to block by appname, ip, or dns name, my choice. A lot of apps use python to get to the net. I cannot block python completely. By ip or dns name would be more appropriate.
Yes, that would be great ... But it takes a huge rewrite of code and many working day. it's the issues 33
12:58pm no crashing. Usually crashes within 5 minutes. I think you got it man.
1:42pm crash. Began playing a youtube video on facebook. Where can I find a log? Total lockup. KDump did not work. Hard force off.
Are you use my last commit ? If KDump work, the dump can be found in /var/crash/.
Yes, but i have update code yesterday. I will try to reproduce the crash.
Nothing in /var/crash. kdump setup properly. I downloaded all code fresh right before 12:21pm.
kdump-config show DUMP_MODE: kdump USE_KDUMP: 1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR: /var/crash crashkernel addr: 0x2a000000 current state: ready to kdump
kexec command: /sbin/kexec -p --command-line="BOOT_IMAGE=/vmlinuz-4.4.0-22-generic root=/dev/mapper/vg-root ro nouveau.blacklist=1 intel_pstate=disable transparent_hugepage=never quiet splash nomdmonddf nomdmonisw irqpoll maxcpus=1 nousb" --initrd=/boot/initrd.img-4.4.0-22-generic /boot/vmlinuz-4.4.0-22-generic
ok, this seems to be the answer why kdump fails for me. It has to do with /boot being on a separate partition (I also use LVM), found this on a website:
If /boot is on a separate partition, Kernel Crash Dump on Ubuntu will fail because of a ---> bug which still exists in 14.04 Trusty, can you believe it? At least it is the case for me using LVM and a separate /boot.
The workaround is to
unmount /boot and then mount it to some other mount point, e.g. /mnt
copy the contents over to /boot (the one on the same block device as /). e.g. rsync -axHAX --progress --stats /mnt/ /boot
Trigger a crash by using echo c | sudo tee /proc/sysrq-trigger
If all good, you'll see kernel crash dump in /var/crash in forms of (uname -r)-yyyymmddHHmm.crash and a yyyymmddHHmm directory with dmesg and dump.
If you want to analyze the crash dump, you'll need crash, run it like below: crash /usr/lib/debug/boot/vmlinux-$(uname -r) /var/crash/yyyymmddHHmm/dump.yyyymmddHHmm For more info regarding crash, read the manual. BTW: DO NOT forget to reload the kdump-tools configuration after changing /etc/default/kdump-tools by kdump-config load.
I copied my /boot folder and made a local copy in / for it. On boot I >umount /boot to unmount the boot partition. Even with /boot being in the / folder kdump still fails. I do: su - echo c > /proc/sysrq-trigger total lockup, no kdump. Hard to test Douane when kdump fails to make a report.
New version ! Rather than exclude statements do not return a socket, I filter only those who have one. This should be more stable. If it does not work, I could not know or just the problem without your Kdump. Douane does not crash on my test computers ...
@bash64 , thank you for your patience and your help.
ok, I can try again. I have no working crash dump. Linux Mint 18 will be out soon (ubuntu 16.04). Should fix the issues or at least let me run crash dump.
Using new code. Compiled at 1am may 22nd. Going let a video stream run all night in Firefox. That will usually lock it up if it is going to.
The culprit might be /usr/lib/firefox/plugin-container. It almost always happen when playing videos, but not always. Video streams generate a LOT of constant traffic.
2:21am crashed again. Wrecked my raid. Raid is rebuilding.
I'm sorry for not being able to find the cause of the crash, and worries that it generates on your computer each test. I will install Douane and Kdump on a maximum computers around me to try to reproduce this bug. Thank you again for your patience.
Will it help if I install it using vmware? Will kdump work with vmware?
Yes, Kdump work with VMware. But I can not crash my Linux Mint Vmware computer...
I run PGL (Peer Guardian for Linux). Could they be conflicting?
Original website: https://launchpad.net/~jre-phoenix/+archive/ubuntu/ppa
Open a terminal and copy and paste each line:
sudo apt-add-repository ppa:jre-phoenix/ppa sudo apt-get update sudo apt-get install pglcmd pgld pglgui
This pull request integrates changes in the netfilter hook function in relation to the version of the kernel. Tested successfully on Ubuntu 14.04 with Kernel 3.13.0-85, Ubuntu 15.10 with Kernel 4.2.0-35 and Ubuntu 16.04 with Kernel 4.4.0-22. This fix correct freeze problem for all kernel version.