Using syntastic for linux driver development

[tx6es]

Hi,

Hope I didn't open a thread for something that is still opened. I'm trying to get syntastic working on Linux kernel source code to get errors while developing a little driver. I know it's strange to use it for such development, but I just wanted to see if it could work. So far it works fine for my usual projects, but I can't manage to make it work when dealing with linux/includes files.

I tried to make my problem description as detailed as possible. (sorry for the big block).

The error message with g:systastic_c_compiler set to 'gcc' is:

Error detected while processing function 47_UpdateErrors..47_CacheErrors..14..SyntaxCheckers_c_gcc_GetLocList..SyntasticMake:
line   22:
E40can't open file /tmp/vZxyc/2

I use an automated include generation script that set the following variables: 'g:syntastic_c_compiler_options' with:

    -D__KERNEL__
    -DCONFIG_AS_CFI=1
    -DCONFIG_AS_CFI_SIGNAL_FRAME=1
    -DCONFIG_AS_CFI_SECTIONS=1
    -DCONFIG_AS_FXSAVEQ=1
    -nostdinc
    -DMODULE
    -DKBUILD_STR(s)=#s
    -DKBUILD_BASENAME=KBUILD_STR(main)
    -DKBUILD_MODNAME=KBUILD_STR(main)

and 'g:syntastic_c_include_dirs' list with the following content:

    -I/$HOME/path_to_my_project_sources/
    -I/$HOME/Downloads/linux-3.2.41/include/
    -I/$HOME/Downloads/linux-3.2.41/include/keys/
    -I/$HOME/Downloads/linux-3.2.41/include/misc/
    -I/$HOME/Downloads/linux-3.2.41/include/linux/
    -I/$HOME/Downloads/linux-3.2.41/include/linux/usb/
    -I/$HOME/Downloads/linux-3.2.41/include/linux/input/
    ... and some others I need

With $HOME expanded to my true home directory.

I also tried to put these values into the 'g:syntastic_cpp_compiler_options' and 'b:syntastic_c_cflags' variables, but it still doesn't work

When setting the variable 'b:syntastic_ccflags' with all the -I/* options and without the -D* ones, it puts me directly into the linux/init.h header on BufWritePost event for my driver file (with the autojump option set to 1). The loc list is populated with a lots of error and warning on all the linux/.h that I include in my driver code. So Syntastic does find the header but the compiler misses the macros definitions to handle it correctly. But putting the -D_ option into that variable or into any other one raises the initial error E40 on /tmp/...

When not touching to the b:syntastic_c_cflags option and putting everything on the g:syntastic_c_compiler_options, syntastic just raise one error in the location list saying it cannot find the linux/init.h file. But it is pointed by one of the include directives.

I've also been reading this:

https://github.com/scrooloose/syntastic/pull/241
https://github.com/scrooloose/syntastic/pull/347

so I put the following into my vimrc:

    let g:syntastic_c_checker_checkpatch_location = '/usr/bin/checkpatch.pl'

with execution rights and : (tested both)

    let g:syntastic_c_checkers = ['checkpatch', 'gcc']
    or
    let g:syntastic_c_checkers = ['checkpatch']

I do have checkpatch.pl in my $PATH and vim compiled with perl support. But it still doesn't work.

I tested splint and sparse checkers without any success. Both are saying they cannot find the linux/*.h files.

I'm almost certain to miss something but I can't see what.

Vim version: 7.3.950 Syntastic version: commit 68cfe6513ba9432d69d534a90bd2761f80b40ce8

Thanks for your help.

[lcd047]

1. E40 is a known problem, see #234. There is no obvious cause for it in syntastic, and at this point I strongly suspect it to be caused by a race condition in Vim. Upgrading to the latest Vim from Mercurial might help (I haven't triggered the bug in a while with recent Vims).
2. With the gcc checker, you can put all those options in an option file. The wiki article on the gcc checker explains all the available knobs.
3. g:syntastic_c_include_dirs is supposed to look like this: ['includes', 'headers', ...]. You probably want to add the -I forms to g:syntastic_c_compiler_options (or to the config file mentioned above) instead.
4. g:syntastic_cpp_compiler_options only applies to C++ files.
5. g:syntastic_c_compiler_options are global options and b:syntastic_c_cflags apply only to the current buffer, but they are otherwise identical. Yes, there are way too many ways to do the same thing.
6. Syntastic doesn't cope well with errors in the included files. You can either set g:syntastic_c_remove_include_errors to 1 to omit these errors, or you can try the buffers branch, which is supposed to address that.
7. You don't need to mess with g:syntastic_c_checker_checkpatch_location.
8. Please define what you mean by "checkpatch.pl doesn't work".
9. For making splint and sparse work, please read the corresponding wiki pages. They describe the relevant options.

[tx6es]

Ok, thanks for the help.

I didn't specified it, but I did use the list for g:syntastic_c_include_dirs without the '-I'. It is like you said. But in that case, with all the -D* options set into the g:syntastic_c_compiler_options, I get the E40 error.

I also tried to but everything into the g:syntastic_c_compiler_options keeping the g:syntastic_c_include_dirs like you said (even if it doesn't seem to be useful), but it keeps raising the E40 error.

I tested it with vim's version 7.3.1136 (the last one) but I still get the E40. So I guess there's nothing to do but wait for race condition you suspect to be corrected in vim.

Concerning checkpatch.pl, It doesn't work because it doesn't say anything, even if I do include an obvious mistake in my code.

Unless you see an error with this configuration settings, I'll close the issue and wait to see if updating vim does solve the E40. In such case, I'll notify it.

Thanks.

[lcd047]

Most of the C and C++ checkers in syntastic accept a config file, and they all use the same function for loading it. I suggest the following setup:

let g:syntastic_c_config_file = '.config'
let g:syntastic_cpp_config_file = '.config'
let g:syntastic_oclint_config_file = '.config'
let g:syntastic_sparse_config_file = '.config'
let g:syntastic_splint_config_file = '.config'
let g:syntastic_c_sparse_post_args = '-gcc-base-dir ' .
        \ system("gcc -v 2>&1 | sed -n '/^Reading specs/ { s#.* /#/#; s#/[^/]*$##; p; }'")

Then you can put all the defines and include junk in a file .config in the top directory of your project, and it will be used by all checkers.

Back to E40: if you can consistently reproduce it, that's an opportunity to run a few experiments. Please update syntastic to the master HEAD, then set:

let &makeef = '/tmp/syntastic_' . getpid() . '_##'

and run the checker. Does that change anything? Does anything change if you put the defines etc. in a config file? What if you remove some of the defines, but not all? Is the size of the config file close to the line length limit in your shell (10KB in bash)?

As for checkpatch.pl, check with :SyntasticInfo that it's actually active. If it is, does it produce the expected results if you run it on the command line like this:

checkpatch.pl --no-summary --no-tree --terse --file file.c

If it does, but it doesn't work from syntastic, then please open a separate bug report.

[tx6es]

Ok.

Updated to the master HEAD. checkpatch.pl is working fine, I made a mistake when I set it, sorry about that.

But the error is still the same with other checkers:

I set the following:

let &makeef = '/tmp/syntastic_' . getpid() . '_##'

It doesn't change anything, sill getting the E40:

Error detected while processing function 53_UpdateErrors..53_CacheErrors..21..SyntaxCheckers_c_gcc_GetLocList..SyntasticMake:
line   26:
E40: Can't open errorfile /tmp/syntastic_8699_86990

Concerning the config file, I just have the perfect one for this purpose: the .clang_complete file! It does contains everything needed. It's 5.7 kb so no risk to get close to the line lenght, but it does have every option declared in a single line. I set the option like this:

    let g:syntastic_c_config_file = '.clang_complete'

I was wondering if putting options into one or multiple lines in the config file did matter. But I don't think it does since you must be talking about the length of the command line formatted by your plugin: (please confirm that)

I tried both nevertheless and got 2 different situations:

1. When putting everything on the same line in the file, it does raise this error:

   driver.c |8 col 24 error| fatal error: linux/init.h: No such file or directory
   

   so it's working but it doesn't use the includes. (I don't see any mistake in my single line config file but I may have made one considering what follows).

2. When I use it like I do with clang_complete, with each option on a single line: It raises the E40 error but: removing all the -D* makes it work, and it breaks into the kernel includes source code (which is normal, it cannot understand it without the macro definitions). So the macro definitions must be the problem. and after some tests, here are the guilty ones:

   -DKBUILD_STR(s)=#s
   -DKBUILD_BASENAME=KBUILD_STR(main)
   -DKBUILD_MODNAME=KBUILD_STR(main)
   

   Removing these 3 ones still makes syntastic breaking into one of the kernel header because the checker doesn't understand something (because those are missing).

Keeping the macro containing the '#' raises the following error:

Error detected while processing function 53_UpdateErrors..53_CacheErrors..21..SyntaxCheckers_c_gcc_GetLocList..SyntasticMake:
line   26:
E194: No alternate file name to substitute for '#': gcc -x c -fsyntax-only -std=gnu99 'driver.c'  ... rest of the command line with all -I -D*

Removing the one with the # makes me think that Bash seems to be part of the problem: It raises two errors: the following one and the E40 (just after).

"driver.c" 426L, 10749C written/bin/bash: -c: line 0: syntax error near unexpected token `('

Considering what I tested, I strongly suspect the bash error is the cause of the E40 one in this context. But I need to solve it to see what it does.

So I tried escaping the '(', ')' and '#' but it did not change anything for both errors. Escaping the character may not be a good solution but I had no time to test something else.

What do you think?

[lcd047]

This is very useful, thank you!

The options in the config file should be passed one per line. If you put everything on a single line you break handling of includes, essentially because the parser tries to be too smart about relative paths. :)

Yeah, the parser doesn't do proper escaping ('#' is a problem too because it has a special meaning to Vim). I'll think of a solution and (hopefully) post a fix tomorrow. Sadly, even if this solves the E40 for C, the problem also happens in cases that don't involve that parser.

[lcd047]

Ok, please try the gcc_refactor branch. This hopefully does proper escaping of shell and Vim special characters in config files, and in g:syntastic_c_include_dirs lists. Doing escaping in the rest of the parameters is too hard (mainly because they would first need to be split into separate options, and handling things like \\ and \[space] would be a pain), so the joy of doing that is left to the user. :) But config files should be safe, specifically including your defines above. I'll add notes to the wiki if this turns out to actually work.

@kongo2002: I would really appreciate your ok with all this.

[tx6es]

Ok,

Switched to gcc_refactor. When using the configuration file it doesn't raise any errors, but it is still breaking in a kernel header.

So I tried to set the 'g:syntastic_c_compiler_options' with the macro definitions manually: It raises the following:

Error detected while processing function 53_UpdateErrors..53_CacheErrors..21..SyntaxCheckers_c_gcc_GetLocList..SyntasticMake:
line   26:
E194: No alternate file name to substitute for '#': gcc -x c -fsyntax-only -std=gnu99 'driver.c'  ... rest of the command line with all -I -D*

So I tried escaping the '(', ')' and '#' characters. The previous error is solved but the E40 is here. It does not display a bash error this time but the bash error is still triggered since I get in my shell after closing vim. I also tried to do this:

let &makeef = '/tmp/syntastic_' . getpid() . '_##'

But it doesn't change anything.

The E40 is the only error I get.

Hope it helps.

[lcd047]

In order to escape options in g:syntastic_c_compiler_options you need the equivalent of shellescape(fnameescape(option)) for each option. It's probably much easier to just go with the config file.

What breaks with the config file? Did you put each option on a single line?

[tx6es]

Yes, each option is on a single line with the config file. So \n breaks When using it I don't get any errors (neither E40 nor bash). In fact, it just takes the .clang_complete file. But the checker still does not understand the includes. What's strange is that clang_complete works fine with these flags, so those must be correct. Are there some default flags that are added by syntastic?

[lcd047]

The only flags used for gcc are -x c -fsyntax-only -std=gnu99.

Please set g:syntastic_debug to 1, run the checker with the config file, and look at :mes.

[tx6es]

:mes says nothing but this (with g:syntastic_debug set to 1)

Messages maintainer: Bram Moolenaar 
"driver.c" 426L, 10745C

So I think everything works with syntastic. In fact, the location list error I get is somewhat different than the others I saw until now. I may have an idea about where to look at. It's possible that in this case clang and gcc are not behaving the same way...

[lcd047]

Sorry, this doesn't make any sense. You should see debugging messages after you run the checker with g:syntastic_debug set to 1. If you don't, the checker didn't run at all.

[tx6es]

Ok, my mistake, I need to explicitly run SyntasticCheck to get something into mes. Just Saving the file did not fill it. Here it is:

Messages maintainer: Bram Moolenaar 
"driver.c" 426L, 10745C
syntastic: debug: CacheErrors: Invoking checker: gcc
syntastic: debug: SyntasticMake: called with options: {'errorformat': '%-G%f:%s:,%-G%f:%l: %#error: %#(Each undeclared identifier is reported only%.%#,%-G%f:%l: %#error: %#for each function it appears%.%#,%-GIn f
ile included%.%#,%-G %#from %f:%l\,,%f:%l:%c: %trror: %m,%f:%l:%c: %tarning: %m,%f:%l:%c: %m,%f:%l: %trror: %m,%f:%l: %tarning: %m,%f:%l: %m', 'makeprg': 'gcc -x c -fsyntax-only -std=gnu99 ''-I../includes'' ''-I.
./include'' ''-I..'' ''-Iinclude'' ''-I.'' ''-Iincludes'' ''driver.c''  ''-I/$HOME/dls/driver_source_dir/.'' ''-I/usr/include/linux/'' ''-D__KE
RNEL__'' ''-DCONFIG_AS_CFI=1'' ''-DCONFIG_AS_CFI_SIGNAL_FRAME=1'' ''-DCONFIG_AS_CFI_SECTIONS=1'' ''-DCONFIG_AS_FXSAVEQ=1'' ''-nostdinc'' ''-DMODULE'' ''-DKBUILD_STR(s)=\#s'' ''-DKBUILD_BASENAME=KBUILD_STR(main)''
 ''-DKBUILD_MODNAME=KBUILD_STR(main)'' ''-I/usr/src/linux-source-3.2/include/'' ''-I/usr/src/linux-source-3.2/include/linux/'' ''-I/$HOME/Downloads/linux-3.2.41/arch/x86/include/'' ''-I/usr/lib/gcc/x86_64-li
nux-gnu/4.7/include/'' ''-I/usr/src/linux-headers-3.2.0-4-common/arch/x86/include/'' ''-I/usr/src/linux-headers-3.2.0-4-common/include/asm-generic/'' ''-I/usr/src/linux-headers-3.2.0-4-common/include/linux/'' ''-
I/usr/src/linux-headers-3.2.0-4-common/include/'' ''-I/$HOME/dls/driver_source_dir/'' ''-I/$HOME/Downloads/linux-3.2.41/include/'' "and all remaining -I* in double quotes ''}
Pattern not found: error:

But the problem is also that gcc needs extra include, so right now I don't think it's a syntastic problem anymore.

[lcd047]

You need to either expand $HOME, or use relative paths. Environment variables are not expanded in the config file (again, doing that in Vim would be too hard, sorry)...

[tx6es]

No, it's like in my first message, my home path is quite long in fact, I just run sed over it to avoid a too long message.

So unless you see an error I think we can close this thread.

Thanks for your help.

[lcd047]

Well, Pattern not found: error: is abnormal, but otherwise the message looks correct.

[tx6es]

I think I'm responsible for this one. I did a little modification in the plugin triggering sequence (for my needs), using it the normal way removes it.

Thanks for your help!