Closed Olivia5k closed 5 years ago
Doing some cursory investigation, and it seems that the autogeneration detection scans for certain comments that the Ragel generator does not provide. Might be that the problem is with us then. But, it feels weird that the linters try to parse the .rl
files still.
level=warning msg="[runner] Can't run linter govet: can't eval symlinks for path $GO_REPO/ingestion/syslog/syslog/syslog_parser.rl: lstat $GO_REPO/ingestion/syslog/syslog: no such file or directory"
level=warning msg="[loader/astcache] Can't parse AST of $GO_REPO/scan/NONE: open $GO_REPO/scan/NONE: no such file or directory"
level=warning msg="[runner] Can't process result by autogenerated_exclude processor: can't filter issue result.Issue{FromLinter:\"ineffassign\", Text:\"ineffectual assignment to `p`\", Pos:token.Position{Filename:\"$GO_REPO/scan/NONE\", Offset:117822, Line:3, Column:0}, LineRange:(*result.Range)(nil), HunkPos:0, SourceLines:[]string(nil)}: can't parse file $GO_REPO/scan/NONE: open $GO_REPO/scan/NONE: no such file or directory"
level=warning msg="[runner] Can't process result by nolint processor: can't filter issue result.Issue{FromLinter:\"ineffassign\", Text:\"ineffectual assignment to `p`\", Pos:token.Position{Filename:\"$GO_REPO/scan/NONE\", Offset:117822, Line:3, Column:0}, LineRange:(*result.Range)(nil), HunkPos:0, SourceLines:[]string(nil)}: can't parse file $GO_REPO/scan/NONE: open $GO_REPO/scan/NONE: no such file or directory"
level=warning msg="[runner/source_code] Failed to get lines for file $GO_REPO/scan/NONE: can't read file $GO_REPO/scan/NONE for printing issued line: open $GO_REPO/scan/NONE: no such file or directory"
level=warning msg="[runner/source_code] Failed to get lines for file $GO_REPO/scan/NONE: can't read file $GO_REPO/scan/NONE for printing issued line: open $GO_REPO/scan/NONE: no such file or directory"
$GO_REPO/scan/NONE:3: ineffectual assignment to `p` (ineffassign)
$GO_REPO/scan/NONE:43: ineffectual assignment to `p` (ineffassign)
I added // autogenerated file
comments to all the .rl
and _gen.go
code, just to see if it worked. Get the above as output. Not sure how to proceed with debugging. Happy to test things if you have suggestions!
Notable;
syslog/syslog
does not exist (only the single syslog
), and there are no symlinks at all in the repository.
Upon further investigation, it seems that the error boils down to ineffassign only. Disabling it made the problem go away.
Perhaps this should be an upstream bug to them?
hi, thank you for such an interesting issue and it's investigation!
the reason is that ragel-generated files contain directives like //line filename.rl:1
: they are standard and tell the parser that go file's name isn't filename.go
but filename.rl
.
I will try to fix it.
Hi! I've made some fixes for such issues. Please, check it in the latest release
That's awesome! Unfortunately I have left the company in which these problems were present, so I will not be able to verify that it fixed the original case that I reported.
Since I have no ways of checking if the provided fix solves the issue, you can feel free to close it as fixed.
ok, sorry for delay =(
At my company we use Ragel to define a state machine and parse structured data sets.
The interesting parts of the logs are the warnings that are printed just before the results start popping up;
Several errors happen here - possibly all related to the same issue:
.rl
files but fails (it's not Go code)NONE
for some of the errorsI tried adding
skip-files
, but it doesn't seem to listen about".*\\.rl$"
for some reason. Instead, an even weirder error about symbolink links to the Ragel files pop up (there are no symbolic links in our repo, so that one is Twilight Zone levels of weird to me).Also, since the config file suggests to let you know if the autogeneration detection doesn't work, consider this a notice that
_gen.go
does not seem to be recognized. :slightly_smiling_face:Anyways, it seems to me that all of these issues are related to the linters thinking they should work on the Ragel files and their subsequent autogenerated code. I would suggest that this is wrong and that they should be skipped.
Version of golangci-lint:
golangci-lint --version
(or git commit if you don't use binary distribution) cb5d1da98628504e795c2599f8994462c19683c3 (one commit after1.12.2
)Config file:
cat .golangci.yml
options for analysis running
run:
default concurrency is a available CPU number
concurrency: 4
timeout for analysis, e.g. 30s, 5m, default is 1m
NOTE(thiderman): The CI doesn't have time to complete the
compilation in just one minute, so setting this to 10 instead.
deadline: 10m
exit code when at least one issue was found, default is 1
issues-exit-code: 1
include test files or not, default is true
tests: true
list of build tags, all linters use it. Default is empty list.
build-tags:
mytag
which dirs to skip: they won't be analyzed;
can use regexp here: generated.*, regexp is applied on full path;
default value is empty list, but next dirs are always skipped independently
from this option's value:
vendor$, third_party$, testdata$, examples$, Godeps$, builtin$
skip-dirs:
".ingestion/syslog/."
which files to skip: they will be analyzed, but issues from them
won't be reported. Default value is empty list, but there is
no need to include all autogenerated files, we confidently recognize
autogenerated files. If it's not please let us know.
skip-files:
- ".*\.pb\.go$"
- ".*_gen\.go$"
- ".*\.rl$"
output configuration options
output:
colored-line-number|line-number|json|tab|checkstyle, default is "colored-line-number"
format: colored-line-number
print lines of code with issue, default is true
print-issued-lines: true
print linter name in the end of issue text, default is true
print-linter-name: true
all available settings of specific linters
linters-settings: errcheck:
report about not checking of errors in type assetions:
a := b.(MyStruct)
;govet:
report about shadowed variables
golint:
minimal confidence for issues, default is 0.8
gofmt:
simplify code: gofmt with
-s
option, true by defaultgoimports:
put imports beginning with prefix after 3rd-party packages;
gocyclo:
minimal code complexity to report, 30 by default (but we recommend 10-20)
maligned:
print struct with more effective memory layout or not, false by default
dupl:
tokens count to trigger issue, 150 by default
goconst:
minimal length of string constant, 3 by default
depguard: list-type: blacklist include-go-root: false packages:
github.com/davecgh/go-spew/spew misspell:
Correct spellings using locale preferences for US or UK.
Default is to use a neutral variety of English.
Setting locale to US will correct the British spelling of 'colour' to 'color'.
locale: US lll:
max line length, lines longer will be reported. Default is 120.
'\t' is counted as 1 character by default, and can be changed with the tab-width option
line-length: 120
tab width in spaces. Default to 1.
tab-width: 1 unused:
treat code as a program (not a library) and report unused exported identifiers; default is false.
XXX: if you enable this setting, unused will report a lot of false-positives in text editors:
if it's called for subdir of a project it can't find funcs usages. All text editor integrations
with golangci-lint call it on a directory with the changed file.
check-exported: false unparam:
call graph construction algorithm (cha, rta). In general, use cha for libraries,
and rta for programs with main packages. Default is cha.
algo: cha
Inspect exported functions, default is false. Set to true if no external program/library imports your code.
XXX: if you enable this setting, unparam will report a lot of false-positives in text editors:
if it's called for subdir of a project it can't find external interfaces. All text editor integrations
with golangci-lint call it on a directory with the changed file.
check-exported: false nakedret:
make an issue if func has more lines of code than this setting and it has naked returns; default is 30
max-func-lines: 30 prealloc:
XXX: we don't recommend using this linter before doing performance profiling.
For most programs usage of prealloc will be a premature optimization.
Report preallocation suggestions only on simple loops that have no returns/breaks/continues/gotos in them.
True by default.
simple: true range-loops: true # Report preallocation suggestions on range loops, true by default for-loops: false # Report preallocation suggestions on for loops, false by default gocritic:
which checks should be enabled; can't be combined with 'disabled-checks';
default are: [appendAssign assignOp caseOrder dupArg dupBranchBody dupCase flagDeref
ifElseChain regexpMust singleCaseSwitch sloppyLen switchTrue typeSwitchVar underef
unlambda unslice rangeValCopy defaultCaseOrder];
all checks list: https://github.com/go-critic/checkers
enabled-checks:
which checks should be disabled; can't be combined with 'enabled-checks'; default is empty
disabled-checks:
- regexpMust
settings: # settings passed to gocritic
captLocal: # must be valid enabled check name
checkLocals: true
rangeValCopy: sizeThreshold: 32
linters: enable:
go version go1.11.2 linux/amd64 GOARCH="amd64" GOBIN="" GOCACHE="/home/thiderman/.cache/go-build" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOOS="linux" GOPATH="/home/thiderman" GOPROXY="" GORACE="" GOROOT="/usr/lib/go" GOTMPDIR="" GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64" GCCGO="gccgo" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build757637135=/tmp/go-build -gno-record-gcc-switches"
golangci-lint -v run --config build/lint/golangci.yml $GO_REPO/... level=info msg="[config_reader] Used config file build/lint/golangci.yml" level=info msg="Gocritic enabled checks: [rangeValCopy]" level=info msg="[lintersdb] Active 8 linters: [deadcode errcheck govet ineffassign megacheck structcheck typecheck varcheck]" level=info msg="[loader] Go packages loading at mode load deps types and syntax took 5.258998345s" level=info msg="[loader] SSA repr building timing: packages building 34.781778ms, total 4.034596775s" level=info msg="[loader] SSA for megacheck repr building timing: packages building 35.111017ms, total 3.815851712s" level=info msg="[runner] worker.3 took 496.813832ms with stages: varcheck: 232.378871ms, errcheck: 107.600384ms, deadcode: 89.396959ms, structcheck: 67.351445ms, typecheck: 5.682µs" level=info msg="[runner] worker.4 took 526.453441ms with stages: govet: 526.42779ms" level=warning msg="[runner] Can't run linter govet: can't eval symlinks for path $GO_REPO/ingestion/syslog/syslog/syslog_parser.rl: lstat $GO_REPO/ingestion/syslog/syslog: no such file or directory" level=info msg="[runner] worker.2 took 719.760613ms with stages: ineffassign: 719.737115ms" [... AST cache removed ...] level=warning msg="[loader/astcache] Can't parse AST of $GO_REPO/scan/machines/kv.rl: $GO_REPO/scan/machines/kv.rl:1:1: expected 'package', found '%' (and 11 more errors)" level=warning msg="[runner] Can't process result by autogenerated_exclude processor: can't filter issue result.Issue{FromLinter:\"ineffassign\", Text:\"ineffectual assignment to
p
\", Pos:token.Position{Filename:\"company/scan/machines/kv.rl\", Offset:117535, Line:80, Column:0}, LineRange:(result.Range)(nil), HunkPos:0, SourceLines:[]string(nil)}: can't parse file company/scan/machines/kv.rl: $GO_REPO/scan/machines/kv.rl:1:1: expected 'package', found '%' (and 11 more errors)" level=warning msg="[runner] Can't process result by nolint processor: can't filter issue result.Issue{FromLinter:\"ineffassign\", Text:\"ineffectual assignment top
\", Pos:token.Position{Filename:\"company/scan/machines/kv.rl\", Offset:117535, Line:80, Column:0}, LineRange:(result.Range)(nil), HunkPos:0, SourceLines:[]string(nil)}: can't parse file company/scan/machines/kv.rl: $GO_REPO/scan/machines/kv.rl:1:1: expected 'package', found '%' (and 11 more errors)" level=warning msg="[runner/source_code] Failed to get lines for file $GO_REPO/scan/NONE: can't read file $GO_REPO/scan/NONE for printing issued line: open $GO_REPO/scan/NONE: no such file or directory" level=warning msg="[runner/source_code] Failed to get lines for file $GO_REPO/scan/NONE: can't read file $GO_REPO/scan/NONE for printing issued line: open $GO_REPO/scan/NONE: no such file or directory" company/scan/fieldscanner.rl:113: ineffectual assignment tocs
(ineffassign) any; company/scan/fieldscanner_gen.go:2403: ineffectual assignment tocs
(ineffassign) cs = 0 company/scan/fieldscanner.rl:37: ineffectual assignment tocs
(ineffassign) action start_object { fcall object_inner; } company/scan/machines/kv.rl:80: ineffectual assignment top
(ineffassign) kv_sep any => { $GO_REPO/scan/NONE:3: ineffectual assignment top
(ineffassign) $GO_REPO/scan/NONE:43: ineffectual assignment top
(ineffassign) company/scan/tokenscanner.rl:96: ineffectual assignment totyp
(ineffassign) typ := TokenTypeLiteral company/scan/tokenscanner_gen.go:664: ineffectual assignment to_widec
(ineffassign) _widec = int16(data[p]) company/scan/tokenscanner_gen.go:709: ineffectual assignment to_widec
(ineffassign) _widec = int16(data[p]) level=info msg="[runner] worker.1 took 9.990441773s with stages: megacheck: 9.990423498s" level=info msg="[runner] Workers idle times: #2: 9.260875343s, #3: 9.478491445s, #4: 9.458929448s" [... AST parse removed ...] level=info msg="[runner/skip dirs] Skipped dirs: [migrations/3_popo/anomaly migrations/3_popo/situation migrations/2_blixem/after/sequence/api migrations/2_blixem/after/relog migrations/2_blixem/after/sequence/pb migrations/3_popo/framework ingestion/syslog/syslog migrations/1_highlander/migration migrations/3_popo migrations/2_blixem/after/events/pb]" level=info msg="[runner/max_same_issues] 3/6 issues with text \"ineffectual assignment tocs
\" were hidden, use --max-same-issues" level=info msg="[runner/max_same_issues] 3/6 issues with text \"ineffectual assignment top
\" were hidden, use --max-same-issues" level=info msg="[runner/max_same_issues] 1/4 issues with text \"this value ofp
is never used\" were hidden, use --max-same-issues" company/scan/structurescanner_gen.go:7773:1: this value ofp
is never used (megacheck) p = (te) - 1 ^ company/scan/structurescanner_gen.go:7787:3: this value ofp
is never used (megacheck) {p = (te) - 1 ^ company/scan/structurescanner_gen.go:7827:3: this value ofp
is never used (megacheck) {p = (te) - 1 ^ company/ingestion/syslog/parser_gen.go:21:7: constsyslog_first_final
is unused (megacheck) const syslog_first_final int = 42 ^ company/ingestion/syslog/parser_gen.go:22:7: constsyslog_error
is unused (megacheck) const syslog_error int = 0 ^ company/ingestion/syslog/parser_gen.go:24:7: constsyslog_en_main
is unused (megacheck) const syslog_en_main int = 1 ^ level=info msg="Memory: 145 samples, avg is 2149.8MB, max is 3780.7MB" level=info msg="Execution took 23.17017087s" make: *** [Makefile:78: lint] Error 1