search

semgrep / semgrep

Lightweight static analysis for many languages. Find bug variants with patterns that look like source code.
https://semgrep.dev
GNU Lesser General Public License v2.1
10.61k stars 622 forks source link

Semgrep crashes with fatal error on analysing a Go project #5167

Closed leandrosansilva closed 1 year ago

leandrosansilva commented 2 years ago

Describe the bug I have semgrep in a gitlab pipeline and semgrep crashes, printing a backtrace.

GitLab Semgrep analyzer v2.21.0, on gitlab.com (as on May 8th 2022).

To Reproduce Run semgrep via gitlab on the public repository https://gitlab.com/lightmeter/controlcenter. Please run the gitlab pipeline in the branch pinned_branch/reproduce_semgrep_gitlab_crashes (d86100d0183525405d944f53177bb0e3fddf36c4)

This is the job output:

Running with gitlab-runner 14.8.2 (c6e7e194)
  on gitlab-11-huge-linode ekfsM7-C
Resolving secrets
Preparing the "docker" executor
Using Docker executor with image registry.gitlab.com/security-products/semgrep:2 ...
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image registry.gitlab.com/security-products/semgrep:2 ...
Using docker image sha256:d667c8e9965d6e5954c82aa06885b8b97ff00bdea1bbc6db55316b502b480dee for registry.gitlab.com/security-products/semgrep:2 with digest registry.gitlab.com/security-products/semgrep@sha256:e3354ee76a13cfc954e90a8339bfddcb5ba2fbb78450f82435dcf78b2fa463bb ...
Preparing environment
Running on runner-ekfsm7-c-project-17017123-concurrent-34 via localhost...
Getting source from Git repository
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/lightmeter/controlcenter/.git/
Checking out d86100d0 as feature/682_replies_graphs...
Removing api/docs/docs.go
Removing api/docs/swagger.json
Removing api/docs/swagger.yaml
Removing dashboard/mock/dashboard_mock.go
Removing detective/mock/detective_mock.go
Removing domainmapping/generated_list.go
Removing frontend/controlcenter/node_modules/
Removing frontend/controlcenter/src/translation/
Removing insights/core/mock/fetcher_mock.go
Removing insights/core/mock/progress_mock.go
Removing intel/receptor/receptor_mock.go
Removing lightmeter
Removing po/generated.go
Removing recommendation/generated_link_list.go
Removing staticdata/http_vfsdata.go
Removing util/timeutil/timeutil_mock.go
Removing www/

Skipping Git submodules setup
Executing "step_script" stage of the job script
Using docker image sha256:d667c8e9965d6e5954c82aa06885b8b97ff00bdea1bbc6db55316b502b480dee for registry.gitlab.com/security-products/semgrep:2 with digest registry.gitlab.com/security-products/semgrep@sha256:e3354ee76a13cfc954e90a8339bfddcb5ba2fbb78450f82435dcf78b2fa463bb ...
$ /analyzer run
[INFO] [Semgrep] [2022-05-08T08:58:59Z] ▶ GitLab Semgrep analyzer v2.21.0
[INFO] [Semgrep] [2022-05-08T08:58:59Z] ▶ Detecting project
[INFO] [Semgrep] [2022-05-08T08:58:59Z] ▶ Found relevant files in project, analyzing entire repository
[INFO] [Semgrep] [2022-05-08T08:58:59Z] ▶ Running analyzer
[INFO] [Semgrep] [2022-05-08T08:59:30Z] ▶ Creating report
[FATA] [Semgrep] [2022-05-08T08:59:30Z] ▶ tool notification error: Fatal error [ERROR] Semgrep Core — Fatal error
An error occurred while invoking the Semgrep engine. Please help us fix this by creating an issue at https://github.com/returntocorp/semgrep

At line /builds/lightmeter/controlcenter/deliverydb/delivery.go:1: (Failure "Found unexpected raw string literal without delimiters")

====[ BEGIN error trace ]====
Raised at Stdlib.failwith in file "stdlib.ml", line 29, characters 17-33
Called from Parse_go_tree_sitter.raw_string_literal in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 69, characters 4-69
Called from Parse_go_tree_sitter.expression in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 694, characters 46-68
Called from Parse_go_tree_sitter.element in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 1110, characters 23-41
Called from Parse_go_tree_sitter.anon_choice_elem_c42cd9b in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 1017, characters 18-32
Called from Stdlib__list.map in file "list.ml", line 92, characters 20-23
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Parse_go_tree_sitter.literal_value in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 1263, characters 10-188
Called from Parse_go_tree_sitter.expression in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 682, characters 15-35
Called from Parse_go_tree_sitter.expression_list in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 586, characters 11-28
Called from Parse_go_tree_sitter.var_spec in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 1138, characters 15-37
Called from Parse_go_tree_sitter.statement in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 771, characters 25-44
Called from Parse_go_tree_sitter.source_file.(fun) in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 1354, characters 19-35
Called from Stdlib__list.map in file "list.ml", line 92, characters 20-23
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Stdlib__list.map in file "list.ml", line 92, characters 32-39
Called from Parse_go_tree_sitter.source_file in file "src/parsing/tree_sitter/Parse_go_tree_sitter.ml", line 1350, characters 2-369
Called from Parse_tree_sitter_helpers.wrap_parser in file "src/parsing/tree_sitter/Parse_tree_sitter_helpers.ml", line 157, characters 13-29
Called from Parse_target.run_parser in file "src/parsing/Parse_target.ml", line 100, characters 18-24
Re-raised at Parse_target.run in file "src/parsing/Parse_target.ml", line 193, characters 26-65
Called from Parse_target.parse_and_resolve_name_use_pfff_or_treesitter in file "src/parsing/Parse_target.ml", line 364, characters 30-60
Called from Run_semgrep.xtarget_of_file in file "src/runner/Run_semgrep.ml", line 384, characters 13-96
Called from CamlinternalLazy.force_lazy_block in file "camlinternalLazy.ml", line 31, characters 17-27
Re-raised at CamlinternalLazy.force_lazy_block in file "camlinternalLazy.ml", line 36, characters 4-11
Called from Common.with_time in file "src/pfff/commons/Common.ml", line 215, characters 12-16
Called from Match_search_rules.matches_of_patterns in file "src/engine/Match_search_rules.ml", line 264, characters 8-67
Called from Match_search_rules.matches_of_xpatterns in file "src/engine/Match_search_rules.ml", line 602, characters 6-55
Called from Match_search_rules.matches_of_formula in file "src/engine/Match_search_rules.ml", line 1037, characters 4-55
Called from Match_search_rules.check_rule in file "src/engine/Match_search_rules.ml", line 1068, characters 4-62
Called from Common.set_timeout in file "src/pfff/commons/Common.ml", line 1429, characters 12-16
Re-raised at Common.set_timeout in file "src/pfff/commons/Common.ml", line 1446, characters 6-13
Called from Match_rules.timeout_function in file "src/engine/Match_rules.ml", line 48, characters 4-73
Called from Match_rules.check.(fun) in file "src/engine/Match_rules.ml", line 105, characters 19-792
Called from Common.partition_either.part_either in file "src/pfff/commons/Common.ml", line 964, characters 15-18
Called from Match_rules.check in file "src/engine/Match_rules.ml", line 85, characters 4-1023
Called from Run_semgrep.semgrep_with_rules.(fun) in file "src/runner/Run_semgrep.ml", line 487, characters 13-265
Called from Run_semgrep.iter_targets_and_get_matches_and_exn_to_errors.(fun) in file "src/runner/Run_semgrep.ml", line 305, characters 21-29
Called from Stdlib__fun.protect in file "fun.ml", line 33, characters 8-15
Re-raised at Stdlib__fun.protect in file "fun.ml", line 38, characters 6-52
Called from Memory_limit.run_with_memory_limit in file "src/system/Memory_limit.ml", line 77, characters 6-62
Called from Run_semgrep.iter_targets_and_get_matches_and_exn_to_errors.(fun) in file "src/runner/Run_semgrep.ml", line 296, characters 17-1023
=====[ END error trace ]=====

Expected behavior Semgrep should not crash.

What is the priority of the bug to you?

Environment Gitlab runner gitlab-runner 14.8.2 on GitLab Enterprise Edition 15.0.0-pre 40047f5889b (the one deployed on gitlab.com as of May 8th 2022)

Use case I'm using semgrep to analyze a go+vuejs based project on gitlab: https://gitlab.com/lightmeter/controlcenter

r2c-demo commented 2 years ago

This issue is synced in Linear at https://linear.app/r2c/issue/PA-1262/semgrep-crashes-with-fatal-error-on-analysing-a-go-project. Note: this link is for r2c use only and is not accessible publicly.

leandrosansilva commented 2 years ago

Probably a duplicate of #4917

stale[bot] commented 2 years ago

This issue is being marked stale because there hasn't been any activity in 14 days and either it wasn't prioritized or its priority is high. Please apply the appropriate priority:* label before removing the stale label.

stale[bot] commented 2 years ago

Stale-bot has closed this stale item. Please reopen it if this is in error.

aryx commented 1 year ago

This should work with latest semgrep.