Included files are not always found

dszoboszlay commented 11 months ago

Describe the bug

Following your CodeBEAM presentation on ELP I'm trying out this exciting tool on our code base, but run into a problem with included files are not always being found (which leads to reporting all kind of errors, as expected).

To Reproduce

Clone the brod project from https://github.com/kafka4beam/brod

Add eqwalizer_support & eqwalizer_rebar3 to the rebar.config according to this project's README:

{deps, [ {kafka_protocol, "4.1.3"}
    , {snappyer, "1.2.9"}
+       ,  {eqwalizer_support,
+    {git_subdir,
+        "https://github.com/whatsapp/eqwalizer.git",
+        {branch, "main"},
+        "eqwalizer_support"}}
    ]}.
+
+{project_plugins, [ { eqwalizer_rebar3
+                    , { git_subdir
+                      , "https://github.com/whatsapp/eqwalizer.git"
+                      , {branch, "main"}
+                      , "eqwalizer_rebar3"
+                      }
+                    }
+                  ]}.
+

Build the project with rebar3 compile, and verify that there aren't any compilation issues
From e.g. brod_cli:kf/3 navigate to kpro:find/3's definition using ELP.

Expected behavior

There should be no warnings reported on kpro.erl by ELP, and I should be able to use go to the definition on the include line: -include("kpro_private.hrl").

Actual behavior

ELP complains that _No definition found for 'kproprivate' and reports a lot of (false) warnings and errors on the kpro module itself.

Context

ELP Version (output of elp version): elp 1.1.0+build-2023-10-16
Editor used: VS Code

alanz commented 11 months ago

I just checked this with current master (da0b80b04ab28e75ab1220398bfabc183d1f41fa) and it works as described. We will make a new release for it.

alanz commented 11 months ago

Please try again with today's release, and re-open if it is still a problem.

dszoboszlay commented 9 months ago

Sorry for the late reply! I finally found some time to play with elp again, and it looks like I still cannot navigate to kpro_private.hrl.

Context

ELP Version (output of elp version): elp 1.1.0+build-2023-12-15
- Editor used: VS Code

dszoboszlay commented 9 months ago

@alanz I'm not allowed to re-open this issue, but please consider it to be re-opened!

alanz commented 8 months ago

@dszoboszlay I took a look at this again, against brod at 6966f0244b87904363cb8053ac0364ff281411d9.

If I put my cursor on kpro:find( on line 1263

kf(FieldName, Struct) -> kpro:find(FieldName, Struct).

It takes me to _build/default/lib/kafka_protocol/src/kpro.erl line 448

find(Field, Struct) -> kpro_lib:find(Field, Struct).

Is this what you are expecting, or should there be some other behaviour?

dszoboszlay commented 8 months ago

@alanz Yes, navigating to kpro:find/2 works. However, this part of my original issue still stands:

There should be no warnings reported on kpro.erl by ELP, and I should be able to use go to the definition on the include line: -include("kpro_private.hrl").

elp reports 22 errors and 2 warnings for me in kpro.erl, all because it cannot find kpro_private.hrl. These are the errors by the way:

[{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1227",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1227",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "function decode_batches/1 undefined",
    "source": "elp",
    "startLineNumber": 68,
    "startColumn": 1,
    "endLineNumber": 73,
    "endColumn": 11
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1295",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1295",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "type batch_decode_result() undefined",
    "source": "elp",
    "startLineNumber": 75,
    "startColumn": 1,
    "endLineNumber": 138,
    "endColumn": 16
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1295",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1295",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "type batch_meta() undefined",
    "source": "elp",
    "startLineNumber": 75,
    "startColumn": 1,
    "endLineNumber": 138,
    "endColumn": 16
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1295",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1295",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "type compress_option() undefined",
    "source": "elp",
    "startLineNumber": 75,
    "startColumn": 1,
    "endLineNumber": 138,
    "endColumn": 16
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1295",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1295",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "type incomplete_batch() undefined",
    "source": "elp",
    "startLineNumber": 75,
    "startColumn": 1,
    "endLineNumber": 138,
    "endColumn": 16
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1295",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1295",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "type str() undefined",
    "source": "elp",
    "startLineNumber": 75,
    "startColumn": 1,
    "endLineNumber": 138,
    "endColumn": 16
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "E1516",
        "target": {
            "$mid": 1,
            "external": "https://whatsapp.github.io/erlang-language-platform/docs/erlang-error-index/e/E1516",
            "path": "/erlang-language-platform/docs/erlang-error-index/e/E1516",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "can't find include file \"kpro_private.hrl\"",
    "source": "elp",
    "startLineNumber": 140,
    "startColumn": 10,
    "endLineNumber": 140,
    "endColumn": 28
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "E1507",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/e/E1507",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "undefined macro 'null'",
    "source": "elp",
    "startLineNumber": 146,
    "startColumn": 23,
    "endLineNumber": 146,
    "endColumn": 28
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1295",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1295",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "type compress_option() undefined",
    "source": "elp",
    "startLineNumber": 167,
    "startColumn": 43,
    "endLineNumber": 167,
    "endColumn": 58
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "E1508",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/e/E1508",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "undefined macro 'incomplete_batch/1'",
    "source": "elp",
    "startLineNumber": 183,
    "startColumn": 29,
    "endLineNumber": 183,
    "endColumn": 46
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1252",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1252",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "record kafka_message undefined",
    "source": "elp",
    "startLineNumber": 184,
    "startColumn": 20,
    "endLineNumber": 184,
    "endColumn": 36
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1252",
        "target": {
            "$mid": 1,
            "external": "https://whatsapp.github.io/erlang-language-platform/docs/erlang-error-index/l/L1252",
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1252",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "record kpro_req undefined",
    "source": "elp",
    "startLineNumber": 196,
    "startColumn": 16,
    "endLineNumber": 196,
    "endColumn": 27
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1252",
        "target": {
            "$mid": 1,
            "external": "https://whatsapp.github.io/erlang-language-platform/docs/erlang-error-index/l/L1252",
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1252",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "record kpro_rsp undefined",
    "source": "elp",
    "startLineNumber": 197,
    "startColumn": 16,
    "endLineNumber": 197,
    "endColumn": 27
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "E1507",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/e/E1507",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "undefined macro 'no_compression'",
    "source": "elp",
    "startLineNumber": 198,
    "startColumn": 28,
    "endLineNumber": 198,
    "endColumn": 43
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "E1507",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/e/E1507",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "undefined macro 'KPRO_NO_BATCH_META'",
    "source": "elp",
    "startLineNumber": 245,
    "startColumn": 23,
    "endLineNumber": 245,
    "endColumn": 42
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1295",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1295",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "type batch_meta() undefined",
    "source": "elp",
    "startLineNumber": 247,
    "startColumn": 19,
    "endLineNumber": 247,
    "endColumn": 29
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "E1508",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/e/E1508",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "undefined macro 'incomplete_batch/1'",
    "source": "elp",
    "startLineNumber": 248,
    "startColumn": 32,
    "endLineNumber": 248,
    "endColumn": 49
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "E1507",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/e/E1507",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "undefined macro 'snappy'",
    "source": "elp",
    "startLineNumber": 272,
    "startColumn": 29,
    "endLineNumber": 272,
    "endColumn": 36
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1295",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1295",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "type compress_option() undefined",
    "source": "elp",
    "startLineNumber": 300,
    "startColumn": 44,
    "endLineNumber": 300,
    "endColumn": 59
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1308",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1308",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "spec for undefined function decode_batches/1",
    "source": "elp",
    "startLineNumber": 310,
    "startColumn": 1,
    "endLineNumber": 310,
    "endColumn": 54
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1295",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1295",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "type batch_decode_result() undefined",
    "source": "elp",
    "startLineNumber": 310,
    "startColumn": 35,
    "endLineNumber": 310,
    "endColumn": 54
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "E1507",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/e/E1507",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 8,
    "message": "undefined macro 'INT'",
    "source": "elp",
    "startLineNumber": 312,
    "startColumn": 23,
    "endLineNumber": 312,
    "endColumn": 27
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1296",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1296",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 4,
    "message": "type batch_meta_key() is unused",
    "source": "elp",
    "startLineNumber": 237,
    "startColumn": 1,
    "endLineNumber": 242,
    "endColumn": 20
},{
    "resource": "/home/dszoboszlay/src/brod/_build/default/lib/kafka_protocol/src/kpro.erl",
    "owner": "_generated_diagnostic_collection_name_#0",
    "code": {
        "value": "L1296",
        "target": {
            "$mid": 1,
            "path": "/erlang-language-platform/docs/erlang-error-index/l/L1296",
            "scheme": "https",
            "authority": "whatsapp.github.io"
        }
    },
    "severity": 4,
    "message": "type batch_meta_val() is unused",
    "source": "elp",
    "startLineNumber": 244,
    "startColumn": 1,
    "endLineNumber": 244,
    "endColumn": 55
}]

Also, if you go to kpro.erl line 140 (-include("kpro_private.hrl").) and try to navigate to the header file, then this will fail.

robertoaloi commented 7 months ago

@dszoboszlay I can reproduce. Let me take a look.

robertoaloi commented 7 months ago

@dszoboszlay This should actually be solved by #13, which is merged now. It would be great if you could confirm this is solved using the latest ELP version.

dszoboszlay commented 7 months ago

@robertoaloi Yes, thank you, it's indeed working now with the latest ELP version. This ticket can be closed.

WhatsApp / erlang-language-platform