realworldocaml / mdx

Execute code blocks inside your documentation
ISC License
269 stars 45 forks source link

OCaml warning detection heuristic has false positives #434

Closed ncik-roberts closed 9 months ago

ncik-roberts commented 1 year ago

Currently, the OCaml warning detection heuristic looks for the string "Warning".

As you can imagine, this has false positives, but, more relevantly, it has false positives for the Location library in the compiler distribution. Location.t gets printed as Warnings.loc by the top-level, which meets the heuristic.

Input:

```ocaml
# #require "compiler-libs"
```

```ocaml
let loc = Location.none
```

Output of mdx:

```ocaml
# #require "compiler-libs"
```

```ocaml
let loc = Location.none
```
```mdx-error
val loc : Warnings.loc =
  {Location.loc_start =
    {Lexing.pos_fname = "_none_"; pos_lnum = 0; pos_bol = 0; pos_cnum = -1};
   loc_end =
    {Lexing.pos_fname = "_none_"; pos_lnum = 0; pos_bol = 0; pos_cnum = -1};
   loc_ghost = true}
```

This is confusing behavior, as it's not clear what about the output meets the standard of "mdx-error".

ncik-roberts commented 1 year ago

Here's an example OCaml warning that we want the heuristic to catch:

# let f None = ();;
Warning 8 [partial-match]: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
Some _
val f : 'a option -> unit = <fun>

This suggests that looking for the whitespace-separated token Warning would work. (It's still fragile, but it's a lightweight way to improve the heuristic.)