`dune build @doc` does not rebuild pages

[francoisthire]

Expected Behavior

I have modified some mli interface and I expect to see the changes after running dune build @doc.

Actual Behavior

After running dune build @doc, I see nothing (the command exit code is $0$). Worst than that, if I undo my modifications (the git staged area is clear), the index pages is produced, but the link to individual modules' documentation are dangling pointers. The first time I ran dune build @doc they were not. I also tried dune build @doc --force, but it has no effect.

I use dune cache. I have tried to deactivate it (using --cache=disabled CLI option), but it does not change anything.

I also tried to remove the _build directory but the problem still occur.

Reproduction

I am not sure how to reproduce this.

Specifications

• Version of dune (output of dune --version): 3.7.0
• Version of ocaml (output of ocamlc --version): The OCaml toplevel, version 4.14.1
• Operating system (distribution and version): Linux Debian Bullseye

[rgrinberg]

@jonludlam or @Julow can we get a confirmation?

[jonludlam]

With the cache enabled I am seeing some odd effects - I suspect it may be because we don't currently have proper directory targets.

[emillon]

While independently looking for other things, I noticed the following: odoc html-generate generates one html file per module, but only records the top-level one as a target. Not sure that's the same.

[rgrinberg]

That definitely sounds wrong. We never got the rules working under sandboxing, so I'm not surprised by stuff like this.

[emillon]

The odoc html-targets command will output the list of generated files. Is there a way to use this, or do we have to use directory targets for this to work properly?

[rgrinberg]

Using directory targets would be superior, but you can run the command to generate the targets if you want as well.

[emillon]

how would that work? we'd have to run the command to set up the rule, but the output of this command depends on the contents of the input file as well. I'm not sure if the build system supports that.

[rgrinberg]

If the input file is outside the directory in which you're generating the rules, it should work fine.

[emillon]

I tried but I don't understand one thing: I can generate a file that contains the list of targets. But I'd like to add the contents of that file as targets to an action (for example using Action_builder.With_targets.add. But this requires a plain list of files, and things like Action_builder.lines_of return a string list Action_builder.t (which is not surprising to me - this is how we're enforcing that targets are static). So how am I supposed to read the file? Is there an example in Dune where we do something similar? thanks

[rgrinberg]

I can generate a file that contains the list of targets.

You probably don't want this intermediate step. The issue is that you're producing targets for some directory dir, well, where are you doing to put this file with the list of targets? If it's in dir, you will immediately create a cycle. So now you need to think of some other directory where you'll be saving the list of targets. That's all solvable, but you've created some unnecessary headache for yourself.

Alternatively, we can stop caring about re-running odoc to regenerate the list of targets on subsequent invocation of dune. Instead, we can just run it directly and just make sure we build the dependencies it needs explicitly (with the Build_system api).

The closest thing we have to doing this is coq_config.ml or melange_binary.ml

[emillon]

That's fantastic, thanks. I was definitely holding it wrong. I wrote the following:

let html_targets ~dir sctx odocl =
  let* exe_result = odoc_program sctx in
  let exe =
    match exe_result with
    | Ok s -> s
    | Error e -> Action.Prog.Not_found.raise e
  in
  let* () = Build_system.build_file (Path.build odocl) in
  Memo.map ~f:(List.map ~f:(Path.Build.relative dir))
  @@ Memo.of_reproducible_fiber
  @@ Process.run_capture_lines ~dir:(Path.build dir) ~display:Quiet Strict exe
  @@ [ "html-targets"
     ; "-o"
     ; "."
     ; Path.reach ~from:(Path.build dir) (Path.build odocl)
     ]

And this works; but now when I'm defining a rule based on that I have an error because the targets are in different subdirectories. I guess that's a deal breaker. odoc has a --flat option to generate everything in the same directory, that should work better with dune.

[rgrinberg]

doesn't it make sense to use directory targets then?

[jonludlam]

I used directory targets in #7840 for this reason.

[jonludlam]

you definitely don't want to use the --flat output mode - that's intended just for testing.

[arvidj]

Hi, we're getting hit by this bug again. Any ideas? I'd be happy to write a patch to attempt to use the directory targets solution, but I have no experience contributing to dune and I don't know where to start. I'll try to have a look at:

• 7840

• coq_config.ml or melange_binary.ml

as mentioned above.