Fixes to vectorized calls

[njroussel]

This PR fixes various issues in vectorized calls (and some minor orther stuff).

Minor fixes unrelated to calls in:

• 88567877bb7517daa15d83ff463bb341f39e3c7a
• 72741701937ecc81bcad2026a86cba83b9bbb7d4
• ead64a42fa8ed51ed2acfc3070e69bfe2d79480b

Everything else is somehow related to vectorized calls, either only in symbolic mode, or evaluated mode, or for getters or even fordr.dispatch/dr.switch. It might be easier review this PR by looking at the individual commits as they all are independent fixes.

Relies on a fix contained in https://github.com/mitsuba-renderer/drjit-core/pull/88

[wjakob]

These fixes almost all look good to me. Just two comments:

• 9bb4f49e684adafe6488fb311251b61a1bce1977 may conflict with https://github.com/mitsuba-renderer/drjit/commit/7fc0c14ef75cee3678b3ea9d223d6550e1aa50d8. I changed the traverse_pair function so that it can broadcast one argument when size consistency checks are disabled, which was needed to run dr.backward(x) for ArrayXf-typed values.

• Can you explain the context of https://github.com/mitsuba-renderer/drjit/pull/224/commits/6299a0f2c2e03ac7d745cd8b7f9951b2a45d75ad? It seems a bit drastic. For example, it may be useful in some cases to disable symbolic if statements, which switches them over to a mask-based implementation. In contrast to other uses of evaluated mode, such evaluated conditions can actually be included within other symbolic operations. Right now, the system should give you a useful warning message when putting an evaluated operation that actually needs evaluation within a symbolic op.

[njroussel]

For https://github.com/mitsuba-renderer/drjit/commit/6299a0f2c2e03ac7d745cd8b7f9951b2a45d75ad, I agree that (uint32_t) JitFlag::SymbolicCalls | (uint32_t) JitFlag::SymbolicLoops should be enough. This has to do with the couple places where we make dummy calls to zero-initialize a return value (here and here). For example, in evaluated mode we'd need to make a call to a dr.switch target just to figure out the return type. If that target function has a symbolic loop or an other vectorized call in it, then the tracing would be halted as it's most likely going to try to execute them in evaluated mode even though we expliclitly want them to be symbolic here as we only need to figure out the return type. I'll change the code, such that it only applies to the two cases mentioned above and without the SymbolicConditionals. Does that sound good?

[wjakob]

I'll change the code, such that it only applies to the two cases mentioned above and without the SymbolicConditionals. Does that sound good?

Instead of changing the JIT flags, how about changing the interpretation of the default execution policy? For example, in src/extra/cond.cpp, we would change the following lines:

diff --git a/src/extra/cond.cpp b/src/extra/cond.cpp
index 3b776dd4..1a9ac2d7 100644
--- a/src/extra/cond.cpp
+++ b/src/extra/cond.cpp
@@ -580,8 +580,16 @@ bool ad_cond(JitBackend backend, int symbolic, const char *label, void *payload,
     if (strchr(label, '\n') || strchr(label, '\r'))
         jit_raise("'label' may not contain newline characters.");

-    if (symbolic == -1)
-        symbolic = (int) jit_flag(JitFlag::SymbolicConditionals);
+    if (symbolic == -1) {
+        uint32_t flags = jit_flags();
+        if (flags & (uint32_t) JitFlag::SymbolicScope) {
+            // We're inside some other symbolic operation, cannot use evaluated mode
+            symbolic = 1;
+        } else {
+            symbolic = (int) flags & (uint32_t) JitFlag::SymbolicConditionals;
+        }
+    }

For src/extra/loop.cpp, the change would be near-identical. For src/extra/call.cpp, make sure to rebase on master. Over the weekend, I added a similar symbolic=-1/0/1 argument to the operation.

[njroussel]

@wjakob That makes sense to me, I've made those changes in https://github.com/mitsuba-renderer/drjit/pull/224/commits/eeb2328f064b28fd74367143c62c508aa6cba445.

I do think this is what gives us the most flexibility, but is also a bit confusing. Let me recap:

For some control flow that is nested in a symbolic computation the following things could happen:

• If the mode is manually specified and it is symbolic, the code runs as expected
• If the mode is manually specified and it is evaluated, an error message is raised
• If the mode isn't specified but the JitFlag for this construct is symbolic, the code runs as expected
• If the mode isn't specified but the JitFlag for this construct is not symbolic, a warning is raised and a the inner construct is forcefully ran in symbolic mode

(The last item should be pretty rare, I don't think the warning will be too loud.)

I think this gives us the granularity needed to allow user to force "modes" very precisely on certain constructs, have a decent automatic system and also easily have scoped mode changes across a big section of code