Error: Don't know how to compare array and array (hlc) when compiling Heaps.io for HL/C with DX12

[dipyalov]

Haxe: latest git Hashlink: latest git Heaps.io: latest git

I'm compiling a sample Heaps.io project with DX12 backend to HL/C target. Compiler throws "Error: Don't know how to compare array and array (hlc)" message.

My investigation has shown the following:

The "flushPipeline" method of heaps/h3d/DX12Driver.hx has the following code:

    function flushPipeline() {
        if( !needPipelineFlush ) return;
        needPipelineFlush = false;
        var signature = pipelineSignature;
        var signatureSize = PSIGN_LAYOUT + currentShader.inputCount;
        adlerOut.setI32(0, 0);
        hl.Format.digest(adlerOut, signature, signatureSize, 3);
        var hash = adlerOut.getI32(0);
        var pipes = currentShader.pipelines.get(hash);
        if( pipes == null ) {
            pipes = new hl.NativeArray(1);
            currentShader.pipelines.set(hash, pipes);
        }

And something between:

var pipes = currentShader.pipelines.get(hash);
if( pipes == null ) {

makes HL/C generate the following C code:

    r36 = r0->currentShader;
    if( r36 == NULL ) hl_null_access();
    r35 = r36->pipelines;
    if( r35 == NULL ) hl_null_access();
    r37 = r32;
    r34 = haxe_ds_IntMap_get(r35,r37);
    r38 = (varray*)r34;
    r39 = (varray*)hl_dyn_castp(&r34,&t$_dyn,&t$_array);
    ### WRONG 0 ### r38 == r39 -> label$6a3cd68_69_40 ###;
    hl_assert();
    label$6a3cd68_69_40:
    r40 = r38;
    r43 = r40;
    if( r43 ) goto label$6a3cd68_69_54;

The ### WRONG ### part is my modification to debug this as the haxe compiler actually fails at this point because it can't generate comparison statement for 2 arrays.

[dipyalov]

Forgot to mention - I have --debug option set. And my investigation has shown that this code is generated for debugging purposes in unsafe_cast HL op produced here:

https://github.com/HaxeFoundation/haxe/blob/584a42cb60370b6bf652061e7950603a7500e97e/src/generators/genhl.ml#L2208C4-L2208C4

Still don't understand how it should be fixed yet.

[dipyalov]

Should replacing unsafe_cast_to ctx ret rt e.epos with unsafe_cast_to ~debugchk:false ctx ret rt e.epos be OK? Or is there a special idea to not set debugchk to false?

[onehundredfeet]

I'm getting a similar error, but I have no idea where in my codebase it's coming from. It doesn't show up when compiling for hashlink HL

Error: Don't know how to compare bytes and bytes (hlc)

[onehundredfeet]

I was able to narrow it down to hl.Bytes. But that makes little sense.

class Junk{
    public static function exists2(a : hl.Bytes) {
        return a != null;
    }   
}

class HelloWorld {
    public static function main() {

        trace('Hello, World! ${Junk.exists2(null)}'); 
    }
}

Error: Don't know how to compare bytes and bytes (hlc)

[onehundredfeet]

I don't see HBytes anywhere here, but my ocaml is not great.

let common_type ctx e1 e2 for_eq p =
    let t1 = to_type ctx e1.etype in
    let t2 = to_type ctx e2.etype in
    let rec loop t1 t2 =
        if t1 == t2 then t1 else
        match t1, t2 with
        | HUI8, (HUI16 | HI32 | HI64 | HF32 | HF64) -> t2
        | HUI16, (HI32 | HI64 | HF32 | HF64) -> t2
        | (HI32 | HI64), HF32 -> t2 (* possible loss of precision *)
        | (HI32 | HI64 | HF32), HF64 -> t2
        | (HUI8|HUI16|HI32|HI64|HF32|HF64), (HUI8|HUI16|HI32|HI64|HF32|HF64) -> t1
        | (HUI8|HUI16|HI32|HI64|HF32|HF64), (HNull t2) -> if for_eq then HNull (loop t1 t2) else loop t1 t2
        | (HNull t1), (HUI8|HUI16|HI32|HI64|HF32|HF64) -> if for_eq then HNull (loop t1 t2) else loop t1 t2
        | (HNull t1), (HNull t2) -> if for_eq then HNull (loop t1 t2) else loop t1 t2
        | HDyn, (HUI8|HUI16|HI32|HI64|HF32|HF64) -> HF64
        | (HUI8|HUI16|HI32|HI64|HF32|HF64), HDyn -> HF64
        | HDyn, _ -> HDyn
        | _, HDyn -> HDyn
        | _ when for_eq && safe_cast t1 t2 -> t2
        | _ when for_eq && safe_cast t2 t1 -> t1
        | HBool, HNull HBool when for_eq -> t2
        | HNull HBool, HBool when for_eq -> t1
        | HObj _, HVirtual _ | HVirtual _, HObj _ | HVirtual _ , HVirtual _ -> HDyn
        | HFun _, HFun _ -> HDyn
        | _ ->
            abort ("Don't know how to compare " ^ tstr t1 ^ " and " ^ tstr t2) p
    in
    loop t1 t2

But this is the regular HL code emitter, which works. It's just the HL/C equivalent that doesn't work.

[onehundredfeet]

in hl2c.ml I wonder if adding HByte comparison case at line 747 should be:

match rtype a, rtype b with
            | (HUI8 | HUI16 | HI32 | HF32 | HF64 | HBool | HI64), (HUI8 | HUI16 | HI32 | HF32 | HF64 | HBool | HI64) ->
                phys_compare()
            | HBytes, HBytes ->
                phys_compare()

[onehundredfeet]

@Simn I saw your HArry / HArry compare case change and thought this was something similar.

[onehundredfeet]

Add PR.

https://github.com/HaxeFoundation/haxe/pull/11610