Serializing JIT compilation

[soufianekhiat]

Is they is a way to serialize a jit compilation?

Use case:

// Build the Func f;
Halide::Target target = Halide::get_host_target();
uint64_t hash = HashFunc( f, target ); // TODO
Pipeline* pPipeline = nullptr;
if ( ExistOnDisk( hash ) ) // TODO
{
    pPipeline = GetPipelineFromDisk( hash ); // TODO
}
else
{
    pPipeline = Pipeline( f );
    pPipeline->auto_schedule( GetCurrentAutoSchedule(), target, Halide::MachineParams( GetCoreCount(), IsGPU() ? GetGPUMemory() : GetCPUMemory(), GetBalance( IsGPU() ) ) );
    pPipeline->compile_jit( target );
    StoreOnDisk( hash, f ); // TODO
}

I try to look on Module but all data are read-only.

I know most of users are focus on using AOT. I would like to push to the direction of JIT. Currently to make the usage of JIT more viable would be a way to serialize the jit compilation on the disk of final user. Like C#, Java they cache the Bytecode. On our case does they is a way to cache the final compilation? Module? IR? Binary? Hotload of dll/so?

[abadams]

I wish llvm could emit shared objects directly. The only way to do this now I think requires compiling to an object, then shelling out to a linker to turn that into a .so, and then dlopening the .so.

[alexreinking]

I wish llvm could emit shared objects directly. The only way to do this now I think requires compiling to an object, then shelling out to a linker to turn that into a .so, and then dlopening the .so.

Is there not a way to do this by linking to lld? We do something like this for lldWasm, don't we?

[abadams]

I was under the impression lld was not available as a library, just as a command-line tool.

[steven-johnson]

I was under the impression lld was not available as a library, just as a command-line tool.

Nope, it's totally available as a library. We use lldWasm in this way.

[abadams]

Might be cool to support shared library as an output type then, as it would enable the sort of thing requested in this issue.

[mikewoodworth]

One option might be to emit bytecode of the module post optimization... though I have no idea what percentage of the overall JIT time that would save. I know Apple did something similar for OpenCL kernels in the past.

[soufianekhiat]

After a try I maybe missed something. I produced an obj with: compile_to_object. And linked it with: lld-link.exe /defaultlib:libcmt.lib /wholearchive /machine:x64 /subsystem:windows /dll dumpObj /out:dumpObj.dll

Note: Currently my target is Windows only.

Checking the format of the obj generated:

File: dumpObj
Format: COFF-x86-64
Arch: x86_64
AddressSize: 64bit

By checking the content of the generated DLL I don't see my function which is present on the object by checking with: llvm-nm.exe myObjGeneratedWith_compile_to_object

I can see (here dummycos of a linear buffer): 00000000 T dummycos 00000000 t dummycos.par_for.Cos.s0.x.x 00000000 T dummycos_argv 00000000 T dummycos_metadata

[alexreinking]

By checking the content of the generated DLL I don't see my function which is present on the object by checking with:

@soufianekhiat - it looks like your comment got cut off?

[soufianekhiat]

@alexreinking my bad fixed.

[soufianekhiat]

After trying to dump differently with /wholearchive:dumpObj.lib:

lld-link.exe /defaultlib:libcmt.lib /implib:dumpObj.lib /wholearchive /wholearchive:dumpObj.lib /machine:x64 /subsystem:windows /dll dumpObj.obj /out:dumpObj.dll
lld-link: error: dumpObj.lib(dumpObj.dll): .idata$4 should not refer to special section 0

Without the dll is just empty:

link.exe /dump /exports dumpObj.dll
Microsoft (R) COFF/PE Dumper Version 14.00.24234.1
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file dumpObj.dll

File Type: DLL

  Summary

        1000 .00cfg
        9000 .data
        1000 .gehcont
        1000 .pdata
        B000 .rdata
        1000 .reloc
       13000 .text
        1000 _RDATA

Same for:

>clang.exe -O0 -shared -o dumpDLL2.dll dumpObj.obj -LdumpObj.lib
>C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\dumpbin.exe" /exports dumpDLL2.dll

Produces:

Dump of file dumpDLL2.dll

File Type: DLL

  Summary

        9000 .data
        1000 .pdata
        B000 .rdata
        1000 .reloc
       13000 .text
        1000 _RDATA

[soufianekhiat]

Solution add export by hand with option: /wholearchive /include:coscos /export:coscos Relevant ref: https://stackoverflow.com/a/58913687 Ideally we would like either output directly a dll or a def file: https://stackoverflow.com/a/62360059