shader-slang / slang

Making it easier to work with shaders
http://shader-slang.com
MIT License
2.13k stars 180 forks source link

Unexpected crash when compiling using Slang API #4656

Closed chaoticbob closed 3 months ago

chaoticbob commented 3 months ago

I was using the API calls below before to compile shaders at runtime without a problem. Sorry, I don't remember which commit exactly. However, when I tried to windows release asset from slang-2024.1.30, I got a crash. I checked out the latest source from the repo to make sure it wasn't an issue with the release asset.

When compiling the psmain from the shader below, I'm getting a crash in this stack:

>   slang.dll!std::optional<Slang::CapabilityAtomSet>::has_value() Line 391 C++
    slang.dll!Slang::CapabilitySet::AtomSets::Iterator::operator bool() Line 211    C++
    slang.dll!Slang::validateEntryPoint(Slang::EntryPoint * entryPoint, Slang::DiagnosticSink * sink) Line 533  C++
    slang.dll!Slang::Module::_discoverEntryPoints(Slang::DiagnosticSink * sink, const Slang::List<Slang::RefPtr<Slang::TargetRequest>,Slang::StandardAllocator> & targets) Line 2586    C++
    slang.dll!Slang::FrontEndCompileRequest::checkEntryPoints() Line 953    C++
    slang.dll!Slang::FrontEndCompileRequest::checkAllTranslationUnits() Line 2824   C++
    slang.dll!Slang::Linkage::loadParsedModule(Slang::RefPtr<Slang::FrontEndCompileRequest> compileRequest, Slang::RefPtr<Slang::TranslationUnitRequest> translationUnit, Slang::Name * name, const Slang::PathInfo & pathInfo) Line 3382   C++
    slang.dll!Slang::Linkage::loadModule(Slang::Name * name, const Slang::PathInfo & filePathInfo, ISlangBlob * sourceBlob, const Slang::SourceLoc & srcLoc, Slang::DiagnosticSink * sink, const Slang::Dictionary<Slang::Name *,Slang::Module *,Slang::Hash<Slang::Name *>,std::equal_to<Slang::Name *>> * additionalLoadedModules, Slang::ModuleBlobType blobType) Line 3601  C++
    slang.dll!Slang::Linkage::loadModuleFromBlob(const char * moduleName, const char * path, ISlangBlob * source, Slang::ModuleBlobType blobType, ISlangBlob * * outDiagnostics) Line 1221  C++
    slang.dll!Slang::Linkage::loadModuleFromSource(const char * moduleName, const char * path, ISlangBlob * source, ISlangBlob * * outDiagnostics) Line 1247    C++
    slang.dll!Slang::Linkage::loadModuleFromSourceString(const char * moduleName, const char * path, const char * source, ISlangBlob * * outDiagnostics) Line 1256  C++
    101_color_cube_vulkan_slang.exe!CompileSlangSimple(const std::string & shaderSource, const std::string & entryPoint, const std::string & profile, std::vector<unsigned int,std::allocator<unsigned int>> * pSPIRV, std::string * pErrorMsg) Line 3334   C++
    101_color_cube_vulkan_slang.exe!main(int argc, char * * argv) Line 153  C++

Can I get a sanity check to see if I'm missing something? I checked using slangc and the shader compiles with the psmain entry point without an issue.

Shader

struct CameraProperties {
    float4x4 MVP;
};

[[vk::push_constant]]
ConstantBuffer<CameraProperties> Cam;

struct VSOutput {
    float4 PositionCS : SV_POSITION;
    float3 Color      : COLOR;
};

[shader("vertex")]
VSOutput vsmain(float3 PositionOS : POSITION, float3 Color : COLOR0)
{
    VSOutput output = (VSOutput)0;
    output.PositionCS = mul(Cam.MVP, float4(PositionOS, 1));
    output.Color = Color;
    return output;
}

[shader("pixel")]
float4 psmain(VSOutput input) : SV_TARGET
{
    return float4(input.Color, 1);   
}

API Function

bool res = CompileSlangSimple(gShaders, "psmain", "ps_6_0", &spirvFS, &errorMsg);

bool CompileSlangSimple(
    const std::string&     shaderSource,
    const std::string&     entryPoint,
    const std::string&     profile,
    std::vector<uint32_t>* pSPIRV,
    std::string*           pErrorMsg)
{
    Slang::ComPtr<slang::IGlobalSession> globalSession;
    if (SLANG_FAILED(slang::createGlobalSession(globalSession.writeRef())))
    {
        return false;
    }

    slang::TargetDesc targetDesc = {};
    targetDesc.format            = SLANG_SPIRV;
    targetDesc.profile           = globalSession->findProfile(profile.c_str());
    targetDesc.flags             = SLANG_TARGET_FLAG_GENERATE_SPIRV_DIRECTLY;

    targetDesc.flags |= SLANG_TARGET_FLAG_GENERATE_WHOLE_PROGRAM;

    // Must be set in target desc for now
    targetDesc.forceGLSLScalarBufferLayout = true;

    // Compiler options for Slang
    std::vector<slang::CompilerOptionEntry> compilerOptions;
    {    
        // Force Slang language to prevent any accidental interpretations as GLSL or HLSL
        {
            slang::CompilerOptionEntry entry = {slang::CompilerOptionName::Language};
            entry.value.stringValue0         = "slang";

            compilerOptions.push_back(entry);
        }

        // Use specified entry point instead of main
        {
            compilerOptions.push_back(
                slang::CompilerOptionEntry{
                    slang::CompilerOptionName::VulkanUseEntryPointName,
                    slang::CompilerOptionValue{slang::CompilerOptionValueKind::Int, 1}
            });
        }
    }

    slang::SessionDesc sessionDesc       = {};
    sessionDesc.targets                  = &targetDesc;
    sessionDesc.targetCount              = 1;
    sessionDesc.defaultMatrixLayoutMode  = SLANG_MATRIX_LAYOUT_COLUMN_MAJOR;
    sessionDesc.compilerOptionEntries    = compilerOptions.data();
    sessionDesc.compilerOptionEntryCount = static_cast<uint32_t>(compilerOptions.size());

    Slang::ComPtr<slang::ISession> compileSession;
    if (SLANG_FAILED(globalSession->createSession(sessionDesc, compileSession.writeRef())))
    {
        return false;
    }

    // Load source
    slang::IModule* pSlangModule = nullptr;
    {
        Slang::ComPtr<slang::IBlob> diagBlob;

        pSlangModule = compileSession->loadModuleFromSourceString("grex-module", nullptr, shaderSource.c_str(), diagBlob.writeRef());
        if (pSlangModule == nullptr)
        {
            if (pErrorMsg != nullptr)
            {
                *pErrorMsg = std::string(static_cast<const char*>(diagBlob->getBufferPointer()), diagBlob->getBufferSize());
            }

            return false;
        }
    }

    return true;
}  
ArielG-NV commented 3 months ago

What happened: "ps_6_0" was set for a vertex shader being compiled (which for some reason is handled differently than slangc using "ps_6_0"), this case was (which should be fixed) not handled in Slang correctly.

The solution: Incompatible profile with entry-point should have error'd.