iOS/Metal: Shadows not rendering

[BStringhamVRSK]

On iOS/Metal, no shadows render We are porting our app to iOS, and we are unable to see shadows rendering.

Things we have tried:

• VSM shadows
• PCF shadows
• Changing the near and far planes in camera
• Changing the shadow far plane

The code that drives the Filament libraries is shared on both Windows and iOS, so the calls are identical. Here are some screen shots comparing the two. The IBL lighting is different, but everything else is the same, which includes a directional light with shadows enabled. It seems that either the shadows are failing, or the directional light is not behaving correctly.

I am posting a separate bug report to address the z-buffer fighting issue on the highlighted wall.

Screenshots Windows/OpenGL

iOS/Metal (no shadows)

Smartphone

• Device: iPhone 13 Pro Max
• OS: iOS 16.2

[romainguy]

Do you have code or binaries you can share with us? We won't be able to reproduce easily otherwise.

I explained in your other bug what's going on with the "yellow" wall too.

[BStringhamVRSK]

Thanks for looking into it.

Here's the code we use to create the directional light. In our example, the parameters are: color = {1.0, 0.8, 0.7} intensity = 50000.0 castsShadows = true shadowOpts are constructor defaults.

    LinearColor color(r, g, b);

    EntityID id = _entityMgr->CreateEntity();
    Entity dirEntity;
    _entityMgr->GetEntity(id, dirEntity);

    filament::LightManager::ShadowOptions shadowOpts;
    LightManager::Builder(LightManager::Type::DIRECTIONAL)
        .color(color)
        .intensity(intensity)
        .direction(dir)
        .castShadows(castsShadows)
        .shadowOptions(shadowOpts)
        .build(*_engine, dirEntity);

Here is our view set up code. Use 2 different views: one for the 2D UI and the other for 3D rendering. We have tried to explicitly set view->setShadowingEnabled(true) with no difference.

void FilamentView::SetViewType(Engine* engine, ViewType viewType)
{
    const float DEFAULT_EXPOSURE = 0.8f;
    const float DEFAULT_AO_BIAS = 0.002f;

    _viewType = viewType;

    if (_viewType == ViewType::VIEW_UI)
    {
        AmbientOcclusionOptions aoOptions;
        aoOptions.enabled = false;
        _view->setAmbientOcclusionOptions(aoOptions);

        _view->setAntiAliasing(View::AntiAliasing::NONE);
        _view->setBlendMode(View::BlendMode::TRANSLUCENT);
        _view->setName("UI_View");
        _view->setPostProcessingEnabled(false);
        _view->setShadowingEnabled(false);
        _view->setVisibleLayers(LAYERS_ALL, LAYERS_UI);
    }
    else
    {
        _view->setName("3D_View");
        _view->setVisibleLayers(LAYERS_ALL, LAYERS_3D);
        _view->setAntiAliasing(View::AntiAliasing::FXAA);

        View::BloomOptions bloomOptions;
        bloomOptions.enabled = true;
        bloomOptions.strength = 0.2f;
        bloomOptions.threshold = true;
        bloomOptions.highlight = 100.0f;
        bloomOptions.blendMode = View::BloomOptions::BlendMode::ADD;
        _view->setBloomOptions(bloomOptions);

        ACESToneMapper acesMapper;
        ColorGrading* colorGrad = ColorGrading::Builder()
            .toneMapper(&acesMapper)
            .exposure(DEFAULT_EXPOSURE)
            .build(*engine);
        _view->setColorGrading(colorGrad);

        AmbientOcclusionOptions aoOptions;
        aoOptions.enabled = true;
        aoOptions.bias = DEFAULT_AO_BIAS;
        _view->setAmbientOcclusionOptions(aoOptions);
    }
}

Are there other relevant code/settings you would like me to provide?

[BStringhamVRSK]

Here is our view render code:

bool FilamentRenderContext::Render_Internal(FilamentView& view, FilamentView& viewUI, bool allowFrameSkip)
{
    if (_renderer == nullptr || _swapChain == nullptr)
        return false;
    try
    {
        if (!_renderer->beginFrame(_swapChain) && allowFrameSkip)
        {
            return false;           // need to skip a frame
        }

        // Render 3D view
        _renderer->setClearOptions(*_options3D);
        _renderer->render(view._view);

        // Render UI view
        _renderer->setClearOptions(*_optionsUI);
        _renderer->render(viewUI._view);

        // NOTE: Caller must call _renderer->endFrame()
        return true;
    }
    catch (...)
    {
        return false;
    }
}

[bejado]

Are you able to reproduce on other iPhone devices? What version of Filament are you using?

I don't see any issues with your code that would affect shadowing. One thing to check is that all of your renderables have their bounding boxes correctly set. Additionally, it would be helpful if you could provide a Metal trace to see if the shadows are rendering correctly.

[BStringhamVRSK]

We're using Filament 1.31.1 - should have mentioned it before. I'll try your suggestions.

[loganmlawrence]

Filament_Metal_Trace.trace.zip

Hi @bejado, we've captured a Metal trace (attached) to try and track down this shadow issue. We aren't seeing any shadow pass in the trace, but we do on our Windows version. We are also trying some other experiments to see if we can figure out why.

[BStringhamVRSK]

Also, we are able reproduce missing shadows on an iPad: Model: iPad Pro 4th Gen OS: iPad OS ver 16.1

[bejado]

Thanks, that's helpful. Since the trace doesn't show a shadow pass, this means Filament thinks there shouldn't be any visible shadows.

I suspect that shadowMap.hasVisibleShadows() is returning false. If you're able to set breakpoints in Filament code, you could investigate and see where in ShadowMap::updateDirectional mHasVisibleShadows is being set to false.

[BStringhamVRSK]

We are not able to set breakpoints in the Filament iOS code, but we were able to add some debug output. We discovered that the wsShadowReceiversVolume is empty (with default min/max values) in ShadowMap::updateDirectional and is causing it to return without creating the shadow map.

I combed through our code, and I do not see any place where we might not be passing the "receivesShadows" flag correctly.

Our geometry almost always uses multiple materials and geometries (usually a triangle geometry and a line geometry) which we create using the RenderableManager::Builder, and we always pass in a value for receiveShadows (usually set to true).

So far as I can tell (by watching the Windows version) the bounding box information all seems valid at the point we are passing data to RenderableManager::Builder.

[bejado]

Could you add some logs here in ShadowMap::initSceneInfo? This is where wsShadowReceiversVolume is computed. Either that lambda isn't getting called (because the receiveShadows flag isn't set) or receiver.min/receiver.max aren't properly set.

[loganmlawrence]

@bejado Yes, the logging shows that receiveShadows is always False in ShadowMap::visitScene, so max and min are never being set properly.

[BStringhamVRSK]

@bejado We think this problem is being caused by the way the data is getting marshalled between our C# code and the our C++ wrapper code. We discovered that our receiveShadows flag is set to true in C#, but arrives in the C++ code as false. If we force the C++ code to always set it to true, shadows show up as expected. We will look at some fixes for this tomorrow and let you know, but I'm pretty sure this is the (very unexpected) problem.

[BStringhamVRSK]

@bejado @romainguy We've confirmed that it is the data marshalling that was causing our bug. Thank you for your help in guiding our efforts to find the problem. Closing the issue.

[bejado]

Glad you were able to find the issue.