facebookresearch / nocturne

A data-driven, fast driving simulator for multi-agent coordination under partial observability.
MIT License
258 stars 29 forks source link

Impossible to create render texture (failed to link the target texture to the frame buffer) #28

Open BenQLange opened 2 years ago

BenQLange commented 2 years ago

When running examples/create_env.py, I get this error:

Impossible to create render texture (unsupported anti-aliasing level) Requested: 4 Maximum supported: 1. Impossible to create render texture (failed to link the target texture to the frame buffer).

It seems that it might be caused by SFML package. To get SFML>=2.5, I downloaded it from: https://www.sfml-dev.org/download.php (apt-get install libsfml-dev installs version 2.4.2). Has anyone faced this issue?

eugenevinitsky commented 2 years ago

Oh yes, this is an issue with your machine I suspect that we haven't seen yet on our machines but I think we should have a solution; the machine appears to support a particular anti-aliasing level that is lower than the machines on our cluster.

@nathanlct is there a way to detect the maximum level of antialiasing and use that? Or make the antialiasing level set in the configs so that users can fix this when they see the issue?

In the meantime, we'll have a PR fixing this open in a few minutes.

eugenevinitsky commented 2 years ago

If you want a quick fix you can pull in #30 but we may take a second to merge that while we figure out a prettier solution

BenQLange commented 2 years ago

Thanks for the quick reply. It fixed the first error but the second still remains: Impossible to create render texture (failed to link the target texture to the frame buffer).

I've run basic example from SFML tutorial (https://www.sfml-dev.org/tutorials/2.5/start-vc.php) and the SFML package worked fine

eugenevinitsky commented 2 years ago

Oof, sorry to hear that. We're looking into this ASAP. Could you tell us what GPU you have or any other information you have about your machine?

nathanlct commented 2 years ago

@BenQLange Also, would you be able to try the second example at this link: https://www.sfml-dev.org/tutorials/2.5/graphics-draw.php

ie. the example that creates a render texture, which seems to be the problem here. That should help us locate the issue.

BenQLange commented 2 years ago

@eugenevinitsky, running on a cluster with: Tesla T4, Driver Version: 470.82.01, CUDA Version: 11.4. Visualizing on Mac with XQuartz.

@nathanlct, on it

BenQLange commented 2 years ago

@nathanlct, I've done a sprite and off-screen drawing tutorials. All worked as advertised.

eugenevinitsky commented 2 years ago

Just to confirm, the error is happening on your macbook or on the cluster? If it's the cluster I'll see if I can make a machine with those specs

BenQLange commented 2 years ago

Cluster

eugenevinitsky commented 2 years ago

kk, this might take a little bit longer as I'm not able to reproduce your issues yet; we will try to reproduce it and get back to you. However, the line that's causing your issue is presumably the rendering line in create_env.py? The rest of Nocturne hopefully should work for you on the cluster besides rendering so if you don't need rendering hopefully you are not blocked?

If you don't need rendering on the cluster but just need other nocturne functionality, you could test if the other functionality works by running examples/nocturne_functions.py

nathanlct commented 2 years ago

@BenQLange We're trying to reproduce the error you're getting. Just confirming since you mentioned rendering on Mac and I'm not sure what effects this might have, is the following correct:

int main() { sf::RenderTexture renderTexture; renderTexture.create(500, 500); renderTexture.clear(); renderTexture.display(); }

BenQLange commented 2 years ago

kk, this might take a little bit longer as I'm not able to reproduce your issues yet; we will try to reproduce it and get back to you. However, the line that's causing your issue is presumably the rendering line in create_env.py? The rest of Nocturne hopefully should work for you on the cluster besides rendering so if you don't need rendering hopefully you are not blocked?

If you don't need rendering on the cluster but just need other nocturne functionality, you could test if the other functionality works by running examples/nocturne_functions.py

Ok ok. Thanks for helping. I run examples/nocturne_functions.py. I got the following error at line 128:

/home/bernard.lange/nocturne/examples/rendering/scene_with_peds.png Error executing job with overrides: [] TypeError: Unregistered type : nocturne::Pedestrian

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "examples/nocturne_functions.py", line 128, in main pedestrians = scenario.getPedestrians() TypeError: Unable to convert function return value to a Python type! The signature was (self: nocturne_cpp.Scenario) -> List[nocturne::Pedestrian]

eugenevinitsky commented 2 years ago

Yep, that's on us. We have some missing pedestrian functionality that we just realized, there'll be a PR in shortly to fix it. Otherwise though, that run confirms that everything besides rendering and pedestrian functionality is working for you!

BenQLange commented 2 years ago

Yep, that's on us. We have some missing pedestrian functionality that we just realized, there'll be a PR in shortly to fix it. Otherwise though, that run confirms that everything besides rendering and pedestrian functionality is working for you!

Great. Thank you!

BenQLange commented 2 years ago

@BenQLange We're trying to reproduce the error you're getting. Just confirming since you mentioned rendering on Mac and I'm not sure what effects this might have, is the following correct:

  • when you get the error, are you using the same process of running nocturne on the cluster and visualizing on your Mac?
  • if you just run the following code on the cluster without any Mac visualization, does it work?
#include <SFML/Graphics.hpp>

int main()
{
    sf::RenderTexture renderTexture;
    renderTexture.create(500, 500);
    renderTexture.clear();
    renderTexture.display();
}
eugenevinitsky commented 2 years ago

We'll keep looking into this tomorrow but in the meantime if you pull master we've fixed the issues with nocturne_functions.py If there's any other blocking issues on the repo do let us know!

nathanlct commented 2 years ago

@BenQLange Would you be able to try the following (at your convenience!):

#include <SFML/Graphics.hpp>

int main()
{
    sf::RenderTexture renderTexture;
    renderTexture.create(2000, 2000);
    renderTexture.clear(sf::Color::Blue);
    renderTexture.display();

    const sf::Image image = renderTexture.getTexture().copyToImage();
    const unsigned char* pixels = (const unsigned char*)image.getPixelsPtr();

    sf::RenderTexture renderTexture2;
    sf::ContextSettings texture_settings;
    texture_settings.antialiasingLevel = std::min(sf::RenderTexture::getMaximumAntialiasingLevel(), 4u);
    renderTexture2.create(2000, 2000, texture_settings);
    renderTexture2.clear(sf::Color::Blue);
    renderTexture2.display();
}

That would allow us to test a few other differences I'm seeing between our code and the SFML example.

BenQLange commented 2 years ago

@nathanlct, I think we are getting closer. This script gives the same error. It also gives additional two new errors:

libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
Impossible to create render texture (failed to link the target texture to the frame buffer)

The 'Impossible...' error seems to be caused by line 'texture_settings.antialiasingLevel...' . If I comment it out, the output is:

libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast

The first two errors seems to be XQuartz related: https://github.com/XQuartz/XQuartz/issues/144. Not sure, if it has anything to do with 'Impossible...' error

eugenevinitsky commented 2 years ago

Thank you for testing this out! @nathanlct should respond in a bit (though he's in French timezone so I'm not sure when)

I think I have a "hack" that will solve this temporarily while we try to find a permanent solution; I think if you in the code you are running do:

import os
del os.environ["DISPLAY"]

that will put things into headless mode and then everything should render properly using pyvirtualdisplay as the renderer

BenQLange commented 2 years ago

Thank you for testing this out! @nathanlct should respond in a bit (though he's in French timezone so I'm not sure when)

I think I have a "hack" that will solve this temporarily while we try to find a permanent solution; I think if you in the code you are running do:

import os
del os.environ["DISPLAY"]

that will put things into headless mode and then everything should render properly using pyvirtualdisplay as the renderer

Then display forwarding breaks:

python nocturne_functions.py 
Failed to open X11 display; make sure the DISPLAY environment variable is set correctly
Aborted (core dumped)
BenQLange commented 2 years ago

Setting the environment variable LIBGL_ALWAYS_INDIRECT to 1 fixes the libGL errors (It's XQuartz related). Now I only have:

Impossible to create render texture (failed to link the target texture to the frame buffer)

BenQLange commented 2 years ago

Oh wait a second.

This line breaks X11 forwarding:

import os
os.environ["DISPLAY"] = ""

But this line doesn't and also no 'Impossible...' error appear:

import os
del os.environ["DISPLAY"]

It works! Thanks!

You can close it now, or later when you find more permanent solution.

Here is a visualized scene: scene_with_peds

eugenevinitsky commented 2 years ago

YESSSSSS. Okay, we'll try to find a more permanent solution for you and will ping you on this issue when we do. Thank you for helping us debug this!

BenQLange commented 2 years ago

Sounds good. Happy to help!

nathanlct commented 2 years ago

Glad the problem is solved @BenQLange! :)

I'm a bit confused though as to what happens on Mac vs. on your cluster, are you saying that it's only working with X11 forwarding, and still erroring if you run it on the cluster, unless we remove the anti-aliasing setters? If so, would you able to print what sf::RenderTexture::getMaximumAntialiasingLevel() returns (without X11 forwarding)?

BenQLange commented 2 years ago

My understanding is that the package runs on the cluster, and only visualization through XQuartz/X11 forwarding is done on Mac. If I enable X11 forwarding, sf::RenderTexture::getMaximumAntialiasingLevel() returns 16. If I don't enable X11 forwarding nothing gets printed and I get:

Failed to open X11 display; make sure the DISPLAY environment variable is set correctly
Aborted (core dumped)

In nocturne:

I don't need X11 forwarding for tasks without rendering. E.g. imitation learning script works without it.

When I run a script where a scene is rendered (e.g. create_env.py/rendering.py),

1) without X11 forwarding it will crash:

Failed to open X11 display; make sure the DISPLAY environment variable is set correctly
Aborted (core dumped)

2) with X11 forwarding but without Eugene's fix, it will run but no scene is rendered (empty output images). And this is returned: Impossible to create render texture (failed to link the target texture to the frame buffer)

3) with X11 forwarding and with Eugene's fix, it will run just fine.

Hope it helps

eugenevinitsky commented 2 years ago

Definitely helps! Also, this'll be much clearer once we upgrade the docs but Nocturne can run on cluster or local machine since the simulator itself runs on CPU; it's just the RL / IL examples that are GPU accelerated