search

fegennari / 3DWorld

3D Procedural Game Engine Using OpenGL
GNU General Public License v3.0
1.17k stars 92 forks source link

Compilation, Makefile and README.linux #2

Closed Lecrapouille closed 5 years ago

Lecrapouille commented 5 years ago

Edit: I rephrase because I was in urge and I had not time to read myself the 1st time.

Compilation problem

SHA1 2f36272844a0d135308846cc0448c24a52ad02de The compilation is broken with gli. I tried the currently present folder then the git cloned version but got the same problem:

In file included from dependencies/gli/gli/format.hpp:6:0,
                 from dependencies/gli/gli/gli.hpp:39,
                 from src/image_io.cpp:25:
dependencies/gli/gli/type.hpp:42:24: error: ‘qualifier’ has not been declared
  template <typename T, qualifier P>

I have a Debian 9.8, amd 64, and g++-6.0. I dunno yet how to fix this. Note that glib compiles well by itself.

Some confusing points inside the README.linux:
Some confusing points inside the Makefile

all: $(TARGET)

%.o : %.C $(BUILD)/%.d $(CPP) $(CPPFLAGS) -MMD -c $< -o $(abspath $(BUILD)/$@) %.o : %.cc $(BUILD)/%.d $(CPP) $(CPPFLAGS) -MMD -c $< -o $(abspath $(BUILD)/$@) %.o : %.cpp $(BUILD)/%.d $(CPP) $(CPPFLAGS) -MMD -c $< -o $(abspath $(BUILD)/$@)

$(OBJS): | $(BUILD) $(BUILD): @mkdir -p $(BUILD)


Explainations:
Firstly, feel free to rename BUILD variable, it indicates the folder holding your *.o files. You have to refer this folder to VPATH. The `all:` rule now simply depends on your binary file. For compiling your c(pp) files you have to add the `-o`option else *.o are created next to your Makefile instead of inside `$(BUILD)` folder. Note that you also have to depend of `%.d` files, so added them in the dependency rule. I added `abspath` but this is optional but this helps coverage tool like `gcov` to find the source. Finaly `$(OBJS): | $(BUILD)`replace your if then else. `$(BUILD)` rule is called before  building `$(OBJS)`. Also note that you no longer need of `../../` for TARGA, GLUI ... 

* Unfortunaltly there are still problems with `*.d` files: they are created at the root of the directory and makefile will recompiling all c(pp) files everytime. So replace your code based on `touch/cat` commands by this code which does the same thing:

$(BUILD)/%.d: ; .PRECIOUS: $(BUILD)/%.d -include $(patsubst %,$(BUILD)/%.d,$(basename $(OBJS)))


With all these changes you simply can go to the root of the project and type make! No more ln or mv :)

* Make clean can now simply remove the folder $(BUILD).
* I attached my Makefile to this issue. [makefile.txt](https://github.com/fegennari/3DWorld/files/3244687/makefile.txt) Note: because of github I had to add the `.txt` extension else it refuses to attach it.

As said previously I could not finish compiling because of gli so I cannot  check if my Makefile is fully correct. For more information you can see my Makefile [repo](https://github.com/Lecrapouille/MyMakefile).

##### chmod
* There are plenty of ASCII files in executable mode. You should fix their rights with a `chmod 644` git manages the chmod rights.
Lecrapouille commented 5 years ago

And here the backward trace backward.txt

fegennari commented 5 years ago

Yeah I remember running into the GLM default matrix != identity problem. I wrote my own wrapper that called identity() in the constructor.

I have no idea why that assert fails. Have you made any code changes? Some of the errors I saw in your big diff where you added all those 'f's to the floating-point numbers could have broken it. Did you revert that set of changes? If your stat is clean, then maybe it's some compiler issue. I don't see how I can debug it if I can't reproduce it. What gcc version are you using? My only thought is to disable the "#pragma omp parallel for" around the BVH construction, or make sure do_mt_build=0 in that case. It could be a thread problem.

fegennari commented 5 years ago

Try adding this line to cobj_bsp_tree.h line 134 inside obj_okay(): assert(c.is_normalized());

If that fails, some object with a denormalized bounding cube was added to the tree. Then you can print out the data for the failing object. If it doesn't fail, then something got corrupted in memory at some point. Maybe start by calling c.print_bounds(): if (!c.is_normalized()) {c.print_bounds(); assert(0);}

I haven't actually checked this code...

fegennari commented 5 years ago

I made an attempt to fix this problem. There was a very strange error where a class member variable wasn't being properly initialized in the constructor, and renaming the argument (???) fixed it. I also updated a conditional to make it safer and added debug printouts into that failing case. Can you try it now to see if it still fails with that assert?

Lecrapouille commented 5 years ago

I did not change float literals. I'm using an older desktop (Ubuntu 18.10 AMD 32bits) because I'm out of my home. My g++ is 7.4.0. Now I'm passing this step but now no kidding I'm failing at the next assertion :(

3DWorld/src/csg.cpp:193: unsigned int cube_t::get_split_dim(float&, float&, unsigned int) const: Assertion `dim_sz >= 0.0' failed.

I'm going to try without openmp

Lecrapouille commented 5 years ago

Even disabling openmp I got the same error. PS: it may be interessing to wrap #ifdef _OPENMP functions to avoid this compilation error:

snow.o : Dans la fonction « create_snow_map(voxel_map&) » :
3DWorld/src/snow.cpp:501 : référence indéfinie vers « omp_get_thread_num »
Universe_control.o : Dans la fonction « omp_get_thread_num_3dw() » :
3DWorld/src/Universe_control.cpp:63 : référence indéfinie vers « omp_get_thread_num »
collect2: error: ld returned 1 exit status
makefile:32: recipe for target '3dworld' failed
fegennari commented 5 years ago

Which scene is this failing on? The mapx scene doesn't have enough objects to enable the parallel OpenMP tree build, so I don't think it's a threading problem. The shapes aren't modified anyway. The new assert is probably the same problem, it's just failing at a different place in the tree building. Can you share the stack trace for that one? It seems like it could be uninitialized memory access. Can you run under valgrind? That's a similar OS and gcc to the one I'm using, and I don't have this problem. I don't see how it can be a 32 vs. 64 bit problem here.

I need to figure out what the bounding cube values are before I can guess at the problem. Are they reasonable numbers but just wrong? Are they all random garbage numbers with large exponents from uninitialized memory? Are they NaN values from some divide-by-zero or other such thing?

The omp_get_threadnum() calls are supposed to be guarded by #ifdef OPENMP. I must have missed some. I should change it to the way I have at work so all of those omp* functions are abstracted with a wrapper that does nothing in the case where OpenMP is disabled.

Lecrapouille commented 5 years ago

The default mapx scene . No Valgrind error found. backward.txt As an alternative investigation, you can freely (through your GitHub account) access to Coverity Scan. This tool does static code analysis. You compile against their tool. They freeze you 1 week after the first upload to check your bugs. After that, you can see bugs the tool found.

Lecrapouille commented 5 years ago

PS: and yep its NaN for d[i][1] and d[i][0]

Lecrapouille commented 5 years ago

ooooh I compiled with -O0 + backtrace + no openmp (that I did not restored) and that passed:) I got the scene but this time really running at 5 -15fps (and the ESC key looks be broken, I could escape with 'm'). I'll try again valgrind.

Lecrapouille commented 5 years ago

Here the valgrind valgrind.txt except errors from the nouveau_driver valgind found nothing

fegennari commented 5 years ago

I don't think any of those valgrind uninitialized errors are related to the assert. Maybe it's because you can't load the dragon model. It could be that it still adds it but with some garbage bounding cube. I'll check tonight. If that's not it, then I have no idea what's going on. Can you send me the text written to the terminal for a failing run? I want to compare to see if the numbers are any different.

Lecrapouille commented 5 years ago

This ? I'm not sure about what you really mean by "Can you send me the text written to the terminal for a failing run". Dragon is for nothing because I have it since the beginning.

 obj/3dworld
Starting 3DWorld
Using config file mapx/config_mapx.txt.
Loading.....
Loading Sounds....................................................
...GL Initialized.
loading textures done
max TIUs: 16, max combined TIUs: 48
Texture Load time = 1429
OpenGL Version: 4.5 (Compatibility Profile) Mesa 18.2.8
Renderer: AMD RS880 (DRM 2.50.0 / 4.15.0-51-generic, LLVM 7.0.0)
Vendor: X.Org
GLSL Shader Version: 4.50
mesh = 128x128, scene = 10x10
Generating Scene...
Read input mesh file 'mapx/mesh128.txt'.
Surface generation time = 6
Matrix generation time = 9
 Gen Landscape Texture time = 50
Landscape Texture generation time = 59
Volume+Motion matrix generation time = 62
Scenery generation time = 62
..Error opening model3d file for read: ../models/dragon.model3d
Error reading model3d file ../models/dragon.model3d
Error reading model file data from file ../models/dragon.model3d; Model will be skipped
Reading model3d file model_data/fish/fishOBJ.model3d
Model3d File Load time = 0
loading material library fishOBJ.mtl
Model Texture Load time = 55
model stats:
bcube: -0.150792,0.774795, -1.39715,-1.27357, -3.40216,-2.42594 
center: 0.312001, -1.33536, -2.91405, size: 0.925586, 0.123583, 0.976219
verts: 2213, quads: 1104, tris: 69, blocks: 2, mats: 1
Total Model3d Load time = 55
Model File Load/Process time = 55
Create and Xform Model3d Polygons time = 1
polygons: 1214, grid: 14x2x5
Model3d Polygons to Cubes time = 1
grid size: 140, cubes out: 45
.1218 => 1479
Negative Shape Processing time = 2
1479 => 1598
Cube Overlap Removal time = 2
 Add Fixed Cobjs time = 5
bins = 16384, ne = 9803, cobjs = 26071, ent = 119447, per c = 4, per bin = 7
3dworld: /home/qq/3DWorld/src/csg.cpp:194: unsigned int cube_t::get_split_dim(float&, float&, unsigned int) const: Assertion `dim_sz >= 0.0' failed.
Abandon (core dumped)

Strange why this bug appears with -O3 and disappears with -O0. On another part, I do not understand why on Debian a g++-6.0 does not show bugs that the same g++ version detects on Ubuntu (I'm talking about my personal projects) !!!!

fegennari commented 5 years ago

That's interesting. The number of v_coll_matrix entries here is 119,447, while it's 119,471 in the other run that doesn't fail. That shows that the code diverges before building the BVH. Is the only difference the optimization flag? I wonder if it's some sort of compiler bug? Or maybe it's some type of C++ undefined behavior that the compiler handles differently depending on optimization mode. I have no idea how to debug something like that. It should be possible to push that error check earlier into the flow and eventually track it down to where the bad value is set/inserted, if you're willing to keep running experiments. I've run on 4 different machines and haven't seen the failure.

fegennari commented 5 years ago

Is there a way to force gcc to zero all variables? Maybe just memset all fields of coll_obj to zero in the constructor? The only thing I can think of is some uninitialized variable, or maybe an invalid operator<() used in a sort function. Maybe the memcmp() in obj_layer::operator<() is picking up some unused garbage bytes? But then I would think valgrind can find it. Can you run valgrind on the -O3 build?

fegennari commented 5 years ago

I consistently get this line without the dragon model on Windows and gcc with -O3 and -O0: bins = 16384, ne = 9803, cobjs = 27576, ent = 121505, per c = 4, per bin = 7

You get this: bins = 16384, ne = 9803, cobjs = 26071, ent = 119447, per c = 4, per bin = 7

I'm not sure what accounts for the difference. I'm using gcc-7.4 as well. It's possibly some difference in the floating-point math. For example, I have a line: val = sqrt(1.0 - normal.x); normal.x should always be <= 1.0, but maybe there's some double vs. float problem and it comes out slightly negative? I added a max with 0.0 that should fix it. I also added various other asserts, a divide-by-zero check, and a better error message printout for that assert. Maybe you can try it again to see if anything has changed?

Lecrapouille commented 5 years ago

Ok good observations about the number of objects. I gonna redo tests with this in mind. First, compilation errors with the current Makefile:

3DWorld/src/mesh_intersect.cpp:378:38: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  assert((tree[level] - tree[0]) + ix < (int)bsp_data.size());

3DWorld/src/ship.h:1074:7: warning: ‘<anonymous>.u_ship_base::wpt_center’ may be used uninitialized in this function [-Wmaybe-uninitialized]
 class u_ship_base { // Note: Can be created on the stack and copied

3DWorld/src/cars.cpp:801:13: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (*ix != (i - cars.begin())) {check_collision(*i, cars[*ix]);}

3DWorld/src/gen_buildings.cpp:1698:64: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   float const door_ztop((!doors.empty() && (i - parts.begin()) == unsigned(door_part)) ? doors.front().pts[2].z : 0.0);

Second, I compiled with your last patch, no assertions occurred and bins = 16384, ne = 9803, cobjs = 26071, ent = 119471, per c = 4, per bin = 7 whatever with -O3 or -O0. So seems ok now :)

fegennari commented 5 years ago

Why don't I see those compiler warnings? That's odd. I know I fixed a whole lot of signed vs. unsigned problems.

I'm guessing it was that sqrt() function. I've had problems like this in the past.

I still don't understand why your cobjs and entries counts are different from mine. There may be something else wrong. Are you using the GLM from inside the GLI directory rather than the one directly inside dependencies? That could be the problem. GLI uses an older GLM. It may be a version from back when matrices were initialized with all zeros rather than an identity matrix by default. You shouldn't be using that version of GLM. I'll go back and see if switching to the older GLM inside of GLI changes my numbers.

Lecrapouille commented 5 years ago

PS: reverting to the previous commit: in -O3 or -O2:

bins = 16384, ne = 9803, cobjs = 26071, ent = 119447, per c = 4, per bin = 7

In -O0:

bins = 16384, ne = 9803, cobjs = 26071, ent = 119471, per c = 4, per bin = 7

I'm not a big fan of -O3 or with -Os (especially with -g) but I'm not an expert. You can compile with -O2 -g and rename your application 3dworld-debug. Then call strip -R .comment -R .note -R .note.ABI-tag 3dworld for the release (a binary without debugging symbols).

For the computations, 1.0 is double while 1.0f is float. I'm not expert too but I think when mixing double and float, computations are made in double then cast to float. So it's ok except maybe more CPU computational. For computations with Nan: I guess than max(0.0f, NaN) will return 0.0f. Some people preferer that retun NaN.

fegennari commented 5 years ago

The error is generating NaN in the first place. What do you expect this to do: float f = 1.0; float v = sqrt(1.0 - f); I would think this would result in v=0 because 1.0 can be represented exactly in a floating-point number. But if the compiler does some strange optimization, maybe this can return NaN because the value going into sqrt() is some tiny negative number? Can you test this code on your compiler with different optimizations and print the results? If you do get NaN, does replacing the math with "v = sqrt(1.0f - f)" fix it?

Lecrapouille commented 5 years ago

Why don't I see those compiler warnings? That's odd. I know I fixed a whole lot of signed vs. unsigned problems.

Like I said, I have Debian and Ubuntu with the same version of g++ and got different warnings (or worse errors). So yep, more archi you can compile, better it is. Else, use docker :)

Are you using the GLM from inside the GLI directory rather than the one directly inside dependencies?

In my Makefile I made -I$(GLI)/external for fixing the initial error of this post

In file included from dependencies/gli/gli/format.hpp:6:0,
                 from dependencies/gli/gli/gli.hpp:39,
                 from src/image_io.cpp:25:
dependencies/gli/gli/type.hpp:42:24: error: ‘qualifier’ has not been declared
  template <typename T, qualifier P>

So this is the nested GLM. What about your Visual Studio Makefile are you using the external one ?

It may be a version from back when matrices were initialized with all zeros rather than an identity matrix by default.

Is this is just that that means you missed a -DGLM_FORCE_CTOR_INIT in your Linux Makefile.

You shouldn't be using that version of GLM. I'll go back and see if switching to the older GLM inside of GLI changes my numbers.

Compare with your VS Makefile.

Lecrapouille commented 5 years ago

Your VS makefile seems to use the external GLM:

      <AdditionalIncludeDirectories>jpeg-9a;glew-2.0.0\include;Targa;C:\Program Files %28x86%29\OpenAL 1.1 SDK\include;freealut-1.1.0-bin\include;libpng-1.2.20;zlib-1.2.1;tiff-4.0.3\libtiff;dependencies\glm;freeglut-2.8.1\include\GL;dependencies\gli;src</AdditionalIncludeDirectories>
      <PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;ENABLE_JPEG;ENABLE_PNG;ENABLE_TIFF;ENABLE_DDS;_CRT_SECURE_NO_DEPRECATE;%(PreprocessorDefinitions)</PreprocessorDefinitions>

With is similar of -I of gnuMakefile.

Lecrapouille commented 5 years ago

Ok I changed my Makefile to link with external GLM:

GLM=dependencies/glm
INCLUDES= ...  -I$(GLM)

But got the same error :(

Looking in GLM: dependencies/glm/glm/detail/type_mat4x4.inl they totally changed the code in comparison of the nested GLM. They apply the identity matrix while the holder code init nothing.

Lecrapouille commented 5 years ago

The error is generating NaN in the first place. What do you expect this to do: float f = 1.0; float v = sqrt(1.0 - f); I would think this would result in v=0 because 1.0 can be represented exactly in a floating-point number. But if the compiler does some strange optimization, maybe this can return NaN because the value going into sqrt() is some tiny negative number? Can you test this code on your compiler with different optimizations and print the results? If you do get NaN, does replacing the math with "v = sqrt(1.0f - f)" fix it?

Do you said that about: float const ni(sqrt(max(0.0, (1.0 - norm[i]*norm[i])))); ? sqrt(0.0) is 0.0 but sqrt(NaN) I dunno I guess it s NaN, I have to test. But now sqrt(max(0.0, NaN)) will be 0.0 because max(0.0, NaN) is 0.0. I think the problem is your norm[i] having bad values. I can check this.

fegennari commented 5 years ago

Can you try the latest version of GLM on GitHub? Maybe they fixed that qualifier error. Also, can you try adding -DGLM_FORCE_CTOR_INIT in the makefile? I think GLM is the root of the problem and disagreement between our builds. I'll go back later and initialize all of my GLM vectors/matrices to try and avoid this problem in the future. None of that code is really performance critical anyway.

For that NaN question, I believe the NaN is generated by calling sqrt() on a negative number. The value of norm[i] is probably close to 1.0. If the number is properly normalized, it should never be greater than 1.0, so 1.0 - norm[i]*norm[i] should never be negative. But if the compiler did some sort of tricky floating-point optimization, I can believe it somehow gets a negative number going into the sqrt() and NaN coming out.

I tried reproducing this on gcc 5.4 at work and I can't seem to get NaN.

Lecrapouille commented 5 years ago

Ok I'll try the new GLM. But I change my mind concerning norm[i] at NaN. In I just added in cylinder_3dw::calc_bcube a assert(std::isfinite(norm[i]) && "norme mal formee"); and even with -O2 now it passes. I think finally norms are ok but you still have memory overflow somewhere that inserting your code or my assert make change of location. I checked 1.0 - norm[i]*norm[i]

Lecrapouille commented 5 years ago

I'm not sure about the -DGLM_FORCE_CTOR_INIT seems they changed

Lecrapouille commented 5 years ago

I definitively hate GLM: dependencies/gli/gli/core/././bc.inl:81:66: error: ‘uint_t’ has not been declared Guys, what about continuous integration and writing unit tests instead of breaking APIs ? :-1:

fegennari commented 5 years ago

norm[i] isn't the problem. It's the bounding cube that's the problem. You should test those values, in the variable "d[3][2]" from base class cube_t.

I know GLM is a pain. The reason I'm using an old version is because I don't want to have to change my code like the last time I updated it. It tends to not be backwards compatible. If you can point me to an alternative to GLM that has vectors and matrix transforms that are compatible with GLSL I can try it.

Lecrapouille commented 5 years ago

You can try this really fast API: http://arma.sourceforge.net/ I used mine really slow and inspired by https://github.com/Reedbeta/reed-util and its blog http://www.reedbeta.com/blog/on-vector-math-libraries/ The default with Armadillo is you cannot overload operators (for using Max-Plus algebra for example, but I think you do not care)

Lecrapouille commented 5 years ago

norm[i] isn't the problem. It's the bounding cube that's the problem. You should test those values, in the variable "d[3][2]" from base class cube_t.

Sorry but your code is hard to understand for me (no comments). I'm not sure to follow you. Why d[3][2] while in calc_bcube d[i][0] and d[i][0] are used. Can you write me the code that I can test.

Lecrapouille commented 5 years ago

Ok I have to work on my project but yep on cylinder_3dw::calc_bcube, just adding the max fixed the bug. I dunno why, maybe because of sqrt(0.0-epsilon) but I could catch it with assert. But I still think about memory corruption made in ther part of the code because just adding assert() fix it.

Lecrapouille commented 5 years ago

I compiled with g++-8 (+external GLM) this may interesting you: compilation.txt shadow_map.cpp seems to make some memory overflow

fegennari commented 5 years ago

No the variable is d[3][2], a 2D C-style array. Adding the assert probably changed how the compiler optimized the code so that it wasn't able to do whatever optimization was giving a negative result going into the sqrt(). I've seen that before, and it makes sense.

fegennari commented 5 years ago
Lecrapouille commented 5 years ago

Seems alright!

Lecrapouille commented 5 years ago

I'm back on Debian, seems ok here too. And also the old crash when leaving the earth is fixed. I can definitively closed this ticket :) PS: in the README can be nice to give the name of the config as a caption of each screenshot. For example, I don't know which is the config name for the 1st screenshot.

fegennari commented 5 years ago

Thanks for testing this and providing all the feedback!

I'll see if I can figure out how to add captions in README.md. The first screenshot is from config_heightmap.txt. I started with just the heightmap with procedurally generated noise + domain warping + erosion, then added cities + buildings + cars + pedestrians on top of that. The car and pedestrian models are too large for the git repo so you won't see them.

Lecrapouille commented 5 years ago

ouch :( this config is a pain for my graphics card: 1FPS and the "nouveau" GPU driver seems to give hexa values. I had to kill 3dworld with the kill command. I tried again but this time all my Debian get frozen :( I cannot help you for this case. Reproduced twice. Here logs obtained with 3dworld > log1 2> log2 and interesting fact 3dworld seems to run at 20 FPS before get frozen (thread dead lock?) and my Debian died again. 3DWorld_logs.zip

fegennari commented 5 years ago

You need a graphics card with at least 2GB graphics memory for that scene and it's overflowing to system memory. It runs okay until it loads too many tiles. I have the same problem on some machines. You can even see this in your log: nouveau: kernel rejected pushbuf: Cannot allocate memory

The biggest problem is that I have the shadow map resolution set way too high. I really should implement CSMs rather than having one shadow map per nearby tile. Try changing this: shadow_map_sz 4096 to shadow_map_sz 1024 in config_heightmap.txt

There also may be too many trees. You can open config.txt (which is included in config_heightmap.txt) and change: max_unique_trees 100 to max_unique_trees 20

For foo1.log, Mesa 13.0.6 is very old, I think about 6 years old. I'm using Mesa 18.

Lecrapouille commented 5 years ago

Ok cool I always wanted to see what was happening when consuming all GPU memory :) The message error is not very explicit. For Mesa this is the problem when using Debian :( Ok I'll give a try with other settings.

PS: Thanks for the notice concerning Mesa, I gonna try to update it https://linuxconfig.org/how-to-install-the-latest-mesa-version-on-debian-9-stretch-linux while you exaggerate a little mesa 13 is only 2017 https://www.mesa3d.org/index.html (you scared me lol)

fegennari commented 5 years ago

If you enable DEBUG_TILES at the top of tiled_mesh.cpp you can see some count and memory usage printouts. For config_heightmap.txt, I see 66 tiles drawn of 341, 660 trees, and 33 shadow maps (of 4096x4096x4 = 64MB!) 2MB of memory for pine trees, 266MB for deciduous trees, 4MB for grass, and 2112MB for shadow maps! I didn't realize it was that high, and there's no reason to have 33 full resolution shadow maps. It should only need 4096x4096 shadow maps for the current tile. That's a total of 2491MB of GPU memory for just the tiles. Buildings, cars, pedestrians, textures, etc. take a few hundred MB more.

The easiest fix is to select a dynamic shadow map texture size based on the distance from the player to the tile. There's no reason to use full resolution shadow maps for distant tiles. This change reduces shadow map memory usage from 2112MB to 480MB - quite a reduction. I can't even tell the difference. I mapped a key to switch between full resolution and dynamic resolution shadow maps and it's hard to spot the occasional few pixels that differ. I can probably be even more aggressive with it, but I want to keep the highest quality shadows for my procedural city blog screenshots.

This is committed, but not well tested, and it needs more work. Adding CSMs is a better solution (though 10x the work). Is this enough to make it run on your machine with the default settings? I don't know.

Lecrapouille commented 5 years ago

Nice memory computations :) I guess tiles are the same idea than used for rendering maps (google, openstreet map) and closed to quadtrees. Do you use frustum culling for loading the necessary objects ? There are some methods used for avoiding shaders to compute on too many pixel/vertices. http://www.adriancourreges.com/projects/

Lecrapouille commented 5 years ago

Ok with your suggestions it works. Here again a crash report :) cras.txt

Lecrapouille commented 5 years ago

Here with valgrind (for this it was more difficult to produce this error, especially running at 1FPS :) valgrind_cars.txt

fegennari commented 5 years ago

For that assert, I'm missing a check for the pedestrian model being valid before drawing it. I had the check for cars but not pedestrians because I've never tested that case. It should be fixed now. I'll get back to you on the valgrind errors later.

For the tile question, 3DWorld's tiled terrain mode uses square tiles of 128x128 mesh vertices. Tiles are generated in a radius around the player as the player walks around the scene. Tiles are populated with vegetation and other objects as the player moves toward them. Most of it is distance based, rather than using the view frustum, to avoid having to do too much work if the player suddenly turns their view. Walking speed is limitation but camera rotations are not. Some of the faster work such as sending data to the GPU is done based on the view frustum, and the actual rendering is certainly done with view frustum culling (and occlusion culling with terrain and buildings).

Lecrapouille commented 5 years ago

Seems to work

fegennari commented 5 years ago

I made some changes:

Lecrapouille commented 5 years ago

Yep Valgrind first errors are made by the driver, I have this similar on my personal projects too. Others are fixed but I still failed in the same assertion while the commit of yesterday was ok for me. pedestr.txt

fegennari commented 5 years ago

Sigh. Now it's failing for cars, in the shadow pass. It's missing the special case check for when none of the car models are valid. I fixed that in the pedestrians case but never tested it for no car models. I only tested it for some of the car models being invalid, and some/all pedestrian models being invalid. I'll fix it tonight.