aurora-opensource / streetscape.gl

Visualization framework for autonomy and robotics data encoded in XVIZ
http://www.streetscape.gl
MIT License
914 stars 222 forks source link

WebGL errors when loading my own mesh #280

Open Algirdyz opened 5 years ago

Algirdyz commented 5 years ago

I have tried loading my own mesh using the util from KITTI example. It loads fine, but I am getting errors afterwards, when things try to render. Screenshot from 2019-03-15 16-13-47

It works fine with the provided car.obj. But I've tried several models of my own and it didn't work.

Is there a specific format we need to use? I've been comparing objs, but can't really pinpoint an issue. My files work fine in https://3dviewer.net/ for example.

ibgreen commented 5 years ago

We use the loaders.gl library for loading meshes. We do have OBJ test files in that repo that load without problems, so your file is probably using some feature in the OBJ format that we haven't tested.

For hopefully obvious reasons we can't commit to supporting all variations of all formats, however if you can share the model (or any model that fails in a similar way) we will do a quick debug pass to see if there is an easy fix or work-around (there likely is, OBJ is a rather simple format).

As a wild guess from the error message it sounds like one of the attributes doesn't have enough vertices - perhaps parsing of part of the file didn't complete?

Algirdyz commented 5 years ago

What is the official way of loading meshes using loaders.gl then? I couldn't really find what exactly is supported in the documentation.

But anyway, here is my model: boat.txt

I changed it to txt because github didn't allow uploading otherwise.

Thanks!

ibgreen commented 5 years ago

I tried your model in the loaders.gl render tests, and it did render, no webgl errors.

The scale was big compared to our reference model so I had to zoom out a bit to see it. The issue might be in the MeshLayer, this layer is being improved for deck.gl v7, next step is to test your model in that layer. @xintongxia

Our focus in the upcoming v7 release of deck.gl is to provide really good support for glTF models, so if you have the ability to convert from OBJ to glTF that could come in handy, as we definitely will want support issues with glTF loading.

gerardsimons commented 5 years ago

Ah so the trick is to use the loaders.gl? Because it looks like the examples in streetscape.gl do not actually use these to load OBJ meshes?

https://github.com/uber/streetscape.gl/blob/master/examples/website-demo/src/utils/load-obj-mesh.js

gerardsimons commented 5 years ago

Ah it seems that the loaders.gl is a wrapper around that webgl-obj-loader. We are trying the gltf one now. What library would you advise to load that into? Any other examples out there that could help us?

ibgreen commented 5 years ago

@gerardsimons

Ah so the trick is to use the loaders.gl? Because it looks like the examples in streetscape.gl do not actually use these to load OBJ meshes?

@xintongxia @twojtasz @Pessimistress Should we make an attempt to update the OBJ loader in streetscape.gl to use loaders.gl? That should make it easier to support issues with loaders.

We are trying the gltf one now. What library would you advise to load that into? Any other examples out there that could help us?

Re glTF: My comment was more of a "heads up", since at this moment, we are finalizing v7 of luma.gl and deck.gl with extensive glTF support. But unless you are willing to deal with the (considerable) pain of using alpha releases, you will want to wait just a little more. We will probably cut the first beta release early next week so if you really wanted to do some pilot testing and give us early feedback that could be a good time.

Among other things there will be a new ScenegraphLayer in deck.gl that will be able to display glTF files. v7 will launch officially sometime in April.

gerardsimons commented 5 years ago

Alright, great, thanks for getting back to us so soon. We really appreciate it. Unfortunately are on a bit of tight deadline here and were hoping to get something up and running sooner. The mesh is the last important piece of the puzzle. So I guess the loaders.gl with the OBJ would be the way to go. Just out of curiosity what tests / code did you run on our model? I could perhaps piece out from that on how to get it into streetscape.gl then.

ibgreen commented 5 years ago

Here is the diff on loaders.gl: https://github.com/uber-web/loaders.gl/compare/master...ib/obj-test

Will delete your model and this branch later on.

gerardsimons commented 5 years ago

Thank you for that @ibgreen! It does look like all goes well there. At a loss where it goes wrong then. Will try and poke at it some more to see where it exactly goes wrong then. If anyone else is willing to chime in on what format or loader to use so it would render in streetscape I would be very happy.

gerardsimons commented 5 years ago

I fixed it! Turns out it was a texture coordinate issue. My model did not have any, but the car did. I did not want to add them though so I change the code in load-obj-mesh.js to

export default function loadOBJMesh(url) {
  return fetch(url)
    .then(resp => resp.text())
    .then(text => {
      const mesh = new Mesh(text);
      const indices = new Uint16Array(mesh.indices);
      const positions = new Float32Array(mesh.vertices);
      const normals = new Float32Array(mesh.vertexNormals);

      if(mesh.textures.length > 0) { // Do not return any texture coordinates at all if there aren't any loaded
        const texCoords = new Float32Array(mesh.textures)
        return {indices, positions, normals, texCoords};
      }
      else {
        return {indices, positions, normals};
      }
    });
}
ibgreen commented 5 years ago

@gerardsimons Congrats and nice finding!

@xintongxia The fact that loaders.gl already handles this (and at least one mode) issue seems like a good reason for streetscape.gl to upgrade to use the OBJLoader from loaders.gl.

tngan commented 5 years ago

@gerardsimons Awesome, it works for me as well.

EliotCao commented 4 years ago

I have downloaded a .obj model and useed it for the ego car in website demo, but it didn't work. and I have test others .obj file, so it. The .obj file is here bugatti