NASA-AMMOS / 3DTilesRendererJS

Renderer for 3D Tiles in Javascript using three.js
https://nasa-ammos.github.io/3DTilesRendererJS/example/bundle/mars.html
Apache License 2.0
1.47k stars 266 forks source link

Traverse the information in the batchtable directly through tileset.json #524

Closed jingyangking closed 2 months ago

jingyangking commented 2 months ago

Hello developer, can we simply load tileset. json to traverse the attribute information in tiles? I noticed that in your example, b3mm was used, which requires the additional use of B3DMLoad() method in the scene. If it is possible to traverse the information in the batchtable solely through tileset.json, I think it would be much more convenient. Is there a feasible method? Thank you!

gkjohnson commented 2 months ago

Hello! It's not clear to me exactly what you're trying to do. You want to load a tileset and just read the batchtable data?

The batch table contents are embedded in the B3DM file so in order to read that data it's required to load and parse the B3DM file.

jingyangking commented 2 months ago

Sorry, I didn't describe it very clearly. What I mean is that by directly loading tilesets.json, we can know the specific path of each b3mm and i3mm. Can we use this to implicitly obtain the attribute information contained in each b3mm and i3mm? As you gave the example, the mouse scan can see the corresponding content, but it needs to load the relevant b3dm and i3dm one by one. If the number of such b3dm and i3dm is very large, will it be too laggy to load? If we could directly use tileset.json to obtain the attribute information of each b3dm and i3dm, it would avoid the step of loading these b3dms and i3dms. I think it would be a good improvement in loading speed. image

gkjohnson commented 2 months ago

If we could directly use tileset.json to obtain the attribute information of each b3dm and i3dm, it would avoid the step of loading these b3dms and i3dms. I think it would be a good improvement in loading speed.

Because that BatchTable attribute information is embedded in the B3DM and I3DM files it's not possible to read this data without downloading and parsing those files.

will it be too laggy to load

This depends on your use case, how complex your tileset is, and how many tiles need to be downloaded.

If we could directly use tileset.json to obtain the attribute information of each b3dm and i3dm, it would avoid the step of loading these b3dms and i3dms. I think it would be a good improvement in loading speed.

Unfortunately this isn't how it works in the specification.

What is your use case more specifically? You would like to inspect the tile metadata without loading or displaying the 3d geometry?

jingyangking commented 2 months ago

If we could directly use tileset.json to obtain the attribute information of each b3dm and i3dm, it would avoid the step of loading these b3dms and i3dms. I think it would be a good improvement in loading speed.

Because that BatchTable attribute information is embedded in the B3DM and I3DM files it's not possible to read this data without downloading and parsing those files.

will it be too laggy to load

This depends on your use case, how complex your tileset is, and how many tiles need to be downloaded.

If we could directly use tileset.json to obtain the attribute information of each b3dm and i3dm, it would avoid the step of loading these b3dms and i3dms. I think it would be a good improvement in loading speed.

Unfortunately this isn't how it works in the specification.

What is your use case more specifically? You would like to inspect the tile metadata without loading or displaying the 3d geometry?

As shown in the following code, I want to obtain information about each b3dm in tileset.json, but I cannot obtain it image image If it's a problem with my code, I hope you can correct it. Thank you

gkjohnson commented 2 months ago

I see in your screenshot that there is a batchTable field on group.children[ 0 ].batchTable. Is this not what you're looking for?

Maybe it would be easier for you make an example showing the code you want to write to access the batchTable? A working example showing what's not possible would be helpful, as well. Sorry I'm having difficulty understanding.

jingyangking commented 2 months ago

The result I obtained from following your prompts is unbelievable, as shown in the following picture image image When I released console. log (group [0]. batchTable), the result actually reported an error image image

gkjohnson commented 2 months ago

Something isn't lining up here. It doesn't make sense for the array length to be 2 with inspectable elements but the first element returns undefined.

Can you please provide a simple demo with codesandbox or a comparable system? And without any UI frameworks like Vue?

jingyangking commented 2 months ago

Something isn't lining up here. It doesn't make sense for the array length to be 2 with inspectable elements but the first element returns undefined.

Can you please provide a simple demo with codesandbox or a comparable system? And without any UI frameworks like Vue?

This is the code I used to operate on Codesandbox and the tiles file for testing. Since I have not used this tool before, I only asked ChatGPT to convert my code in Vue and copy it here. Although there may be some errors, they are all easy to solve small problems. I hope they can meet your needs. Thank you! Batchedbarrel.zip 03e80bb6-e6cf-422f-bf65-5b9720a3f1eb.zip

gkjohnson commented 2 months ago

Although there may be some errors, they are all easy to solve small problems. I hope they can meet your needs.

I need a working example if I'm going to help. Unfortunately I don't have the bandwidth to figure out how to fix and run a zip of code.

3d-tiles-renderer@0.1.1

I see v0.1.1 is being imported. Is this the version of the project you're using? This version is over 4 years old. Please use the latest version of the package.

jingyangking commented 2 months ago

I'm sorry, I currently don't have a better level to provide a good example. I used the latest installation package for the 3D tile renderer, so this error is not related to the version. As for why I am unable to obtain the data in the batch table, I am still not quite sure

gkjohnson commented 2 months ago

Unfortunately I can't help without example code showing the the issue. I have loaded the tileset myself in the example files and am able load the batch table data:

image
jingyangking commented 2 months ago

Unfortunately I can't help without example code showing the the issue. I have loaded the tileset myself in the example files and am able load the batch table data:

image

Can you show me your code? Thank you!

gkjohnson commented 2 months ago

Here's a snippet based on the "mars" example file showing the tileset being loaded and the batch table being logged:

code snippet ```js import { TilesRenderer } from '../src/index.js'; import { Scene, DirectionalLight, AmbientLight, WebGLRenderer, PerspectiveCamera, Group, Box3, } from 'three'; import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls.js'; import { GUI } from 'three/examples/jsm/libs/lil-gui.module.min.js'; let camera, controls, scene, renderer; let tiles; const params = { errorTarget: 12 }; init(); render(); function init() { scene = new Scene(); // primary camera view renderer = new WebGLRenderer( { antialias: true } ); renderer.setPixelRatio( window.devicePixelRatio ); renderer.setSize( window.innerWidth, window.innerHeight ); renderer.setClearColor( 0xd8cec0 ); document.body.appendChild( renderer.domElement ); renderer.domElement.tabIndex = 1; camera = new PerspectiveCamera( 60, window.innerWidth / window.innerHeight, 1, 4000 ); camera.position.set( 20, 10, 20 ); // controls controls = new OrbitControls( camera, renderer.domElement ); // lights const dirLight = new DirectionalLight( 0xffffff ); dirLight.position.set( 1, 2, 3 ); scene.add( dirLight ); const ambLight = new AmbientLight( 0xffffff, 0.2 ); scene.add( ambLight ); // tiles tiles = new TilesRenderer( '../__data/tileset.json' ); scene.add( tiles.group ); onWindowResize(); window.addEventListener( 'resize', onWindowResize, false ); const gui = new GUI(); gui.add( params, 'errorTarget', 0, 100 ); gui.open(); } function onWindowResize() { camera.aspect = window.innerWidth / window.innerHeight; camera.updateProjectionMatrix(); renderer.setSize( window.innerWidth, window.innerHeight ); renderer.setPixelRatio( window.devicePixelRatio ); } function render() { requestAnimationFrame( render ); camera.updateMatrixWorld(); // center the tileset const box = new Box3(); if ( tiles.getBoundingBox( box ) ) { box.getCenter( tiles.group.position ).multiplyScalar( - 1 ); } // log the batch table if ( tiles.group.children[ 0 ] ) { console.log( tiles.group.children[ 0 ].batchTable ); } tiles.setCamera( camera ); tiles.setResolutionFromRenderer( camera, renderer ); tiles.update(); renderer.render( scene, camera ); } ```
jingyangking commented 2 months ago

Thank you for your reply!I think I have found the source of the problem. The information of batchtable is displayed on the console only when the relevant statements are written in render(). If it is written outside of the render() function, it will not be displayed. However, I am not sure why this is happening. Is it because the tiles are constantly updated?

gkjohnson commented 2 months ago

The information of batchtable is displayed on the console only when the relevant statements are written in render(). If it is written outside of the render() function, it will not be displayed. However, I am not sure why this is happening. Is it because the tiles are constantly updated?

The TilesRenderer.update() function must be called every frame in order for the next level of detail tiles to be loaded. It also takes time for tiles and the root tileset json to download so the data will not be available right away. Data download does not begin until update is called initially.

jingyangking commented 2 months ago

That's it, I understand now. Thank you for your answer. Now I have another question, which is how to load tiles through multithreading? Developer, do you have any good ideas? Looking forward to your reply!

gkjohnson commented 2 months ago

which is how to load tiles through multithreading? Developer, do you have any good ideas?

I'm not sure what you mean. What improvements are you expecting? The browser already parallelizes downloads and parsing the tile geometry is not intensive for the most part.

jingyangking commented 2 months ago

Hello developer! My point is not just to rely on the browser for parallel loading, but to use code to double match with the browser. If there are a large number of tiles to load, browser loading alone is not enough. Therefore, multi-threaded loading of them in the code in advance can alleviate some pressure on the browser. Therefore, I want to figure out how to perform multi-threaded operations in the code. I have started trying it out, but because the subthreading requires the tilesRenderer. setCamera (camera); TilesRenderer. setResolutionFromRenderer (camera, renderer) These two sentences of code are processed, and 'camera, renderer' are in the main thread. I am not sure how to pass them to the child thread, so I encountered difficulties. I hope my reply can help you understand the meaning I want to express. Thank you!

gkjohnson commented 2 months ago

to use code to double match with the browser. If there are a large number of tiles to load, browser loading alone is not enough. Therefore, multi-threaded loading of them in the code in advance can alleviate some pressure on the browser.

I'm not sure if I understand this. What specifically is a bottleneck for you and what timing are you trying to reduce? You need to explain your exact use case. These descriptions are too vague to help.

subthreading requires the tilesRenderer. setCamera (camera); TilesRenderer. setResolutionFromRenderer (camera, renderer) These two sentences of code are processed, and 'camera, renderer' are in the main thread.

Then you'll have to recreate them in the Web Worker and / or use one of the resolutions function setting that doesn't rely on the WebGLRenderer.

Ultimately this project is for rendering 3d tiles, though, so running the tiles loader in a separate worker from the the renderer isn't a supported use case.

jingyangking commented 2 months ago

I'm sorry, I couldn't describe the meaning I wanted to convey. The bottleneck I am currently facing is that when loading multiple tiles. jsons at the same time, the loading speed of the webpage will slow down. How can I increase the loading speed? Reduce the time to load tiles files? My idea is to reduce the corresponding time through multithreading, but I am not sure how to construct multithreading. If you create a camera/renderer in webworker, it means initializing the scene in the child thread. This way, the main thread seems to have nothing to do. Is this currently the only method available?

gkjohnson commented 2 months ago

How can I increase the loading speed? Reduce the time to load tiles files? My idea is to reduce the corresponding time through multithreading

As I mention above the browser will already parallelize the requests. I don't expect a WebWorker to help this unless it has been shown otherwise. Unfortunately multithreading is not a silver bullet for improving performance, especially in Javascript.

There are a lot of factors in download times including internet speed, file size, server side compression, usage of the http2 protocol, etc. But these are not specific for 3d tiles. I recommend investigating these topics first if you need to improve download times.

You can also tweak the errorTarget to load fewer tiles and the maxJobs field for the download and parse queues to allow for processing more data at once but it depends on what you need. But without more statistics on your tileset and where it's slow it's not possible to recommend anything specific.

jingyangking commented 2 months ago

Thank you for your reply! Thank you for providing me with several ways to improve loading speed, which has given me new ideas. Meanwhile, I have another issue, which is that when using the ray casting method you provided for object selection, there is a situation where an object is selected but the entire corresponding component is selected. Why is this happening? image image

gkjohnson commented 2 months ago

ray casting method you provided for object selection, there is a situation where an object is selected but the entire corresponding component is selected. Why is this happening?

Highlighting individual objects based on batch id is not supported out of the box. It requires a custom shader. I recommend taking a look at the b3dm example that demonstrates a custom batch id highlight material and the custom material example for how to apply it to the tileset.

jingyangking commented 2 months ago

Thank you for your reply! The shader looks too difficult, and I think it will take me a long time to understand it, but in order to achieve the goal, I think I will continue to do it.

jingyangking commented 2 months ago

Thanks to the example you provided, I am now free to click on any object and highlight it. image However, at the same time, I have a new question. When I import certain tiles, their position and orientation will change to varying degrees. Every time I modify their position and orientation from the scene, some sub objects will be misaligned. How can I solve this problem? Should we consider tileset.json or make modifications in the scene? Thank you. image

gkjohnson commented 2 months ago

I need to ask for functioning examples to offer any more advice. It's not valid to change the transform of one tile manually. But if you're changing the position and rotation of a tile then it should be expected that the transformation is different.

jingyangking commented 2 months ago

Okay, let me provide an example: this is the situation where I directly loaded tiles without making any modifications: image you can see that the position of the building is vertical.Here is a demonstration of my model after making positional changes in the code: image image You can see that the positions of some subcomponents have been misplaced

gkjohnson commented 2 months ago

The tile positions are set correctly on load so you cannot modify their transforms. If you need to reorient the tileset you'll need to adjust the tileset root transform.

jingyangking commented 2 months ago

Is there a way to automatically adjust the position of the root of the block set? It would be troublesome to manually adjust one by one.

gkjohnson commented 2 months ago

one by one.

How many tile sets are you loading?

jingyangking commented 2 months ago

300+

gkjohnson commented 2 months ago

You can modify each tileset root transform on creation or add them to a common parent group as you can any other three.js Object3D.

jingyangking commented 2 months ago

I'd want to know if adding to a common parent group solves this misalignment, and if I have five different sets of tiles, adding them to a common parent group at the same time can also directly avoid this misalignment?

gkjohnson commented 2 months ago

I'd want to know if adding to a common parent group solves this misalignment

Have you tried it? I'm happy to help but unfortunately I have limited free time to answer questions and I'd appreciate it if you take a look at the README and try some of these things out. Similarly providing example code helps keep me from spending time guessing at what's happening and what solutions might be.

jingyangking commented 2 months ago

Okay, I will try it out. Thank you for your reply!

jingyangking commented 2 months ago

I have found a method to modify the rotation position of tiles, which is very simple. Just add a set of matrices to the root node of the tile set, as shown in the following figure: image image This way, the position, rotation, and size of the entire tiles in the scene can be modified by modifying the added matrix, and there will be no problem of some sub component positions being misaligned. I just don't quite understand why this can achieve the operation of the entire tile. I hope you can provide an explanation, thank you!

gkjohnson commented 2 months ago

I just don't quite understand why this can achieve the operation of the entire tile. I hope you can provide an explanation, thank you!

You should modify the TilesRenderer.group object. There's no need to modify the file contents. I recommend reading up on a scene graph and learning more about three.js' matrix transformations if this isn't familiar. The three.js forums can also be a good place for questions general three.js transformations.

jingyangking commented 2 months ago

Thank you for your reply. I need to study this aspect more carefully, although it may seem like it will take a lot of time.

gkjohnson commented 2 months ago

Yeah I'd say the three.js and scene graph basics are a good place to start if you're unfamiliar and need adjust the tile transformations.