Geodan / i3dm.export

Export 3D Instanced Tiles from PostGIS table
MIT License
41 stars 21 forks source link

Export issue in I3dm.export #35

Closed ashitoshp closed 1 year ago

ashitoshp commented 1 year ago

I have imported your Sample to PostGIS and tried the command as below i3dm.export -c "Host=172.16.0.213;Username=postgres;Password=postgres;Database=3Dmodels;Port=5432" -t i3dm.mydata

Getting System.ArgumentOutOfRangeException error

Stacktrace as below

Tool: I3dm.export Version: 2.4.1.0 Exporting i3dm's from i3dm.mydata... Bounding box for table (WGS84): 4.6981, 52.4227, 4.722, 52.4823 Heights for bounding volume: [0 m, 50 m] Maximum instances per tile: 1000 Creating tile: ./output/content/0_0_0.cmpt Writing 1 subtree files... Unhandled exception. System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'index') at System.Linq.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argument) at System.Linq.Enumerable.ElementAt[TSource](IEnumerable1 source, Int32 index) at i3dm.export.Program.<>c.<Main>b__0_1(Options o) in D:\dev\github.com\geodan\i3dm.export\src\Program.cs:line 82 at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult1 result, Action`1 action) at i3dm.export.Program.Main(String[] args) in D:\dev\github.com\geodan\i3dm.export\src\Program.cs:line 25

bertt commented 1 year ago

It seems something goes wrong when writing the tiles. Did you follow the getting started? https://github.com/Geodan/i3dm.export/blob/main/docs/getting_started.md

One thing I noticed the coordinates are different:

Yours: Bounding box for table (WGS84): 4.6981, 52.4227, 4.722, 52.4823

Sample: Bounding box for table (WGS84): 4.7591, 52.2785, 4.9999, 52.4301

sweco-sekrsv commented 1 year ago

I get the same error. I used this sample to setup the databse: https://github.com/Geodan/i3dm.export/blob/main/testdata/create_testdata.sql

And then I use this box renamed to tree.glb: https://github.com/Geodan/i3dm.export/blob/main/docs/Box.glb

This is the output: C:\work\3d_tiles\olp_output\i3dm>i3dm.export -c "Host=localhost;Username=postgres;Password=xxxx;Database=test_gis;Port=5432" -t i3dm.mydata -f cesium Tool: I3dm.export Version: 2.4.1.0 Exporting i3dm's from i3dm.mydata... Bounding box for table (WGS84): 4,6981, 52,4227, 4,722, 52,4823 Heights for bounding volume: [0 m, 50 m] Maximum instances per tile: 1000 Creating tile: ./output/content/0_0_0.cmpt Writing 1 subtree files... Unhandled exception. System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'index') at System.Linq.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argument) at System.Linq.Enumerable.ElementAt[TSource](IEnumerable1 source, Int32 index) at i3dm.export.Program.<>c.<Main>b__0_1(Options o) in D:\dev\github.com\geodan\i3dm.export\src\Program.cs:line 82 at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult1 result, Action`1 action) at i3dm.export.Program.Main(String[] args) in D:\dev\github.com\geodan\i3dm.export\src\Program.cs:line 25

bertt commented 1 year ago

if you follow the getting started document there is an input table of 60K features (it should work), when using the sample database setup script (https://github.com/Geodan/i3dm.export/blob/main/testdata/create_testdata.sql) there is something going wrong

ashitoshp commented 1 year ago

It seems something goes wrong when writing the tiles. Did you follow the getting started? https://github.com/Geodan/i3dm.export/blob/main/docs/getting_started.md

One thing I noticed the coordinates are different:

Yours: Bounding box for table (WGS84): 4.6981, 52.4227, 4.722, 52.4823

Sample: Bounding box for table (WGS84): 4.7591, 52.2785, 4.9999, 52.4301

Thanks for your response. I was able to run your example (mentioned in get started page). However, if I try this on my data, it fails with exact same same issue. The only difference I see in the table is that I have Z values and are negative. Projection is 4326. Any ideas?

bertt commented 1 year ago

the issue occurs when very few cmpt tiles are generated (like 1 or 5). When working with a small dataset you can decrease max_features_per_tile (default 1000) so more tiles are generated. This issue will be fixed in next release.

sweco-sekrsv commented 1 year ago

The getting started works for me as well. However I needed to add this to be able to get it to work: -t_srs EPSG:28992 like this: $ ogr2ogr -t_srs EPSG:28992 -f "PostgreSQL" PG:"host=localhost user=postgres password=postgres dbname=postgres" current_traffic_signs.geojson -nlt POINT -nln traffic_signs

Sounds great that this will be fixed in an upcoming release!

ashitoshp commented 1 year ago

the issue occurs when very few cmpt tiles are generated (like 1 or 5). When working with a small dataset you can decrease max_features_per_tile (default 1000) so more tiles are generated. This issue will be fixed in next release.

Thanks. Got the code to work. The output however is not creating proper json.
While loading in cesium get this error "<!DOCTYPE "... is not valid JSON Cesium.js:8114 A 3D tile failed to load: http://localhost/cesiumtest/data/content/4_7_8.cmpt Cesium.js:8114 Error: Failed to load i3dm Failed to load glTF Failed to load glTF: http://localhost/cesiumtest/data/content/4_7_8.cmpt?compositeIndex=0 Unexpected token '<', " " Attached one of cmpt files for your reference.

file 4_7_8.zip

bertt commented 1 year ago

@ashitoshp somehow a ?github? webpage ended up in the 4_7_8.cmpt file... Maybe try again

bertt commented 1 year ago

@sweco-sekrsv that should not be necessary when following the getting started... what error do you get?

sweco-sekrsv commented 1 year ago

@sweco-sekrsv that should not be necessary when following the getting started... what error do you get?

I got this error: ERROR: ST_Transform: Input geometry has unknown (0) SRID

bertt commented 1 year ago

@sweco-sekrsv that should not be necessary when following the getting started... what error do you get?

I got this error: ERROR: ST_Transform: Input geometry has unknown (0) SRID

You mean in this sql query? https://github.com/Geodan/i3dm.export/blob/main/docs/getting_started.md#create-instances-table. Cannot reproduce that.

ashitoshp commented 1 year ago

@ashitoshp somehow a ?github? webpage ended up in the 4_7_8.cmpt file... Maybe try again

Yes tats my bad! Im able to convert to i3dm now. Just wanted to confirm that the Z value is also taken into consideration while conversion to i3dm. Will check the results and et back if any issues anyways. Thanks for your support.

bertt commented 1 year ago

Just wanted to confirm that the Z value is also taken into consideration while conversion to i3dm.

yes should be

bertt commented 1 year ago

issue is fixed in 2.4.3

bertt commented 1 month ago

@qwork2010 Link works for me https://raw.githubusercontent.com/Amsterdam/mlvb/master/output/asset-registration/current_traffic_signs.geojson

bertt commented 3 weeks ago

I see column 'geom' is null. Is PostGIS installed?

Check with

select postgis_full_version()

bertt commented 3 weeks ago

How did you import the data? Using ogr2ogr?

https://github.com/Geodan/i3dm.export/blob/main/docs/getting_started.md#import-traffic-signs-to-postgis

bertt commented 3 weeks ago

Ok, one thing to check: is column geom in table taffic_sign_instances all null? If so check the step for creating this table https://github.com/Geodan/i3dm.export/blob/main/docs/getting_started.md#create-instances-table

Another thing to check: the database name is 'postgis_34_sample', but you didn't specify it in the i3dm.export command (parameter dbname is missing)

bertt commented 3 weeks ago

Also I see there is an error when running ogr2ogr:

ERROR 1: PROJ: proj_create_from_database: SQLite error on SELECT name, coordinate_system_auth_name, coordinate_system_co de, geodetic_crs_auth_name, geodetic_crs_code, conversion_auth_name, conversion_code, area_of_use_auth_name, area_of_use _code, text_definition, deprecated FROM projected_crs WHERE auth_name = ? AND code = ?: no such column: area_of_use_auth
name

This looks like not compatible versions. I'm using QGIS 3.38, ogr2ogr --version GDAL 3.9.2, PostgreSQL 16.0, PostGIS version 3.4

bertt commented 3 weeks ago

@qwork2010 ok great its working now. Rotation is horizontal limited indeed when creating 3D Tiles 1.0 I3dm/cmpt. When creating 3D Tiles 1.1 glb's using GPU instancing (by adding --use_gpu_instancing true, see https://github.com/Geodan/i3dm.export?tab=readme-ov-file#gpu-instancing) you can define columns 'roll', 'pitch' and 'yaw' (so column 'rotation' is not used)

bertt commented 3 weeks ago

This GPU instance method is in development, not yet much documentation. But the process is the same, you have to fill the rotations in columns 'roll', 'pitch' and 'yaw' (in radians) and add the --use_gpu_instancing true option.

qwork2010 commented 3 weeks ago

This GPU instance method is in development, not yet much documentation. But the process is the same, you have to fill the rotations in columns 'roll', 'pitch' and 'yaw' (in radians) and add the --use_gpu_instancing true option.

@bertt

Thank you very much! I use the command line "i3dm.export-c" Host=localhost;Username=postgres;password=123456;Port=5432 "- t traffic_signs_instances" to run "i3dm.export.exe" software. Excuse me, "add the-use_gpu_instancing true option", where can I set this up? Dear friends, can you give me a specific example to imitate?

bertt commented 3 weeks ago
i3dm.export-c "Host=localhost;Username=postgres;password=123456;Port=5432"- t traffic_signs_instances --use_gpu_instancing true
bertt commented 3 weeks ago

So with the classic method, there is only yaw rotation (in the rotation column)

With the 3D Tiles 1.1 method (using --use_gpu_instancing true) there are the 3 axis rotation possible

image

bertt commented 3 weeks ago

try azimuth in yaw column, elevation in roll column

bertt commented 6 days ago

One thing I notice the rotation angles should be in radians not in degrees.

Maybe the instances are below the terrain? You can test by switching the terrain off in Cesium.

If no luck attach the table (export as geopackage in qgis/gdal) and the used model, I can take a look

bertt commented 5 days ago

Hi, I did a quick test with the data.

Created view:

CREATE view hl2_i5 as (
    SELECT ogc_fid as id, 
    geom,
    scale as scale,
    0 as rotation,
    0 as yaw,
    0 as roll,
    0 as pitch,
    'Guardrail.gltf' as model,
    json_build_array(json_build_object('id',ogc_fid), json_build_object('name',name)) as tags
    from hl2
);

Created Intanced 3D Tiles with 1.0 method (so cmpt files are created):

i3dm.export -c "Host=localhost;Username=postgres;Password=postgres;Database=postgres;Port=5432" -t hl2_i5  -o output --use_gpu_instancing false

Result in CesiumJS:

image

Conclusion: Guardrails are drawn, only the horizontal rotation are wrong. With 3D Tiles 1.0 Instanced tiles only the column 'rotation' is used for horizontal rotations. For example with rotation = 90:

image

So as I see it, there is no need to use --use_gpu_instancing true because only a horizontal rotation is needed. The values of the rotation column should be calculated per point based on the direction of the road and relation to the road (left or right).

bertt commented 5 days ago

And a test with --use_gpu_instancing true (same view) - glb's are created:

image

In this case only the yaw column should get new values (in radians).

Debugging tip: set 'tileset.debugShowBoundingVolume = true;' to find the guardrails in the map.

bertt commented 5 days ago

I don't understand the question. I've used model 'Guardrail.gltf', and data from hl2.gpkg to create instance 3D Tiles. The file hl2_i5.gpkg is empty somehow. In folder 'outputhl2i5' there is a tileset with 3D Tiles 1.1, but I don't have information how this was created.

bertt commented 5 days ago

Can you load and display the models in my 'output hl2i5' folder?

Yeah they load fine but are rotated on 3 axis:

image

when you use zero for angles (see the view in https://github.com/Geodan/i3dm.export/issues/35#issuecomment-2443913385) there is no rotation (only horizontal angle should be adjusted).

Yes ''tileset.debugShowBoundingVolume = true;' is a setting in CesiumJS

bertt commented 4 days ago

Hi,

Option tileset_version has nothing to do with this, it's some metadata in tileset.json for setting a version (v10-test for example)

About the check: In 2.9 there is an error when you use both:

--use_gpu_instancing true (meaning the output is glb files instead of cmpt/i3dm) and

--use_i3dm true (meaning output is I3dm not cmpt)

bertt commented 4 days ago

Due to the large size of the file, we created a repository to share compressed files. Please download 'project. zip' and unzip it.

How can I download? Get some errors (git lfs is installed).

$ git clone https://github.com/qwork2010/hulanproject.git
Error downloading object: project.zip (1d11958): Smudge error: Error downloading project.zip (1d11958e903973f62b7bf8b07647024d0b60e57a5ca7c72911f6cd37656ccd7a): [1d11958e903973f62b7bf8b07647024d0b60e57a5ca7c72911f6cd37656ccd7a] Object does not exist on the server: [404] Object does not exist on the server
$ cd hulanproject
$ git lfs fetch
fetch: Fetching reference refs/heads/master
[1d11958e903973f62b7bf8b07647024d0b60e57a5ca7c72911f6cd37656ccd7a] Object does not exist on the server: [404] Object does not exist on the server
error: failed to fetch some objects from 'https://github.com/qwork2010/hulanproject.git/info/lfs'
bertt commented 4 days ago

Download zip gives an empty project.zip file :-( Maybe use https://wetransfer.com/ or so?)

bertt commented 4 days ago

btemme(@)hotmail.com

bertt commented 3 days ago

Inspected the project, the 3D tiles in folder 'outputhl2i5' do visualize well in CesiumJS 1.118.2 (rotated but that's expected):

image

In the project 'hulan-project' nothing is visualized (but the glb's and subtree files are requested).

I think it's related to the Cesium version used (1.99) - its quite old (current version is 1.122). Can you upgrade (maybe also dependency vite-plugin-cesium) and try again?

qwork2010 commented 2 days ago

So with the classic method, there is only yaw rotation (in the rotation column)

With the 3D Tiles 1.1 method (using --use_gpu_instancing true) there are the 3 axis rotation possible

image

@bertt Hello! Thank you very much for your help. I am now facing the last problem, which is about the rotation of (yaw, roll, pitch). For example, I have model A, and the desired result is to rotate model A by yaw=Π/6 to obtain model Ay, and then rotate model Ay by roll=Π/6 to obtain model Ayr with the correct orientation. That is, the desired result is to rotate first yaw and then roll in turn. However, when using "i3dm.export.exe", by filling in the parameters (yaw=Π/6, roll=Π/6, pitch=0) to rotate the model A, we obtain a model A1, whose direction is not that of Ayr, which means A1≠Ayr. I don't know if my problem description is clear? It is a sequential combination of rotation around two axes. When using "i3dm.export.exe" (yaw=?, roll=?, pitch=?)? Thank you again for your help!

bertt commented 2 days ago

what tool did you use create the desired result?