3dcitydb / importer-exporter

3D City Database client for high-performance import and export of 3D city model data
Apache License 2.0
126 stars 53 forks source link

failed to solve Xlinks reference #201

Closed tfqrsli closed 2 years ago

tfqrsli commented 3 years ago

Hi,

I am tfq and i'm new here. Recently i have an issue which is to import gml (LoD4) file to database using 3dcitydb importer/exporter and the console log show error failed to solve Xlinks reference even pass validation before import. What the possibility to get the error and how to solve.

For the information, i am new for this field and i am using sketchup to modeling the building and plugin sketchup (cityeditor) to assign and convert to gml file type.

I have attaching a picture of console log

I am very happy if someone can help me to guide or solve this problem since already cross my due date to submit.

Regards tfq

Capture

clausnagel commented 3 years ago

Somewhere in your file, there are xlink:href attributes pointing to gml:id values that don't exist. To check this, pick one of the error messages and copy the string between the single quotes. Then search for this string in your input file and you will find your xlink:href attribute (or more than one). Next, remove the # at the start of the string and search again. You will not find a gml:id attribute having this string as value. And thus, your file contains references that cannot be resolved. If you do find such a gml:id then you have found a bug.

If you share the dataset, I can possibly give you more hints.

tfqrsli commented 3 years ago

hi clause,

Thanks for reply. Unfortunately, i cannot give the dataset cause we have an agreement with client to keep this as confidential. However, i've try your suggestion and the result is i found both with # and without for #UPI_10084101410062.S.0B.M2.D4_2_BP.Toe0c5FFd90yx72TM3Gk_BP.mpqwwTEswQ7gz9Y9tR3Z_PG.tJaFjzQvC9gXNa34AOp8

i've attach a both result that i've search in input file.

XMLQUERRYSOLUTION2

XMLQUERRYSOLUTION1

clausnagel commented 3 years ago

Thanks for your tests. Well, hard to guess the cause for the issue without being able to check the dataset.

tfqrsli commented 3 years ago

Hi clause,

We already make a decision and agree to give another simple dataset that provide same error (but in LoD3). However, i've try all the suggestion with new dataset but the same building (LoD4) and the result as below ;-

DBQUERY1

DBQUERY2

Also, i've try to export
UPI_10084101410062.S.0B.M2.D4_2_BP.1H05jB9D20eXY62C6KV0_BP.fOPKVZdkB5HGbu7vGkn4_PG.KiZXbg8CNLv79kTGOEt7 using 3dcitydb importer/exporter and the result are still the same.

EXPORTPROBLEM2

EXPORTPROBLEM

sample dataset.zip

VERSION3DCITYDB

It would be great if you willing to share with me a tutorial or any reference to gain more knowledge or fundamental on how it works, limitation and what should or should not have in 3dcitydb importer/exporter.

Special thanks to you clause, cheers.

tfqrsli commented 3 years ago

additional, i have screenshot during process importing are still running. But the log only show 1000 log if i not mistaken.

PROGRESSIMPORTING

It is possible to solve this problem to change from MULTI_GEOMETRY to another type of geometry that support 3dcitydb through xml viewer/editor? This is because i am not generate manually using FME workbench. I am using plugin from Sketchup.

sorry for asking since i still in this domain.

clausnagel commented 3 years ago

Thanks for sharing the example dataset, @tfqrsli. As assumed above, your dataset uses gml:MultiGeometry objects. In general, a gml:MultiGeometry is a collection of geometries that are allowed to have mixed dimensions: points (0D), curves (1D), surfaces (2D) and solids (3D). The 3DCityDB can handle geometries of different dimensions and collections of geometries having the same dimension. But collections containing geometries of different dimensions are not supported.

Five of the six gml:MultiGeometry objects in your dataset contain only polygons (2D) and can therefore be imported (those gml:MultiGeometry objects are internally mapped to gml:MultiSurface). The sixth gml:MultiGeometry, however, contains both polygons (2D) and line strings (1D). This is not supported and therefore the gml:MultiGeometry is skipped and not imported (see corresponding ERROR messages in the log). And, finally, this is the reason why some of the appearance XLinks pointing to elements from this skipped gml:MultiGeometry cannot be resolved.

How to fix this?

I recommend using the latest version 4.3.0 of the Importer/Exporter: https://github.com/3dcitydb/importer-exporter/releases/tag/v4.3.0

tfqrsli commented 3 years ago

Hello Clause,

Thanks for your explanation, It helps a lot to find a root cause and where it come from. It come from stray line in the model and using suggestion from developer in plugin Sketchup, this issue were solve. Thanks to you Clause.

However, i have another issue during importing again which is i tried to validate and import into dummy database and exporting back. All of this process successfully validate, import and export using 3dcitydb importer/exporter. However, when i import into real database, the console log in 3dcitydb importer/exporter show error

[11:19:03 ERROR] A SQL error occurred: Batch entry 0 insert into citydb.TEXTUREPARAM (SURFACE_GEOMETRY_ID, IS_TEXTURE_PARAMETRIZATION, WORLD_TO_TEXTURE, SURFACE_DATA_ID) values (7548554, 0, NULL, 383),(7548578, 0, NULL, 383),(7548598, 0, NULL, 383),(7548724, 0, NULL, 383),(7548770, 0, NULL, 383),(7548784, 0, NULL, 383),(7548794, 0, NULL, 383),(7548804, 0, NULL, 383),(7548812, 0, NULL, 383),(7548834, 0, NULL, 383),(7548846, 0, NULL, 383),(7548932, 0, NULL, 383),(7548952, 0, NULL, 383),(7548956, 0, NULL, 383),(7548968, 0, NULL, 383),(7140334, 0, NULL, 384) was aborted: ERROR: insert or update on table "textureparam" violates foreign key constraint "texparam_surface_data_fk" Detail: Key (surface_data_id)=(384) is not present in table "surface_data". Call getNextException to see other errors in the batch.

[11:19:03 ERROR] Cause: ERROR: insert or update on table "textureparam" violates foreign key constraint "texparam_surface_data_fk" Detail: Key (surface_data_id)=(384) is not present in table "surface_data".

[11:19:03 ERROR] Cause: ERROR: insert or update on table "textureparam" violates foreign key constraint "texparam_surface_data_fk" Detail: Key (surface_data_id)=(384) is not present in table "surface_data".

I lost my mind since this issue come because both database (dummy and real) through the same way to connect and created. could you give an explanation or suggestion to overcome this issue?

thanks, tfq

yaozhihang commented 3 years ago

Hi, Could you share your test datasets? I want to take a try on my machine.

In the mean time, you could also create a third database and test the import/export again. If it works, then the problem is probably from your "real" database, which might be broken due to some kind of unknown operations.

best

Zhihang

clausnagel commented 3 years ago

@tfqrsli, to be able to support you, please send us more information:

  1. The version of the Importer/Exporter you use ("Help -> About" in the main menu).
  2. The type and version of the different databases you use (click on the "Info" button on the "Database" tab after connecting to each of your different databases)
  3. The full log file of an import operation. Go to "Preferences -> General -> Logging" and activate writing log messages to a file before starting the import operation.
  4. Most importantly, we need a (dummy) test file.

Cheers, Claus

clausnagel commented 2 years ago

Closing due to missing feedback. Feel free to re-open if you have more questions.