yuuhayashi / citygml-osm

CityGMLデータをOSMデータに変換する
Other
19 stars 3 forks source link

横浜の都市部で1st処理がエラー終了する #76

Closed nyampire closed 2 years ago

nyampire commented 2 years ago

v.1.4.3で横浜市のファイルを処理していたところ、一部のファイルにおいて、1stの処理に失敗しているようです。

私が見つけた再現するファイルは、以下の2つです。

それらしいエラー出力は以下です。必要であればその前後も貼ります。

GmlFileReadProcessor : "53391540_bldg_6697_op.gml"
java.io.FileNotFoundException: conversion.json (No such file or directory)
    at java.base/java.io.FileInputStream.open0(Native Method)
    at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
    at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
    at java.base/java.io.FileReader.<init>(FileReader.java:75)
    at osm.surveyor.citygml.ConversionTable.<init>(ConversionTable.java:34)
    at osm.surveyor.citygml.CityModelParser.<init>(CityModelParser.java:63)
    at osm.surveyor.citygml.CitygmlFile.<init>(CitygmlFile.java:24)
    at osm.surveyor.gml.camel.GmlFileReadProcessor.process(GmlFileReadProcessor.java:26)
    at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)
    at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
    at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:76)
    at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
    at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
    at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
    at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:715)
    at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:638)
    at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:248)
    at org.apache.camel.processor.Splitter.process(Splitter.java:129)
    at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
    at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:76)
    at org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:186)
    at org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:86)
    at org.apache.camel.impl.ProducerCache$1.doInProducer(ProducerCache.java:529)
    at org.apache.camel.impl.ProducerCache$1.doInProducer(ProducerCache.java:494)
    at org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:369)
    at org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:494)
    at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:229)
    at org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:144)
    at org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:161)
    at org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:168)
    at osm.surveyor.gml.camel.CitygmlLoad.loadDir(CitygmlLoad.java:39)
    at osm.surveyor.gml.camel.CitygmlLoad.main(CitygmlLoad.java:14)
[main] INFO route6 - Error
[main] INFO route6 - Error: なにかエラーが発生
[main] ERROR org.apache.camel.processor.DefaultErrorHandler - Failed delivery for (MessageId: ID-macbook-air-lan-1649818443738-0-4 on ExchangeId: ID-macbook-air-lan-1649818443738-0-3). Exhausted after delivery attempt: 1 caught: java.lang.NullPointerException. Processed by failure processor: FatalFallbackErrorHandler[Pipeline[[Channel[Log(route6)[Error]], Channel[SetBody(なにかエラーが発生)], Channel[Log(route6)[Error: ${body}]]]]]

Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId              ProcessorId          Processor                                                                        Elapsed (ms)
[route1            ] [route1            ] [file:/Users/nyampire/Downloads/Plateau-Yokohama-extract/esp/./53391540_bldg_66] [     15328]
[route1            ] [process2          ] [Processor@0x52eacb4b                                                          ] [        13]
[route1            ] [to1               ] [direct:gml-file-read                                                          ] [         0]
[route2            ] [process3          ] [Processor@0x5528a42c                                                          ] [      4023]
[route2            ] [process4          ] [Processor@0x2a551a63                                                          ] [        26]
[route2            ] [process5          ] [Processor@0x1a6f5124                                                          ] [       589]
[route2            ] [to2               ] [direct:inRelationMarge                                                        ] [         0]
[route3            ] [process6          ] [Processor@0x1edb61b1                                                          ] [     10263]
[route3            ] [to3               ] [direct:inBuildingGarbage                                                      ] [         0]
[route4            ] [process7          ] [Processor@0xec2bf82                                                           ] [       210]
[route4            ] [to4               ] [direct:inOutlineFactory                                                       ] [         0]
[route5            ] [process8          ] [Processor@0xcc62a3b                                                           ] [        69]
[route5            ] [to5               ] [direct:inOsmMargeWay                                                          ] [         0]
[route6            ] [process9          ] [Processor@0x6cc0bcf6                                                          ] [         5]
[route6            ] [process10         ] [Processor@0x29539e36                                                          ] [        22]
[route6            ] [log3              ] [log                                                                           ] [         2]
[route6            ] [setBody2          ] [setBody[constant{なにかエラーが発生}]                                                  ] [         0]
[route6            ] [log4              ] [log                                                                           ] [         2]

Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
yuuhayashi commented 2 years ago

再現テストをしたわけではないのですが、エラー出力をみると「conversion.json」が見つからないと言っています。 実行ディレクトリの中にconversion.jsonファイルも必要になります。

53391540_bldg_6697_op.gml.zip

nyampire commented 2 years ago

ありがとうございます。なるほどー!と思って conversion.json を置いてみたのですが、やはり通らないみたいです。

$ java -Dfile.encoding=utf-8 -jar citygml-osm-jar-with-dependencies.jar 1st
gml.camel.start();
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.25.4 (CamelContext: camel-1) is starting
[main] INFO org.apache.camel.management.ManagedManagementStrategy - JMX is enabled
[main] INFO org.apache.camel.impl.converter.DefaultTypeConverter - Type converters loaded (core: 195, classpath: 0)
[main] INFO org.apache.camel.impl.DefaultCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route1 started and consuming from: direct://gml-files
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route2 started and consuming from: direct://gml-file-read
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route3 started and consuming from: direct://inRelationMarge
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route4 started and consuming from: direct://inBuildingGarbage
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route5 started and consuming from: direct://inOutlineFactory
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route6 started and consuming from: direct://inOsmMargeWay
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route7 started and consuming from: direct://osm-export
[main] INFO org.apache.camel.impl.DefaultCamelContext - Total 7 routes, of which 7 are started
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.25.4 (CamelContext: camel-1) started in 2.563 seconds
GmlFileReadProcessor : "53391540_bldg_6697_op.gml"
[main] INFO route6 - Error
[main] INFO route6 - Error: なにかエラーが発生
[main] ERROR org.apache.camel.processor.DefaultErrorHandler - Failed delivery for (MessageId: ID-macbook-air-lan-1649826834173-0-4 on ExchangeId: ID-macbook-air-lan-1649826834173-0-3). Exhausted after delivery attempt: 1 caught: java.lang.NullPointerException. Processed by failure processor: FatalFallbackErrorHandler[Pipeline[[Channel[Log(route6)[Error]], Channel[SetBody(なにかエラーが発生)], Channel[Log(route6)[Error: ${body}]]]]]

Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId              ProcessorId          Processor                                                                        Elapsed (ms)
[route1            ] [route1            ] [file:/Users/nyampire/Downloads/Plateau-Yokohama-extract/esp/./53391540_bldg_66] [     15665]
[route1            ] [process2          ] [Processor@0x238b521e                                                          ] [        16]
[route1            ] [to1               ] [direct:gml-file-read                                                          ] [         0]
[route2            ] [process3          ] [Processor@0x1b39fd82                                                          ] [      4135]
[route2            ] [process4          ] [Processor@0x3e2fc448                                                          ] [        22]
[route2            ] [process5          ] [Processor@0x21680803                                                          ] [       583]
[route2            ] [to2               ] [direct:inRelationMarge                                                        ] [         0]
[route3            ] [process6          ] [Processor@0x588ab592                                                          ] [     10577]
[route3            ] [to3               ] [direct:inBuildingGarbage                                                      ] [         0]
[route4            ] [process7          ] [Processor@0xc8b96ec                                                           ] [       180]
[route4            ] [to4               ] [direct:inOutlineFactory                                                       ] [         0]
[route5            ] [process8          ] [Processor@0x4cc61eb1                                                          ] [        45]
[route5            ] [to5               ] [direct:inOsmMargeWay                                                          ] [         0]
[route6            ] [process9          ] [Processor@0x2d8f2f3a                                                          ] [         6]
[route6            ] [process10         ] [Processor@0x2024293c                                                          ] [        22]
[route6            ] [log3              ] [log                                                                           ] [         1]
[route6            ] [setBody2          ] [setBody[constant{なにかエラーが発生}]                                                  ] [         0]
[route6            ] [log4              ] [log                                                                           ] [         2]

Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
yuuhayashi commented 2 years ago

試してみました。

java.lang.NullPointerException
    at osm.surveyor.osm.OsmDom.toOutline(OsmDom.java:201)

ヌルポがでてますね。 特定のGMLでのみ起こるとすると、GMLのデータ側に問題があるようです。 回避策を検討してみます。

yuuhayashi commented 2 years ago

とりあえず、エラーにならないようにしたバージョンを作成しました。

原因はGML側のデータ不良のような感じです。

Screenshot from 2022-04-13 20-29-51

上の建物のOUTLINEを拡大して観てもらうとわかりますが、閉じたエリアでなければならないところが、8の字状になっている(一つのオブジェクトが1点で重なる2つの閉じたエリアになっています)。 古いPLATEAUデータにいくつか発生していたのですが、2020年版でもあるとなると本対策が必要になるかもしれません。

変換されたOSMデータ: 53391550_bldg_6697_op.osm.zip

yuuhayashi commented 2 years ago

fixed : v1.4.4