ngageoint / hootenanny-ui

Hootenanny UI is a submodule of the Hootennany vector conflation project.
ISC License
28 stars 7 forks source link

Issue with Cookie Cutter Conflation #827

Closed kweint closed 7 years ago

kweint commented 7 years ago

Attempted the following method:

Cookie Cutter & Horizontal For this example we’ll need to create two custom translations, one for the DC Street Centerline Data* described in and a second simple translation to ensure that the OSM highway data for DC maintains the correct osm tags.

Ingest DC Street datasets using the recently created custom translation files. Note the different Translation Schema files used to import each dataset.

Hypothesized it could be any issue with the data, but I attempted the above cookie cut conflation on Hoot NOME with zero issues. Would somebody mind trying to replicate the results on hoot release?

sisskind commented 7 years ago

@kweint please drop the error message in. Thanks!

kweint commented 7 years ago

2017-01-13 19:28:43,302 ERROR JobStatusManager:172 - Job with ID: 8e068950-deae-4e84-8969-087cfdbac21d failed: {"childrencount":"3","children":[{"id":"2b4d55b5-92ba-49f9-8106-5443ec7ebbae","detail":"Failed to execute. Cause: Error running conflate:\nUnknown settings option: (cookie.cutter.output.buffer)\nmake: *** [step1] Error 255\n","status":"failed"}],"chainjobstatus":"8e068950-deae-4e84-8969-087cfdbac21d"} 2017-01-13 21:33:41,413 WARN CommandRunnerImpl:89 - Error executing: [make, -f, /home/vagrant/hoot/scripts/makeconflate, ADV_OPTIONS=-D "map.cleaner.transforms=hoot::ReprojectToPlanarOp;hoot::DuplicateWayRemover;hoot::SuperfluousWayRemover;hoot::IntersectionSplitter;hoot::UnlikelyIntersectionRemover;hoot::DualWaySplitter;hoot::ImpliedDividedMarker;hoot::DuplicateNameRemover;hoot::SmallWayMerger;hoot::RemoveEmptyAreasVisitor;hoot::RemoveDuplicateAreaVisitor;hoot::NoInformationElementRemover" -D "small.way.merger.threshold=15" -D "unify.optimizer.time.limit=30" -D "ogr.split.o2s=false" -D "ogr.tds.add.fcsubtype=true" -D "ogr.tds.structure=true" -D "duplicate.name.case.sensitive=true" -D "element.cache.size.node=2000000" -D "element.cache.size.relation=200000" -D "element.cache.size.way=200000" -D "conflate.match.highway.classifier=hoot::HighwayRfClassifier" -D "match.creators=hoot::HighwayMatchCreator;hoot::BuildingMatchCreator;hoot::ScriptMatchCreator,PoiGeneric.js;hoot::ScriptMatchCreator,LinearWaterway.js" -D "merger.creators=hoot::HighwaySnapMergerCreator;hoot::BuildingMergerCreator;hoot::ScriptMergerCreator" -D "search.radius.highway=-1" -D "highway.matcher.heading.delta=5.0" -D "highway.matcher.max.angle=60" -D "way.merger.min.split.size=5" -D "conflate.enable.old.roads=false" -D "waterway.subline.matcher=hoot::MaximalSublineMatcher" -D "waterway.angle.sample.distance=20.0" -D "waterway.matcher.heading.delta=150.0" -D "waterway.auto.calc.search.radius=true" -D "search.radius.waterway=-1" -D "waterway.rubber.sheet.minimum.ties=5" -D "waterway.rubber.sheet.ref=true" -D "writer.include.debug=false" -D "unify.pre.ops=hoot::CookieCutterOp" -D "cookie.cutter.alpha=2500" -D "cookie.cutter.alpha.shape.buffer=0" -D "cookie.cutter.output.buffer=-1000" -D "cookie.cutter.output.crop=false", IS_BIG=false, INPUT1=30, USER_EMAIL=test@test.com, INPUT2=26, TIME_STAMP=1484343221026, REFERENCE_LAYER=1, INPUT2_TYPE=DB, GENERATE_REPORT=false, CONFLATION_TYPE=Horizontal, OUTPUT_NAME=Merged_ImportDirectTest_9d2e52f6, COLLECT_STATS=false, INPUT1_TYPE=DB, jobid=b9782d52-fc5d-4a7a-8b34-4f4db9c12d38, DB_URL=hootapidb://hoot:hoottest@localhost:5432/hoot, OSM_API_DB_URL=osmapidb://hoot:hoottest@localhost:5432/osmapi_test] org.apache.commons.exec.ExecuteException: Process exited with an error: 2 (Exit value: 2) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) ~[commons-exec-1.3.jar:1.3] at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) ~[commons-exec-1.3.jar:1.3] at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) ~[commons-exec-1.3.jar:1.3] at hoot.services.nativeinterfaces.CommandRunnerImpl.exec(CommandRunnerImpl.java:80) [CommandRunnerImpl.class:?] at hoot.services.nativeinterfaces.ProcessStreamInterface.exec(ProcessStreamInterface.java:135) [ProcessStreamInterface.class:?] at sun.reflect.GeneratedMethodAccessor79.invoke(Unknown Source) ~[?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) [spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) [spring-tx-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281) [spring-tx-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) [spring-tx-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) [spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE] at com.sun.proxy.$Proxy37.exec(Unknown Source) [?:?] at hoot.services.nativeinterfaces.JobExecutionManager.exec(JobExecutionManager.java:57) [JobExecutionManager.class:?] at hoot.services.nativeinterfaces.JobExecutionManager$$FastClassBySpringCGLIB$$ee2a09c3.invoke() [JobExecutionManager.class:?] at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) [spring-core-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) [spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) [spring-tx-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281) [spring-tx-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) [spring-tx-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE] at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) [spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE] at hoot.services.nativeinterfaces.JobExecutionManager$$EnhancerBySpringCGLIB$$690c4542.exec() [JobExecutionManager.class:?] at hoot.services.controllers.job.ProcessChainJobRunnable.processJob(ProcessChainJobRunnable.java:313) [ProcessChainJobRunnable.class:?] at hoot.services.controllers.job.ProcessChainJobRunnable.processCommand(ProcessChainJobRunnable.java:137) [ProcessChainJobRunnable.class:?] at hoot.services.controllers.job.ProcessChainJobRunnable.run(ProcessChainJobRunnable.java:85) [ProcessChainJobRunnable.class:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]

As well as the below

2017-01-13 21:33:41,417 ERROR JobStatusManager:172 - Job with ID: 8fdefef3-bca5-423a-84f8-263d3656072d failed: {"childrencount":"3","children":[{"id":"b9782d52-fc5d-4a7a-8b34-4f4db9c12d38","detail":"Failed to execute. Cause: Error running conflate:\nUnknown settings option: (cookie.cutter.output.buffer)\nmake: *** [step1] Error 255\n","status":"failed"}],"chainjobstatus":"8fdefef3-bca5-423a-84f8-263d3656072d"}

brianhatchl commented 7 years ago

@bwitham Is this another option that has been removed from core? Unknown settings option: (cookie.cutter.output.buffer)

brianhatchl commented 7 years ago

I'm going to tackle this ticket #345, so this specific regression won't happen again.

kweint commented 7 years ago

Tested and resolved.