Open cikorka opened 10 years ago
Něco jako
xmlstarlet ed -N gml=http://www.opengis.net/gml/3.2 -u "//gml:Polygon[@gml:id='HZJ.306045.1']/gml:exterior/gml:LinearRing/gml:posList" -v '-734372.15 -1047921.67 -734355.06 -1047912.68 -734348.31 -1047909.13 -734312.40 -1047888.64 -734292.41 -1047879.00 -734282.19 -1047878.36 -734262.23 -1047920.28 -734194.43 -1047894.99 -734189.73 -1047893.13 -734113.78 -1047905.98 -734113.72 -1047906.09 -734108.43 -1047916.26 -734104.96 -1047922.93 -734074.35 -1047985.34 -734073.97 -1047986.12 -734066.45 -1048005.14 -734060.70 -1048019.67 -734050.88 -1048049.21 -734050.65 -1048050.13 -734049.90 -1048053.16 -734044.86 -1048073.40 -734035.67 -1048110.31 -734030.47 -1048131.21 -734026.82 -1048150.65 -734026.58 -1048151.91 -734024.52 -1048162.64 -734020.52 -1048184.05 -734020.05 -1048186.57 -734018.83 -1048193.12 -734018.56 -1048194.57 -734017.71 -1048199.07 -734014.37 -1048216.82 -733993.24 -1048215.87 -733986.70 -1048215.58 -733986.17 -1048215.56 -733966.89 -1048214.69 -733923.88 -1048212.76 -733915.88 -1048212.40 -733904.19 -1048211.88 -733898.19 -1048211.61 -733892.19 -1048211.34 -733886.19 -1048211.07 -733880.19 -1048210.80 -733874.19 -1048210.53 -733871.27 -1048210.40 -733868.19 -1048210.49 -733862.19 -1048210.66 -733856.19 -1048210.82 -733850.19 -1048210.99 -733840.00 -1048211.28 -733816.10 -1048211.95 -733809.88 -1048212.73 -733798.64 -1048214.15 -733798.17 -1048214.21 -733762.09 -1048218.75 -733775.81 -1048252.96 -733780.82 -1048273.39 -733781.59 -1048288.82 -733779.78 -1048302.51 -733779.03 -1048305.26 -733773.33 -1048326.32 -733764.40 -1048359.32 -733763.99 -1048361.27 -733747.29 -1048440.14 -733739.81 -1048475.47 -733640.53 -1048463.47 -733513.86 -1048448.17 -733470.11 -1048442.88 -733466.47 -1048442.44 -733478.64 -1048378.78 -733479.62 -1048373.97 -733479.44 -1048373.93 -733239.99 -1048656.99 -733266.99 -1048682.00 -733308.33 -1048646.06 -733361.08 -1048613.59 -733438.19 -1048585.18 -733517.38 -1048587.06 -733523.42 -1048587.21 -733892.73 -1048678.52 -733897.29 -1048679.58 -734268.14 -1048765.78 -734316.84 -1048765.78 -734374.99 -1048749.00 -734439.99 -1048720.00 -734543.10 -1048622.94 -734543.26 -1048622.71 -734540.92 -1048621.05 -734489.82 -1048584.84 -734438.19 -1048578.57 -734372.77 -1048565.67 -734368.38 -1048564.81 -734364.85 -1048564.11 -734343.13 -1048560.75 -734274.36 -1048552.15 -734270.85 -1048551.71 -734270.28 -1048547.22 -734255.63 -1048433.20 -734245.47 -1048354.18 -734245.53 -1048343.79 -734245.78 -1048300.31 -734265.68 -1048258.19 -734266.07 -1048177.97 -734266.22 -1048146.00 -734266.30 -1048129.69 -734266.61 -1048062.89 -734266.71 -1048041.81 -734267.07 -1047969.43 -734272.40 -1047961.82 -734331.52 -1047938.02 -734372.15 -1047921.67' 20141031_OB_554782_UKSH.xml.gz | gzip > 20141031_OB_554782_UKSH_good.xml.gz
nepomůže? Poslední dvojice koordinátů je tam navíc. Chyba na ČÚZK nahlášena asi půl roku, ale nic se neděje.
I get an out of memory when i give the statement above. I actually only need all street addresses with their X-Y coordinate.. In the input i still see those. Imported they seem to have been converted to a lonng string.. How does that work?
@cikorka @shr3k Také mě uvedený příkaz spadne na nedostatek paměti.
Přepsal jsem to na:
zcat 20150331_OB_554782_UKSH.xml.gz|sed "s/></>\n</g" |sed 's/\-734331.52 \-1047938.02 \-734372.15 \-1047921.67 \-733798.64 \-1048214.12/\-734331.52 \-1047938.02 \-734372.15 \-1047921.67/' | gzip > 20150331_OB_554782_UKSH_good.xml.gz
nicméně, vypadá to, že daný soubor obsahuje nevalidních dat více:
Processing file tst/20150331_OB_554782_UKSH_good.xml.gz
Warning: Ignoring unsupported element urn:cz:isvs:ruian:schemas:VymennyFormatTypy:v1 VerzeVFR
Warning: Ignoring unsupported element urn:cz:isvs:ruian:schemas:VymennyFormatTypy:v1 PlatnostDatK
Warning: Ignoring unsupported element urn:cz:isvs:ruian:schemas:CommonTypy:v1 ISUI
Warning: Ignoring unsupported element urn:cz:isvs:ruian:schemas:CommonTypy:v1 ISKN
Exception in thread "main" org.postgresql.util.PSQLException: ERROR: parse error - invalid geometry
Hint: "SRID=5514;MULTIPOLYGON()" <-- parse error at position 25 within geometry
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2103)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:381)
at com.fordfrog.ruian2pgsql.convertors.AbstractSaveConvertor.insertItem(AbstractSaveConvertor.java:284)
at com.fordfrog.ruian2pgsql.convertors.AbstractSaveConvertor.saveData(AbstractSaveConvertor.java:246)
at com.fordfrog.ruian2pgsql.convertors.AbstractSaveConvertor.convert(AbstractSaveConvertor.java:168)
at com.fordfrog.ruian2pgsql.convertors.CollectionConvertor.processElement(CollectionConvertor.java:93)
at com.fordfrog.ruian2pgsql.convertors.AbstractConvertor.convert(AbstractConvertor.java:77)
at com.fordfrog.ruian2pgsql.convertors.DataConvertor.processElement(DataConvertor.java:242)
at com.fordfrog.ruian2pgsql.convertors.AbstractConvertor.convert(AbstractConvertor.java:77)
at com.fordfrog.ruian2pgsql.convertors.ExchangeFormatConvertor.processElement(ExchangeFormatConvertor.java:73)
at com.fordfrog.ruian2pgsql.convertors.AbstractConvertor.convert(AbstractConvertor.java:77)
at com.fordfrog.ruian2pgsql.convertors.MainConvertor.processElement(MainConvertor.java:315)
at com.fordfrog.ruian2pgsql.convertors.MainConvertor.readInputStream(MainConvertor.java:294)
at com.fordfrog.ruian2pgsql.convertors.MainConvertor.processFile(MainConvertor.java:251)
at com.fordfrog.ruian2pgsql.convertors.MainConvertor.convert(MainConvertor.java:138)
at com.fordfrog.ruian2pgsql.App.main(App.java:133)
Zkoušel jsem v rychlosti opravit kód, ale nestačí ošetřovat exception při ukládání:( Víte někdo jak ten soubor načíst?
Já psal něco jako, ne detailní návod. Ano, chyb geometrií je tam víc, konkrétně dvě, HZJ.132021.1 a HZJ.306045.1, které je nutno opravit.
<gml:MultiSurface gml:id="HZJ.132021" srsName="urn:ogc:def:crs:EPSG::5514" srsDimension="2"></gml:MultiSurface>
nahradit
<gml:MultiSurface gml:id="HZJ.132021" srsName="urn:ogc:def:crs:EPSG::5514" srsDimension="2"><gml:surfaceMember><gml:Polygon gml:id="HZJ.132021.1"><gml:exterior><gml:LinearRing><gml:posList>-734372.25 -1047921.70 -734348.24 -1047909.34 -734311.83 -1047888.72 -734292.38 -1047879.13 -734281.89 -1047878.68 -734262.16 -1047920.50 -734194.64 -1047894.94 -734190.08 -1047893.22 -734113.86 -1047906.06 -734104.99 -1047922.72 -734089.76 -1047953.66 -734073.81 -1047986.17 -734060.42 -1048020.06 -734050.42 -1048049.56 -734043.82 -1048075.32 -734029.97 -1048131.04 -734023.94 -1048162.86 -734017.27 -1048198.78 -734013.89 -1048216.91 -733937.08 -1048213.37 -733870.71 -1048210.46 -733815.56 -1048211.95 -733798.64 -1048214.12 -733798.64 -1048214.14 -733797.67 -1048351.39 -734112.00 -1048356.00 -734245.36 -1048354.29 -734245.36 -1048354.25 -734245.66 -1048326.20 -734245.73 -1048300.52 -734265.65 -1048258.38 -734265.91 -1048218.54 -734266.14 -1048177.83 -734266.20 -1048146.05 -734266.24 -1048129.76 -734266.32 -1048104.25 -734266.47 -1048062.90 -734266.51 -1048034.72 -734266.60 -1048008.03 -734266.91 -1047988.91 -734266.96 -1047969.48 -734272.49 -1047961.72 -734312.95 -1047945.60 -734347.54 -1047931.62 -734372.25 -1047921.70</gml:posList></gml:LinearRing></gml:exterior></gml:Polygon></gml:surfaceMember></gml:MultiSurface>
a
<gml:MultiSurface gml:id="HZJ.306045" srsName="urn:ogc:def:crs:EPSG::5514" srsDimension="2"><gml:surfaceMember><gml:Polygon gml:id="HZJ.306045.1"><gml:exterior><gml:LinearRing><gml:posList>-734372.15 -1047921.67 -734355.06 -1047912.68 -734348.31 -1047909.13 -734312.40 -1047888.64 -734292.41 -1047879.00 -734282.19 -1047878.36 -734262.23 -1047920.28 -734194.43 -1047894.99 -734189.73 -1047893.13 -734113.78 -1047905.98 -734113.72 -1047906.09 -734108.43 -1047916.26 -734104.96 -1047922.93 -734074.35 -1047985.34 -734073.97 -1047986.12 -734066.45 -1048005.14 -734060.70 -1048019.67 -734050.88 -1048049.21 -734050.65 -1048050.13 -734049.90 -1048053.16 -734044.86 -1048073.40 -734035.67 -1048110.31 -734030.47 -1048131.21 -734026.82 -1048150.65 -734026.58 -1048151.91 -734024.52 -1048162.64 -734020.52 -1048184.05 -734020.05 -1048186.57 -734018.83 -1048193.12 -734018.56 -1048194.57 -734017.71 -1048199.07 -734014.37 -1048216.82 -733993.24 -1048215.87 -733986.70 -1048215.58 -733986.17 -1048215.56 -733966.89 -1048214.69 -733923.88 -1048212.76 -733915.88 -1048212.40 -733904.19 -1048211.88 -733898.19 -1048211.61 -733892.19 -1048211.34 -733886.19 -1048211.07 -733880.19 -1048210.80 -733874.19 -1048210.53 -733871.27 -1048210.40 -733868.19 -1048210.49 -733862.19 -1048210.66 -733856.19 -1048210.82 -733850.19 -1048210.99 -733840.00 -1048211.28 -733816.10 -1048211.95 -733809.88 -1048212.73 -733798.64 -1048214.15 -733798.17 -1048214.21 -733762.09 -1048218.75 -733775.81 -1048252.96 -733780.82 -1048273.39 -733781.59 -1048288.82 -733779.78 -1048302.51 -733779.03 -1048305.26 -733773.33 -1048326.32 -733764.40 -1048359.32 -733763.99 -1048361.27 -733747.29 -1048440.14 -733739.81 -1048475.47 -733640.53 -1048463.47 -733513.86 -1048448.17 -733470.11 -1048442.88 -733466.47 -1048442.44 -733478.64 -1048378.78 -733479.62 -1048373.97 -733479.44 -1048373.93 -733239.99 -1048656.99 -733266.99 -1048682.00 -733308.33 -1048646.06 -733361.08 -1048613.59 -733438.19 -1048585.18 -733517.38 -1048587.06 -733523.42 -1048587.21 -733892.73 -1048678.52 -733897.29 -1048679.58 -734268.14 -1048765.78 -734316.84 -1048765.78 -734374.99 -1048749.00 -734439.99 -1048720.00 -734543.10 -1048622.94 -734543.26 -1048622.71 -734540.92 -1048621.05 -734489.82 -1048584.84 -734438.19 -1048578.57 -734372.77 -1048565.67 -734368.38 -1048564.81 -734364.85 -1048564.11 -734343.13 -1048560.75 -734274.36 -1048552.15 -734270.85 -1048551.71 -734270.28 -1048547.22 -734255.63 -1048433.20 -734245.47 -1048354.18 -734245.53 -1048343.79 -734245.78 -1048300.31 -734265.68 -1048258.19 -734266.07 -1048177.97 -734266.22 -1048146.00 -734266.30 -1048129.69 -734266.61 -1048062.89 -734266.71 -1048041.81 -734267.07 -1047969.43 -734272.40 -1047961.82 -734331.52 -1047938.02 -734372.15 -1047921.67 -733798.64 -1048214.12</gml:posList></gml:LinearRing></gml:exterior></gml:Polygon></gml:surfaceMember></gml:MultiSurface>
nahradit
<gml:MultiSurface gml:id="HZJ.306045" srsName="urn:ogc:def:crs:EPSG::5514" srsDimension="2"><gml:surfaceMember><gml:Polygon gml:id="HZJ.306045.1"><gml:exterior><gml:LinearRing><gml:posList>-734372.15 -1047921.67 -734355.06 -1047912.68 -734348.31 -1047909.13 -734312.40 -1047888.64 -734292.41 -1047879.00 -734282.19 -1047878.36 -734262.23 -1047920.28 -734194.43 -1047894.99 -734189.73 -1047893.13 -734113.78 -1047905.98 -734113.72 -1047906.09 -734108.43 -1047916.26 -734104.96 -1047922.93 -734074.35 -1047985.34 -734073.97 -1047986.12 -734066.45 -1048005.14 -734060.70 -1048019.67 -734050.88 -1048049.21 -734050.65 -1048050.13 -734049.90 -1048053.16 -734044.86 -1048073.40 -734035.67 -1048110.31 -734030.47 -1048131.21 -734026.82 -1048150.65 -734026.58 -1048151.91 -734024.52 -1048162.64 -734020.52 -1048184.05 -734020.05 -1048186.57 -734018.83 -1048193.12 -734018.56 -1048194.57 -734017.71 -1048199.07 -734014.37 -1048216.82 -733993.24 -1048215.87 -733986.70 -1048215.58 -733986.17 -1048215.56 -733966.89 -1048214.69 -733923.88 -1048212.76 -733915.88 -1048212.40 -733904.19 -1048211.88 -733898.19 -1048211.61 -733892.19 -1048211.34 -733886.19 -1048211.07 -733880.19 -1048210.80 -733874.19 -1048210.53 -733871.27 -1048210.40 -733868.19 -1048210.49 -733862.19 -1048210.66 -733856.19 -1048210.82 -733850.19 -1048210.99 -733840.00 -1048211.28 -733816.10 -1048211.95 -733809.88 -1048212.73 -733798.64 -1048214.15 -733798.17 -1048214.21 -733762.09 -1048218.75 -733775.81 -1048252.96 -733780.82 -1048273.39 -733781.59 -1048288.82 -733779.78 -1048302.51 -733779.03 -1048305.26 -733773.33 -1048326.32 -733764.40 -1048359.32 -733763.99 -1048361.27 -733747.29 -1048440.14 -733739.81 -1048475.47 -733640.53 -1048463.47 -733513.86 -1048448.17 -733470.11 -1048442.88 -733466.47 -1048442.44 -733478.64 -1048378.78 -733479.62 -1048373.97 -733479.44 -1048373.93 -733239.99 -1048656.99 -733266.99 -1048682.00 -733308.33 -1048646.06 -733361.08 -1048613.59 -733438.19 -1048585.18 -733517.38 -1048587.06 -733523.42 -1048587.21 -733892.73 -1048678.52 -733897.29 -1048679.58 -734268.14 -1048765.78 -734316.84 -1048765.78 -734374.99 -1048749.00 -734439.99 -1048720.00 -734543.10 -1048622.94 -734543.26 -1048622.71 -734540.92 -1048621.05 -734489.82 -1048584.84 -734438.19 -1048578.57 -734372.77 -1048565.67 -734368.38 -1048564.81 -734364.85 -1048564.11 -734343.13 -1048560.75 -734274.36 -1048552.15 -734270.85 -1048551.71 -734270.28 -1048547.22 -734255.63 -1048433.20 -734245.47 -1048354.18 -734245.53 -1048343.79 -734245.78 -1048300.31 -734265.68 -1048258.19 -734266.07 -1048177.97 -734266.22 -1048146.00 -734266.30 -1048129.69 -734266.61 -1048062.89 -734266.71 -1048041.81 -734267.07 -1047969.43 -734272.40 -1047961.82 -734331.52 -1047938.02 -734372.15 -1047921.67</gml:posList></gml:LinearRing></gml:exterior></gml:Polygon></gml:surfaceMember></gml:MultiSurface>
Díky, HZJ.132021.1
v 20150331_OB_554782_UKSH_good.xml.gz
vůbec není, ale stačilo odstranit prázdný HZJ.306045
Tím se opravila další chyba, dále to spadlo na:
Exception in thread "main" org.postgresql.util.PSQLException: ERROR: geometry contains non-closed rings
Hint: "... -1047921.67,-733798.64 -1048214.12))" <-- parse error at position 2582 within geometry
ale zajímavé je, že:
No to je ten ocásek (poslední dvojice bodů) navíc v hranici ZSJ 306045, viz upravená zpráva s konkrétním search / replace... :-)
aha! samozřejmě. Tím se to vysvětluje. Musel jsem to jednou spustit omylem s orginálem.
díky moc.
Napsal jsem znovu jednomu člověku z ČÚZK (o opravu jsem žádal poprvé už před rokem přes podpora@cuzk.cz), jestli by na to nemohl dohlédnout...
Found out how to read the definicni_bod :) ST_AsText(amo.definicni_bod). Will (after translating the msg above) check if i can also correct the three wrong files. Or do i need to import all three or only newest one?
To create a more readable address i made the selection below. Do you have any recomendations? Am i missing handy things? Assuming wrong joins? Thnks already!
select distinct amo_kod
, psc
, obec_nazev
, cobec_nazev
, ulice_nazev
, cislo_domovni
, cislo_orientacni_hodnota
, cislo_orientacni_pismeno
, households
, typ_kod
, to_number(substring(geom from 7 for position(' ' in geom)-7),'99999999.99') SOURADNICE_Y
, to_number(substring(geom from (position(' ' in geom)+1) for position(')' in geom)-(position(' ' in geom)+1)),'99999999.99') SOURADNICE_X
, 0 building_asc_id
, 0 building_desc_id
, null letter_combi
from
(
select amo.kod amo_kod
, adrp_psc PSC
, obc.nazev obec_nazev
, coe.nazev cobec_nazev
, ule.nazev ulice_nazev
, amo.cislo_domovni
, amo.cislo_orientacni_hodnota
, amo.cislo_orientacni_pismeno
, coalesce(dta.pocet_bytu,1) households
, sot.typ_kod
, ST_AsText(amo.definicni_bod) AS geom
from rn_adresni_misto amo
left join rn_ulice ule on amo.ulice_kod = ule.kod
left join rn_detailni_tea dta on amo.kod = dta.adresni_misto_kod
join rn_stavebni_objekt sot on amo.stavobj_kod = sot.kod
join rn_cast_obce coe on sot.cobce_kod = coe.kod
join rn_obec obc on coe.obec_kod = obc.kod
left join rn_okres oks on obc.okres_kod = oks.kod
where 1=1
) x
where not (amo_kod = 3334791 and households = 12)
and not (amo_kod = 24307629 and households = 8)
and not (amo_kod = 27841430 and households = 1)
order by x.psc, x.obec_nazev, x.cobec_nazev, x.Ulice_nazev
, x.cislo_orientacni_hodnota
, x.cislo_domovni
, x.cislo_orientacni_pismeno
;
Fyi the issue has to be corrected at the statistical office. They are working on it, hopefully soon fixed.
Chtělo by to přidat možnost nahlásit chybu, ale pokračovat v importu ostatních geometrií. Další chyby obsahuje třeba letošní 20200505_ST_ZKSH a 20200831_ST_ZKSH ... peklo. Nechápu, jak registr veřejné správy může obsahovat a generovat nevalidní geometrie.