0e1
commented
2 years ago Resumo da ingestão de via e geoaddress, usando a flag:
From file_id=5 inserted type=via_full. +
+
Statistics: +
. +
Before deduplication: +
+
Originals: 4856390 items. +
+
Not Intersecs: 55559 items. +
+
Invalid: 0 items. +
+
Not simple: 1834 items. +
+
Empty: 1671 items. +
+
Small: 3140 items. +
+
Null: 0 items. +
+
Invalid geometry type: 0 items. +
+
Not closed: 0 items. +
+
Inserted in feature_asis: 4795887 items. +
+
Inserted in feature_asis_discarded: 60503 items. +
+
+
After deduplication: +
+
Removed duplicates from feature_asis: 499737 items. +
+
Inserted in feature_asis_discarded (duplicates): 499737 items. +
+
Inserted in feature_asis (aggregated duplicates): 247510 items.+
+
Resulting in feature_asis: 4543660 +
------------------------------------------------------------------------
From file_id=6 inserted type=geoaddress_full. +
+
Statistics: +
. +
Before deduplication: +
+
Originals: 1101237 items. +
+
Not Intersecs: 91705 items. +
+
Invalid: 0 items. +
+
Not simple: 0 items. +
+
Empty: 0 items. +
+
Small: 0 items. +
+
Null: 0 items. +
+
Invalid geometry type: 0 items. +
+
Not closed: 0 items. +
+
Inserted in feature_asis: 1009532 items. +
+
Inserted in feature_asis_discarded: 91705 items. +
+
+
After deduplication: +
+
Removed duplicates from feature_asis: 23696 items. +
+
Inserted in feature_asis_discarded (duplicates): 23696 items. +
+
Inserted in feature_asis (aggregated duplicates): 11662 items.+
+
Resulting in feature_asis: 997498 +
O processo de ingestão do layer via (e provavelmente do layer geoaddress quando se fizer union com centróides dos building do OpenStreetMap) de _pk0004.03 retornou o erro
ERROR: array size exceeds the maximum allowed (1073741823)no contexto
provavelmente quando se aplica _stcollect na função feature_asis_assign_volume: https://github.com/digital-guard/preserv/blob/main/src/ingest-step1-ini.sql#L533
São quase 5 milhões de vias:
Os limites do banco de dados podem serem vistos em: https://www.postgresql.org/docs/current/limits.html
Avaliar como contornar.
Algumas opções:
Como não consideramos as 5 regiões brasileiras como um nível jurisdicional e o problema poderia persistir (especialmente na região sudeste) acho que a opção 2 mais razoável.