state-hiu / 2C-GeoNode

3 stars 9 forks source link

Bulk Upload: Cusco #111

Closed sophialinn closed 6 years ago

sophialinn commented 6 years ago

The partner-provided Cusco data is ready to be uploaded from: https://drive.google.com/drive/folders/0B2Oc3HZSI7QFaVk5NDl6TkVIYTQ?usp=sharing

kalxas commented 6 years ago

Raster uploads:

Found 3 potential layers.

[created] Layer for '/home/ubuntu/tiffs/RAS09.tif' (1/3)

[created] Layer for '/home/ubuntu/tiffs/RAS17.tif' (2/3)

[created] Layer for '/home/ubuntu/tiffs/RAS10.tif' (3/3)

Detailed report of failures:

Finished processing 3 layers in 67.0 seconds.

3 Created layers
0 Updated layers
0 Skipped layers
0 Failed layers
22.333333 seconds per layer
kalxas commented 6 years ago

Vector uploads:

Found 111 potential layers.

[created] Layer for '/home/ubuntu/new/GEOMORFO.shp' (1/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_situacion_limite_distrital.shp' (2/111)

[created] Layer for '/home/ubuntu/new/t_pdu_ambiental_comunidades_campesinas.shp' (3/111)

[created] Layer for '/home/ubuntu/new/t_pdu_ambiental_diversidad_biologica.shp' (4/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_red_vial_vecinal.shp' (5/111)

[created] Layer for '/home/ubuntu/new/lotes_parametros_urbanos_pdu.shp' (6/111)

[created] Layer for '/home/ubuntu/new/t_pdm_ambiental_forestal_pastoreo.shp' (7/111)

[created] Layer for '/home/ubuntu/new/t_pdm_ambiental_lagunas_lagos.shp' (8/111)

[created] Layer for '/home/ubuntu/new/LotesCV.shp' (9/111)

[created] Layer for '/home/ubuntu/new/t_pdu_base_distrito.shp' (10/111)

[created] Layer for '/home/ubuntu/new/t_pdm_grd_vulnerabilidad_riesgo_movimiento_agricola.shp' (11/111)

[created] Layer for '/home/ubuntu/new/LOTES.shp' (12/111)

[created] Layer for '/home/ubuntu/new/t_pdm_economico_esquema_estructuracion_linea.shp' (13/111)

[created] Layer for '/home/ubuntu/new/t_pud_base_borde_urbano.shp' (14/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_prop_area_urbana.shp' (15/111)

[created] Layer for '/home/ubuntu/new/t_pud_ambiental_cont_residuos.shp' (16/111)

[created] Layer for '/home/ubuntu/new/t_pdm_fisico_construido_diag_areas_homogeneas.shp' (17/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_red_vial_departamental.shp' (18/111)

[created] Layer for '/home/ubuntu/new/t_pud_gestionriesgos_proteccion_ambiental_dttocusco.shp' (19/111)

[created] Layer for '/home/ubuntu/new/t_pdu_patrimonio_cultural_sitios_arqueologicos.shp' (20/111)

[created] Layer for '/home/ubuntu/new/t_pdm_fisico_construido_contaminacion_suelos.shp' (21/111)

[created] Layer for '/home/ubuntu/new/FocosRRSS.shp' (22/111)

[created] Layer for '/home/ubuntu/new/RutaRecojo.shp' (23/111)

[created] Layer for '/home/ubuntu/new/CentroAcopio.shp' (24/111)

[created] Layer for '/home/ubuntu/new/BarriosPrmtro.shp' (25/111)

[created] Layer for '/home/ubuntu/new/t_pdm_ambiental_diag_red_hidrica.shp' (26/111)

[created] Layer for '/home/ubuntu/new/Lotes_CaminoReal.shp' (27/111)

[created] Layer for '/home/ubuntu/new/t_pdm_fisico_construido_caracterizacion_rrss.shp' (28/111)

[created] Layer for '/home/ubuntu/new/t_pdm_economico_esquema_estructuracion_poligono.shp' (29/111)

[created] Layer for '/home/ubuntu/new/t_pdu_ambiental_peligro_remocion_masa.shp' (30/111)

[created] Layer for '/home/ubuntu/new/t_pdm_ambiental_areas_intervencion_especial_recuperacion.shp' (31/111)

[created] Layer for '/home/ubuntu/new/RED_HIDRICA.shp' (32/111)

[created] Layer for '/home/ubuntu/new/GEODINAMICA.shp' (33/111)

[created] Layer for '/home/ubuntu/new/HistCult.shp' (34/111)

[created] Layer for '/home/ubuntu/new/Botaderos_CaminoReal.shp' (35/111)

[created] Layer for '/home/ubuntu/new/t_pdm_ambiental_areas_conservacion_uso_sostenible.shp' (36/111)

[created] Layer for '/home/ubuntu/new/t_pdu_fisico_construido_vias_tramo.shp' (37/111)

[created] Layer for '/home/ubuntu/new/RedHidrica.shp' (38/111)

[created] Layer for '/home/ubuntu/new/Manzanas_CaminoReal.shp' (39/111)

[created] Layer for '/home/ubuntu/new/PENDIENTES.shp' (40/111)

[created] Layer for '/home/ubuntu/new/Humedal.shp' (41/111)

[created] Layer for '/home/ubuntu/new/compost_modelos.shp' (42/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_manzanas.shp' (43/111)

[created] Layer for '/home/ubuntu/new/Barrios.shp' (44/111)

[created] Layer for '/home/ubuntu/new/Arboles.shp' (45/111)

[created] Layer for '/home/ubuntu/new/t_pdu_ambiental_sistema_ecologico.shp' (46/111)

[created] Layer for '/home/ubuntu/new/t_pud_gestionriesgos_vulnerabilidad_social.shp' (47/111)

[created] Layer for '/home/ubuntu/new/rt04_rev_vial_departamental2014.shp' (48/111)

[created] Layer for '/home/ubuntu/new/Pendientes_CaminoReal.shp' (49/111)

[created] Layer for '/home/ubuntu/new/Vias.shp' (50/111)

[created] Layer for '/home/ubuntu/new/PELI_INUNDA.shp' (51/111)

[created] Layer for '/home/ubuntu/new/t_pat_fisico_construido_zona_de_aptitud_urbana.shp' (52/111)

[created] Layer for '/home/ubuntu/new/EcologAmbtal_CaminoReal.shp' (53/111)

[created] Layer for '/home/ubuntu/new/EcologAmbtal_SanPedro.shp' (54/111)

[created] Layer for '/home/ubuntu/new/BotaderosInformales.shp' (55/111)

[created] Layer for '/home/ubuntu/new/t_pdu_ambiental_unidad_hidrografica.shp' (56/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_ambito_distrital_metropolitano.shp' (57/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_capitales_distritos.shp' (58/111)

[created] Layer for '/home/ubuntu/new/t_pdu_ambiental_residuos_solidos.shp' (59/111)

[created] Layer for '/home/ubuntu/new/t_pdm_ambiental_zona_agricola.shp' (60/111)

[created] Layer for '/home/ubuntu/new/t_pdu_base_provincias.shp' (61/111)

[created] Layer for '/home/ubuntu/new/t_pat_ambiental_zona_de_conservacion_y_uso_sostenible_de_suelo.shp' (62/111)

[created] Layer for '/home/ubuntu/new/t_pdm_ambiental_areas_proteccion_ecologica_natural_propuesta.shp' (63/111)

[created] Layer for '/home/ubuntu/new/BarrioSanPedro.shp' (64/111)

[created] Layer for '/home/ubuntu/new/t_pud_gestionriesgos_vulnerabilidad_ambiental.shp' (65/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_ccpp_ambito.shp' (66/111)

[created] Layer for '/home/ubuntu/new/t_pdm_grd_diag_geodinamica_compilada.shp' (67/111)

[created] Layer for '/home/ubuntu/new/t_pat_base_ambito.shp' (68/111)

[created] Layer for '/home/ubuntu/new/t_pud_gestionriesgos_vulnerabilidad_economica.shp' (69/111)

[created] Layer for '/home/ubuntu/new/ARU_18.shp' (70/111)

[created] Layer for '/home/ubuntu/new/t_pdu_social_pobreza.shp' (71/111)

[created] Layer for '/home/ubuntu/new/t_pdm_fisico_construido_estructuracion_urbana_diagnostico.shp' (72/111)

[created] Layer for '/home/ubuntu/new/Pendientes_SanPedro.shp' (73/111)

[created] Layer for '/home/ubuntu/new/rt04_rev_vial_vecinal2014.shp' (74/111)

[created] Layer for '/home/ubuntu/new/t_pdu_ambiental_residuos_liquidos.shp' (75/111)

[created] Layer for '/home/ubuntu/new/RRSS_19.shp' (76/111)

[created] Layer for '/home/ubuntu/new/BAS_LIM_DEPARTAMENTO.shp' (77/111)

[created] Layer for '/home/ubuntu/new/t_pdu_fisico_construido_red_agua.shp' (78/111)

[created] Layer for '/home/ubuntu/new/t_pat_grd_proteccion_por_riesgo_de_desastre.shp' (79/111)

[created] Layer for '/home/ubuntu/new/BAS_LIM_PROVINCIA.shp' (80/111)

[created] Layer for '/home/ubuntu/new/Area.shp' (81/111)

[created] Layer for '/home/ubuntu/new/t_pud_gestionriesgos_lotes_vulnerabilidad_ultimo.shp' (82/111)

[created] Layer for '/home/ubuntu/new/MANZANA_POBLA.shp' (83/111)

[created] Layer for '/home/ubuntu/new/CCPP.shp' (84/111)

[created] Layer for '/home/ubuntu/new/t_pdm_fisico_construido_diag_equipamiento_urbano.shp' (85/111)

[created] Layer for '/home/ubuntu/new/t_pdm_fisico_construido_contaminacion_hidrica.shp' (86/111)

[created] Layer for '/home/ubuntu/new/t_pdu_fisico_construido_red_desague.shp' (87/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_ambito_metropolitano.shp' (88/111)

[created] Layer for '/home/ubuntu/new/t_pud_ambiental_escombros.shp' (89/111)

[created] Layer for '/home/ubuntu/new/LotesEncuesta.shp' (90/111)

[created] Layer for '/home/ubuntu/new/CentroTrasferencia.shp' (91/111)

[created] Layer for '/home/ubuntu/new/AGRUPACION.shp' (92/111)

[created] Layer for '/home/ubuntu/new/t_pdm_economico_lugares_turisticos.shp' (93/111)

[created] Layer for '/home/ubuntu/new/t_pdm_grd_diag_peligro_inundacion.shp' (94/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_curvas_nivel.shp' (95/111)

[created] Layer for '/home/ubuntu/new/RutaRecojoRRSS.shp' (96/111)

[created] Layer for '/home/ubuntu/new/t_pat_base_cuerpos_de_agua.shp' (97/111)

[created] Layer for '/home/ubuntu/new/t_pud_uso_suelo_ambiental.shp' (98/111)

[created] Layer for '/home/ubuntu/new/BAS_LIM_DISTRITOS.shp' (99/111)

[created] Layer for '/home/ubuntu/new/ARU_19.shp' (100/111)

[created] Layer for '/home/ubuntu/new/ApvCaminoReal.shp' (101/111)

[created] Layer for '/home/ubuntu/new/GEOMORFO_PDM.shp' (102/111)

[created] Layer for '/home/ubuntu/new/Manzanas_SanPedro.shp' (103/111)

[created] Layer for '/home/ubuntu/new/Lotes_SanPedro.shp' (104/111)

[created] Layer for '/home/ubuntu/new/compost_botaderos_centro_historico.shp' (105/111)

[created] Layer for '/home/ubuntu/new/CercoVivo.shp' (106/111)

[created] Layer for '/home/ubuntu/new/PELI_REMOSION.shp' (107/111)

[created] Layer for '/home/ubuntu/new/t_pdm_ambiental_areas_expansion_propuesta.shp' (108/111)

[created] Layer for '/home/ubuntu/new/RRSS_18.shp' (109/111)

[created] Layer for '/home/ubuntu/new/t_pdm_ambiental_areas_conservacion_ecologica.shp' (110/111)

[created] Layer for '/home/ubuntu/new/t_pdm_base_rios_quebradas.shp' (111/111)

Detailed report of failures:

Finished processing 111 layers in 1023.0 seconds.

111 Created layers
0 Updated layers
0 Skipped layers
0 Failed layers
9.216216 seconds per layer
kalxas commented 6 years ago

Metadata update:

Cusco: Urban Area Classification : Metadata updated!
Cusco: Boundary of Camino Real : Metadata updated!
Cusco: Wetland Trees in Camino Real : Metadata updated!
Cusco: Area around Wetland in Camino Real : Metadata updated!
Cusco: Residual Water in Camino Real : Metadata updated!
Cusco: Residual Water in Camino Real_2 : Metadata updated!
Cusco: Neighborhood Categorizations : Metadata updated!
Cusco: San Pedro Neighborhood : Metadata updated!
Cusco: Neighborhood Categorization Groups : Metadata updated!
Cusco: Departments : Metadata updated!
Cusco: Districts : Metadata updated!
Cusco: Provinces : Metadata updated!
Cusco: Waste Disposal in Camino Real : Metadata updated!
Cusco: Informal Disposal Sites of Camino Real : Metadata updated!
Cusco: Population Centers : Metadata updated!
Cusco: Collection centers in Camino Real : Metadata updated!
Cusco: Transfer centers in Camino Real : Metadata updated!
Cusco: Hedges near Wetland in Camino Real : Metadata updated!
Cusco: Waste Disposal in the Historic Center : Metadata updated!
Cusco: Compost Models : Metadata updated!
Cusco: Green Areas and Wetlands in Camino Real : Metadata updated!
Cusco: Green Areas in San Pedro : Metadata updated!
Cusco: Areas of Contamination in and around Wetland of Camino Real : Metadata updated!
Cusco: Geodynamics : Metadata updated!
Cusco: Geomorphology : Metadata updated!
Cusco: Geomorphological Subclassifications : Metadata updated!
Cusco: Historical and Cultural Sites in Camino Real : Metadata updated!
Cusco: Wetland Area of Camino Real : Metadata updated!
Cusco: Land Use : Metadata updated!
Cusco: Lots in Camino Real : Metadata updated!
Cusco: Land Use : Metadata updated!
Cusco: Lots in San Pedro : Metadata updated!
Cusco: Lots of Camino Real : Metadata updated!
Cusco: Lots from Survey in Camino Real : Metadata updated!
Cusco: Blocks : Metadata updated!
Cusco: Blocks in Camino Real : Metadata updated!
Cusco: Blocks in San Pedro : Metadata updated!
cusco: educational institutions : Failed to update
Exception: Layer matching query does not exist.
Cusco: Flood Risk : Metadata updated!
Cusco: Mass Movement Hazard : Metadata updated!
Cusco: Slopes : Metadata updated!
Cusco: Slopes in Camino Real : Metadata updated!
Cusco: Slopes in San Pedro : Metadata updated!
cusco: housing quality in camino real : Failed to update
Exception: Layer matching query does not exist.
cusco: environmental contamination in camino real : Failed to update
Exception: Layer matching query does not exist.
cusco: waste collection in camino real : Failed to update
Exception: Layer matching query does not exist.
Cusco: Hydrological Network : Metadata updated!
Cusco: Hydric System at Wetland of Camino Real : Metadata updated!
Cusco: Informal Solid Waste Disposal in Camino Real_2 : Metadata updated!
Cusco: Informal Solid Waste Disposal in Camino Real : Metadata updated!
Cusco: Roads : Metadata updated!
Cusco: Local Roads : Metadata updated!
Cusco: Proposed Route of Waste Collection in Camino Real : Metadata updated!
Cusco: Observed Route of Waste Collection in Camino Real : Metadata updated!
Cusco: Areas of Conservation : Metadata updated!
Cusco: Boundary of Cusco Province : Metadata updated!
Cusco: Bodies of Water : Metadata updated!
Cusco: Urban and Rural Settlements  : Metadata updated!
Cusco: At Risk Protection Area : Metadata updated!
Cusco: Ecological Conservation Areas : Metadata updated!
Cusco: Areas of Conservation_2 : Metadata updated!
Cusco: Expansion Proposal : Metadata updated!
Cusco: Recuperation Areas : Metadata updated!
Cusco: Ecological Conservation Areas_2 : Metadata updated!
Cusco: Hydrological Network : Metadata updated!
Cusco: Forest and Pasture  : Metadata updated!
Cusco: Lagoons and Lakes : Metadata updated!
Cusco: Agricultural Areas : Metadata updated!
Cusco: Boundaries of Metropolitan Districts : Metadata updated!
Cusco: Metropolitan Area : Metadata updated!
Cusco: District Capitals : Metadata updated!
Cusco: Metropolitan Population Centers : Metadata updated!
Cusco: Topography : Metadata updated!
Cusco: Block Boundaries : Metadata updated!
Cusco: Urban Areas : Metadata updated!
Cusco: Regional Road Network : Metadata updated!
Cusco: Rural Road Network : Metadata updated!
Cusco: Rivers and Brooks : Metadata updated!
Cusco: Districts : Metadata updated!
Cusco: Economic Planning Lines : Metadata updated!
Cusco: Economic Planning Areas : Metadata updated!
Cusco: Tourist Attractions : Metadata updated!
Cusco: Solid Waste Disposal Analysis : Metadata updated!
Cusco: Water Contamination : Metadata updated!
Cusco: Soil Contamination : Metadata updated!
Cusco: Broad Category Land Use : Metadata updated!
Cusco: Urban Sites : Metadata updated!
Cusco: Urban Structure : Metadata updated!
Cusco: Geodynamics Compiled : Metadata updated!
Cusco: Flood Risk : Metadata updated!
Cusco: Vulnerable Agricultural Land : Metadata updated!
Cusco: Rural Communities : Metadata updated!
Cusco: Biological Diversity : Metadata updated!
Cusco: Mass Movement Hazard : Metadata updated!
Cusco: Liquid Waste : Metadata updated!
Cusco: Solid Waste : Metadata updated!
Cusco: Ecological System : Metadata updated!
Cusco: Watersheds : Metadata updated!
Cusco: District : Metadata updated!
Cusco: Provinces : Metadata updated!
Cusco: Built Water Network : Metadata updated!
Cusco: Built Drain Network : Metadata updated!
Cusco: Built Paths : Metadata updated!
Cusco: Cultural Heritage Archaeological Sites : Metadata updated!
Cusco: Poverty : Metadata updated!
Cusco: Enviromental Contamination : Metadata updated!
Cusco: Debris / Rubble waste : Metadata updated!
Cusco: Urban Boundary : Metadata updated!
Cusco: Risk Management for Vulnerable Lots : Metadata updated!
Cusco: Risk Management Protection Areas : Metadata updated!
Cusco: Environmental Vulnerability : Metadata updated!
Cusco: Economic Vulnerability : Metadata updated!
Cusco: Social Vulnerability : Metadata updated!
Cusco: Waste Observations : Metadata updated! 
Cusco: Roads & Paths : Metadata updated!
kalxas commented 6 years ago

Problem uploading PADRON_IE.shp:

Found 1 potential layers.

[failed] Layer for '/home/ubuntu/new3/PADRON_IE.shp' (1/1)

Detailed report of failures:

/home/ubuntu/new3/PADRON_IE.shp
================
Traceback (most recent call last):
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/geonode/layers/utils.py", line 625, in upload
    metadata_uploaded_preserve=metadata_uploaded_preserve
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/geonode/layers/utils.py", line 499, in file_upload
    defaults=defaults
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/modeltranslation/manager.py", line 389, in get_or_create
    return super(MultilingualQuerySet, self).get_or_create(**kwargs)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/query.py", line 407, in get_or_create
    return self._create_object_from_params(lookup, params)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/query.py", line 439, in _create_object_from_params
    obj = self.create(**params)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/modeltranslation/manager.py", line 381, in create
    return super(MultilingualQuerySet, self).create(**kwargs)  
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/query.py", line 348, in create
    obj.save(force_insert=True, using=self.db)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/polymorphic/models.py", line 74, in save
    return super(PolymorphicModel, self).save(*args, **kwargs) 
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/base.py", line 734, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/base.py", line 761, in save_base
    self._save_parents(cls, using, update_fields)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/base.py", line 786, in _save_parents
    self._save_table(cls=parent, using=using, update_fields=update_fields)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/base.py", line 846, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/base.py", line 885, in _do_insert
    using=using, raw=raw)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs) 
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/query.py", line 920, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 973, in execute_sql
    for sql, params in self.as_sql():
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 931, in as_sql
    for obj in self.query.objs
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py", line 1627, in get_db_prep_save
    self.max_digits, self.decimal_places)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/backends/base/operations.py", line 477, in value_to_db_decimal   
    return utils.format_number(value, max_digits, decimal_places)
  File "/home/ubuntu/venvs/state_2c_geonode/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 200, in format_number
    value = value.quantize(decimal.Decimal(".1") ** decimal_places, context=context)
  File "/usr/lib/python2.7/decimal.py", line 2442, in quantize 
    'quantize with one INF')
  File "/usr/lib/python2.7/decimal.py", line 3873, in _raise_error
    raise error(explanation)
InvalidOperation: quantize with one INF

Finished processing 1 layers in 91.0 seconds.

0 Created layers
0 Updated layers
0 Skipped layers
1 Failed layers
91.000000 seconds per layer
kalxas commented 6 years ago

Data uploaded to http://secondarycities.geonode.state.gov/

kalxas commented 6 years ago

The problem with PADRON_IE.shp did not happen in production, since GeoServer is better optimized in the production environment.