Can't 'use geonode updatelayers' or 'python manage.py' to updatelayers to with geoserver

[manaetches]

Hi all,

The situation is this: I installed geonode 2.4 as per installation instructions at http://geonode.org/blog/2015/11/19/geonode-2.4-released/. Installation on Ubuntu Server 14.04.

I migrated geonode data from 2.0 to 2.4 using capooti's instructions at https://github.com/capooti/geonode/blob/migration_from20_to_24/scripts/migrations/migrate20to24/index.rst, many thanks Capooti. This included the geoserver data along with relevant subdirectories. I double checked the geoserver and yes the layers and stores are present.

I ran 'sudo geonode updatelayers' to sync geoserver layers with geonode, and I get the following error: Not enabling BingMaps base layer as a BING_API_KEY is not defined in local_settings.py file. Traceback (most recent call last):

File "/usr/local/bin/django-admin", line 9, in load_entry_point('Django==1.9.1', 'console_scripts', 'django-admin')() File "/usr/local/lib/python2.7/dist-packages/django/core/management/init.py", line 353, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/dist-packages/django/core/management/init.py", line 327, in execute django.setup() File "/usr/local/lib/python2.7/dist-packages/django/init.py", line 17, in setup configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) File "/usr/local/lib/python2.7/dist-packages/django/utils/log.py", line 71, in configure_logging logging_config_func(logging_settings) File "/usr/lib/python2.7/logging/config.py", line 794, in dictConfig dictConfigClass(config).configure() File "/usr/lib/python2.7/logging/config.py", line 576, in configure '%r: %s' % (name, e)) ValueError: Unable to configure handler 'null': Cannot resolve 'django.utils.log.NullHandler': No module named NullHandler

I ran sudo 'sudo python ./manage.py updatelayers and I get: python: can't open file './manage.py': [Errno 2] No such file or directory

Have I missed something or is this a bug?

I appreciate any guide on this issue.

Cheers

[simod]

Hi,

could you please refer to the mailing list geonode-users@lists.osgeo.org to ask for support?

Regarding your issue, I see you are using django 1.9, this is not compatible with geonode 2.4, 1.6.11 is the supported version.

Also the geonode commands should not be ran as sudo and regarding the latest, manage.py is available only if you install it in custom installation. The command "geonode" is what you should use instead,

[manaetches]

Thanks for that,

I removed django from /usr/local/lib/python2.7/dist-packages

Then in installed django 1.6.11 using 'sudo pip install Django==1.6.11', to install the specific version.

I then ran 'geonode updatelayers', with and without GeoNode-2.4.egg-info. I got the following error: Traceback (most recent call last): File "/usr/local/bin/django-admin", line 5, in from pkg_resources import load_entry_point File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2749, in working_set = WorkingSet._build_master() File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 446, in _build_master return cls._build_from_requirements(requires) File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 459, in _build_from_requirements dists = ws.resolve(reqs, Environment()) File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 628, in resolve raise DistributionNotFound(req) pkg_resources.DistributionNotFound: Django==1.9.1

It looks like Django 1.9.1 is still being recognised.

Is there a way to safely remove/reinstall django 1.6.11 without affecting my geonode system?

Thanks Simone,

Note: I also posted this on geonode-users@lists.osgeo.org, as advised

[simod]

sudo pip uninstall django sudo pip install django==1.6.11

[manaetches]

Hi Simone,

I tried that. I even reinstalled geonode. I get the following when I run 'geonode updatelayers':

File "/usr/bin/django-admin.py", line 5, in management.execute_from_command_line() File "/usr/lib/python2.7/dist-packages/django/core/management/init.py", line 399, in execute_from_command_line utility.execute() File "/usr/lib/python2.7/dist-packages/django/core/management/init.py", line 392, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line 242, in run_from_argv self.execute(_args, _options.dict) File "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line 285, in execute output = self.handle(_args, _options) File "/usr/local/lib/python2.7/dist-packages/geonode/geoserver/management/commands/updatelayers.py", line 108, in handle remove_deleted=remove_deleted) File "/usr/local/lib/python2.7/dist-packages/geonode/geoserver/helpers.py", line 412, in gs_slurp resources = [k for k in resources if k.enabled in ["true", True]] File "/usr/lib/python2.7/dist-packages/geoserver/support.py", line 67, in getter self.fetch() File "/usr/lib/python2.7/dist-packages/geoserver/support.py", line 174, in fetch self.dom = self.catalog.get_xml(self.href) File "/usr/lib/python2.7/dist-packages/geoserver/catalog.py", line 214, in get_xml return parse_or_raise(content) File "/usr/lib/python2.7/dist-packages/geoserver/catalog.py", line 205, in parse_or_raise raise Exception(msg, e) Exception: ('GeoServer gave non-XML response for [GET http://192.168.137.7/geoserver/rest/workspaces/geonode/datastores/nu_bldexp_modelled/featuretypes/nu_bldexp_modelled.xml]:

\n nu_bldexp_modelled\n nu_bldexp_modelled\n \n geonode\n \n \n \n Contents of file\n The building exposure database includes a comprehensive inventory of residential, commercial, public and industrial buildings, consisting of their location, structural characteristics that affect the vulnerability to the effects of natural disasters, and replacement costs. Sophisticated methodologies were adopted for (1) identifying the spatial distribution of buildings in each country, (2) estimating building characteristics, and (3) assessing their unit replacement costs. Raw building datasets, which included approximately 80,000 field surveyed buildings across 11 countries, was critically analyzed and thoroughly processed to develop the building exposure database. The spatial location of buildings for all populated areas of each 15 PIC was assembled with a high level of resolution that balance accuracy and economy.The locations in this layer were (1) manually digitized from high-resolution satellite imagery but not field verified (location = verified, attributes modelled), (2) inferred using image processing techniques and/or census data (coverage: PG, FJ, KI, SB, TL, VU, FM, MH, and, to a lesser extent, SB, CK, TO, and TV, location = modelled and attributes = modelled), or (3) extracted from datasets acquired from government sources (location = verified, and attributes = modelled). Compiled by AIR Worldwide.\n \n Exposure\n \n \n \n text/xml\n other\n http://pcrafi.spc.int/catalogue/csw?outputschema=http%3A%2F%2Fwww.w3.org%2F2005%2FAtom&service=CSW&request=GetRecordById&version=2.0.2&elementsetname=full&id=8358df9a-dd47-11e2-bd87-0050569e003a\n \n \n text/xml\n other\n http://pcrafi.spc.int/catalogue/csw?outputschema=http%3A%2F%2Fgcmd.gsfc.nasa.gov%2FAboutus%2Fxml%2Fdif%2F&service=CSW&request=GetRecordById&version=2.0.2&elementsetname=full&id=8358df9a-dd47-11e2-bd87-0050569e003a\n \n \n text/xml\n other\n http://pcrafi.spc.int/catalogue/csw?outputschema=http%3A%2F%2Fwww.opengis.net%2Fcat%2Fcsw%2F2.0.2&service=CSW&request=GetRecordById&version=2.0.2&elementsetname=full&id=8358df9a-dd47-11e2-bd87-0050569e003a\n \n \n text/xml\n other\n http://pcrafi.spc.int/catalogue/csw?outputschema=urn%3Aoasis%3Anames%3Atc%3Aebxml-regrep%3Axsd%3Arim%3A3.0&service=CSW&request=GetRecordById&version=2.0.2&elementsetname=full&id=8358df9a-dd47-11e2-bd87-0050569e003a\n \n \n text/xml\n FGDC\n http://pcrafi.spc.int/catalogue/csw?outputschema=http%3A%2F%2Fwww.opengis.net%2Fcat%2Fcsw%2Fcsdgm&service=CSW&request=GetRecordById&version=2.0.2&elementsetname=full&id=8358df9a-dd47-11e2-bd87-0050569e003a\n \n \n text/xml\n TC211\n http://pcrafi.spc.int/catalogue/csw?outputschema=http%3A%2F%2Fwww.isotc211.org%2F2005%2Fgmd&service=CSW&request=GetRecordById&version=2.0.2&elementsetname=full&id=8358df9a-dd47-11e2-bd87-0050569e003a\n \n \n GEOGCS["GCS_WGS_1984", \n DATUM["D_WGS_1984", \n SPHEROID["WGS_1984", 6378137.0, 298.257223563]], \n PRIMEM["Greenwich", 0.0], \n UNIT["degree", 0.017453292519943295], \n AXIS["Longitude", EAST], \n AXIS["Latitude", NORTH]]\n EPSG:4326\n \n -170.27344606603856\n -169.45496693396143\n -19.353791674528924\n -18.739932325471077\n GEOGCS["GCS_WGS_1984", \n DATUM["D_WGS_1984", \n SPHEROID["WGS_1984", 6378137.0, 298.257223563]], \n PRIMEM["Greenwich", 0.0], \n UNIT["degree", 0.017453292519943295], \n AXIS["Longitude", EAST], \n AXIS["Latitude", NORTH]]\n \n \n -169.941328\n -169.787085\n -19.132905\n -18.960819\n GEOGCS["WGS84(DD)", \n DATUM["WGS84", \n SPHEROID["WGS84", 6378137.0, 298.257223563]], \n PRIMEM["Greenwich", 0.0], \n UNIT["degree", 0.017453292519943295], \n AXIS["Geodetic longitude", EAST], \n AXIS["Geodetic latitude", NORTH]]\n \n NONE\n true\n true\n \n false\n \n \n geonode:nu_bldexp_modelled\n \n \n 0\n 0\n false\n false\n false\n sudo pip uninstall django > sudo pip install django==1.6.11 > > — > Reply to this email directly or view it on GitHub > https://github.com/GeoNode/geonode/issues/2409#issuecomment-175107611.

[simod]

Not sure why but the xml returned by geoserver is not valid.

Try to reload the layer.

ps let's follow up on the lists from now on.

Regards

[manaetches]

OK, thanks again,

Will use lists going forward. Sorry to be a bother.

Mana

On Wed, Jan 27, 2016 at 10:02 PM, Simone Dalmasso notifications@github.com wrote:

Not sure why but the xml returned by geoserver is not valid.

Try to reload the layer.

ps let's follow up on the lists from now on.

Regards

— Reply to this email directly or view it on GitHub https://github.com/GeoNode/geonode/issues/2409#issuecomment-175523399.

[simod]

no bother at all :)

[manaetches]

Hi Simone,

I'd really like your guide on this if can.

Following the migration of data from 2.0 to 2.4 using https://github.com/capooti/geonode/blob/migration_from20_to_24/scripts/migrations/migrate20to24/index.rst I narrowed down the source of my problem. It appeared that the 'geonode_data' database was not being popupated with the layers. So I manually ran the 'migrate_layers.py' and got the following output:

Note: this is only part of the output but it is consistent with all 459 of my layers.

<class 'psycopg2.IntegrityError'> duplicate key value violates unique constraint "layers_layer_pkey" DETAIL: Key (resourcebase_ptr_id)=(408) already exists. select resourcebase_ptr_id, workspace, store, "storeType", name, typename, popul ar_count, share_count, default_style_id from layers_layer (206, 'geonode', 'fsm_w500_tr', 'coverageStore', 'fsm_w500_tr', 'geonode:fsm_w50 0_tr', 245, 0) Resource id 233 (VU_EQ_HazardMap_10_100 MRP) is now 160

<class 'psycopg2.IntegrityError'> duplicate key value violates unique constraint "layers_layer_pkey" DETAIL: Key (resourcebase_ptr_id)=(160) already exists. select resourcebase_ptr_id, workspace, store, "storeType", name, typename, popul ar_count, share_count, default_style_id from layers_layer (233, 'geonode', 'van_10_100t', 'coverageStore', 'van_10_100t', 'geonode:van10 100t', 226, 0) Resource id 199 (KI_Surface_Soil_Pt10) is now 344

<class 'psycopg2.IntegrityError'> duplicate key value violates unique constraint "layers_layer_pkey" DETAIL: Key (resourcebase_ptr_id)=(344) already exists. select resourcebase_ptr_id, workspace, store, "storeType", name, typename, popul ar_count, share_count, default_style_id from layers_layer (199, 'geonode', 'ki_soils_vs30_pt10', 'coverageStore', 'ki_soils_vs30_pt10', 'g eonode:ki_soils_vs30_pt10', 213, 0) Resource id 283 (MH_EQ_HazardMap_03_500 MRP) is now 126

<class 'psycopg2.IntegrityError'> duplicate key value violates unique constraint "layers_layer_pkey" DETAIL: Key (resourcebase_ptr_id)=(126) already exists. select resourcebase_ptr_id, workspace, store, "storeType", name, typename, popul ar_count, share_count, default_style_id from layers_layer (283, 'geonode', 'mar_03_500t', 'coverageStore', 'mar_03_500t', 'geonode:mar03 500t', 241, 0) Resource id 188 (TV_EQ_HazardMap_PGA_100 MRP) is now 79

duplicate key value violates unique constraint "layers_layer_pkey" DETAIL: Key (resourcebase_ptr_id)=(79) already exists. select resourcebase_ptr_id, workspace, store, "storeType", name, typename, popul ar_count, share_count, default_style_id from layers_layer (188, 'geonode', 'tuv_00_100t', 'coverageStore', 'tuv_00_100t', 'geonode:tuv00 100t', 235, 0) Traceback (most recent call last): File "./migrate_layers.py", line 18, in en_fields = utils.get_en_fields(id) File "/home/mana/geonode-migration_from20_to_24/scripts/migrations/migrate20to

     24/utils.py", line 168, in get_en_fields
dst_cur.execute("SELECT title, abstract, purpose, constraints_other,

supplem ental_information, distribution_description, data_quality_statement FROM base_re sourcebase WHERE id = %s" % id) psycopg2.ProgrammingError: column "none" does not exist LINE 1: ...ata_quality_statement FROM base_resourcebase WHERE id = None

I don't quite understand <class 'psycopg2.IntegrityError'>. Im hoping you can enlighten me on this.

Many thanks in advance,

Mana ^

On Wed, Jan 27, 2016 at 5:43 AM, Simone Dalmasso notifications@github.com wrote:

sudo pip uninstall django sudo pip install django==1.6.11

— Reply to this email directly or view it on GitHub https://github.com/GeoNode/geonode/issues/2409#issuecomment-175107611.