Open defvol opened 10 years ago
Hay que esperar a que ocurra un error nuevamente para evaluar los logs.
[admin@app-datos-gob-mx ~]$ sudo supervisorctl [sudo] password for admin: ckan_fetch_consumer FATAL Exited too quickly (process log may have details) ckan_gather_consumer RUNNING pid 437, uptime 58 days, 23:08:59 supervisor> exit
Log
[admin@app-datos-gob-mx ~]$ tail -n 100 /var/log/ckan/default/fetch_consumer.log
2014-08-06 02:30:46,921 WARNI [ckanext.datastore.plugin] Legacy mode active. The sql search will not be available.
2014-08-06 02:30:46,974 DEBUG [ckanext.harvest.model] Harvest tables defined in memory
2014-08-06 02:30:46,979 DEBUG [ckanext.harvest.model] Harvest tables already exist
2014-08-06 02:30:47,016 DEBUG [ckanext.spatial.model.package_extent] Spatial tables defined in memory
2014-08-06 02:30:47,032 DEBUG [ckanext.spatial.model.package_extent] Spatial tables already exist
2014-08-06 02:30:47,078 DEBUG [ckanext.harvest.queue] Fetch queue consumer registered
2014-08-06 02:30:47,080 INFO [ckanext.harvest.queue] Received harvest object id: 58dc7211-4b27-4b90-85cb-574a1b5d9e8b
2014-08-06 02:30:47,111 DEBUG [ckanext.dcat.harvesters] In DCATHarvester import_stage
2014-08-06 02:30:47,417 INFO [ckanext.dcat.harvesters] Updated dataset with id a7ad366d-8a96-493e-bc04-fe534b57325a
Traceback (most recent call last):
File "/usr/lib/ckan/default/bin/paster", line 9, in <module>
load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/script/command.py", line 104, in run
invoke(command, command_name, options, args[1:])
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/script/command.py", line 143, in invoke
exit_code = runner.run(args)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/paste/script/command.py", line 238, in run
result = self.command()
File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 136, in command
fetch_callback(consumer, method, header, body)
File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/queue.py", line 294, in fetch_callback
fetch_and_import_stages(harvester, obj)
File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/queue.py", line 317, in fetch_and_import_stages
obj.save()
File "/usr/lib/ckan/default/src/ckan/ckan/model/domain_object.py", line 62, in save
self.add()
File "/usr/lib/ckan/default/src/ckan/ckan/model/domain_object.py", line 66, in add
self.Session.add(self)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/sqlalchemy/orm/scoping.py", line 114, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/sqlalchemy/orm/session.py", line 1252, in add
self._save_or_update_state(state)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/sqlalchemy/orm/session.py", line 1268, in _save_or_update_state
self._save_or_update_impl(st_)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/sqlalchemy/orm/session.py", line 1493, in _save_or_update_impl
self._update_impl(state)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/sqlalchemy/orm/session.py", line 1485, in _update_impl
self._attach(state)
File "/usr/lib/ckan/default/lib/python2.6/site-packages/sqlalchemy/orm/session.py", line 1513, in _attach
% (mapperutil.state_str(state), state.key))
sqlalchemy.exc.InvalidRequestError: Can't attach instance <Package at 0x5840950>; another instance with key (<class 'ckan.model.package.Package'>, (u'a7ad366d-8a96-493e-bc04-fe534b57325a',)) is already present in this session.
en el fetch_consumer.log se puede ver:
return self.routing_key + ':' + message[self.routing_key]
TypeError: cannot concatenate 'str' and 'NoneType' objects
esto se parece mucho a un bug que tenía ckanext-spacial y que ya está arreglado https://github.com/ckan/ckanext-spatial/issues/70
el código de ckanext-dcat es similar https://github.com/ckan/ckanext-dcat/blob/master/ckanext/dcat/harvesters.py#L259
El harvester de SEDESOL y el de SCT se quedaron atorados en el fetch_queue y el log dice lo siguiente:
% (mapperutil.state_str(state), state.key)) sqlalchemy.exc.InvalidRequestError: Can't attach instance <Package at 0x579c8d0>; another instance with key (<class 'ckan.model.package.Package'>, (u'f4171abb-2dcc-4962-abdd-9d5ceba4c89d',)) is already present in this session.