Closed dommitchell closed 7 years ago
A bit of an idea. It'll crystallise soon enough. cc @richard-jones @Steven-Eardley did we not have to deal with a similar error on article pages before?
If not we'll investigate more thoroughly (though I think we have enough info to replicate). Below is the error log, and below that, the article's JSON record in Elasticsearch so we can replicate locally.
Looks like we might need to escalate that job which will refresh article metadata based on the journal info - this article has issns so it's reachable via a journal ToC and the discovery API, but has no "journal" section in the JSON. This is I believe planned as part of the Harvester project since it deals with refreshing article metadata automatically too. Nonetheless, to fix this we could create a separate script from the current portality/scripts/journalinfo.py which looks for articles without a "journal" section or which have a "journal" but no "journal.bibjson" (just in case) - and updates them. Let me know if you have any thoughts about doing it differently or if I'm wrong about the cause of this.
2015-12-09 13:52:28,556 - portality.core - ERROR - Exception on /article/000c38480c3d4d1383444cf8b24bc85d [GET]
Traceback (most recent call last):
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/app.py", line 1687, in wsgi_app
response = self.full_dispatch_request()
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/app.py", line 1360, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/app.py", line 1358, in full_dispatch_request
rv = self.dispatch_request()
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/app.py", line 1344, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/cloo/repl/production/doaj/src/doaj/portality/view/doaj.py", line 278, in article_page
return render_template('doaj/article.html', article=article, journal=journal)
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/templating.py", line 125, in render_template
context, ctx.app)
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/templating.py", line 107, in _render
rv = template.render(context)
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/jinja2/environment.py", line 989, in render
return self.environment.handle_exception(exc_info, True)
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/jinja2/environment.py", line 754, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/cloo/repl/production/doaj/src/doaj/portality/templates/doaj/article.html", line 1, in top-level template code
{% extends "base.html" %}
File "/home/cloo/repl/production/doaj/src/doaj/portality/templates/base.html", line 21, in top-level template code
{% block extra_meta_tags %}{% endblock %}
File "/home/cloo/repl/production/doaj/src/doaj/portality/templates/doaj/article.html", line 18, in block "extra_meta_tags"
{% set jbib = journal.bibjson() %}
UndefinedError: 'None' has no attribute 'bibjson'
{
"_index":"doaj",
"_type":"article",
"_id":"000c38480c3d4d1383444cf8b24bc85d",
"_version":2,
"found":true,
"_source":{
"index":{
"publisher":[
"Centro de Estudos de Venenos e Animais Pe\u00e7onhentos - CEVAP, Universidade Estadual Paulista - UNESP"
],
"schema_subject":[
"LCC:Medicine",
"LCC:Zoology",
"DOAJ:Health Sciences",
"LCC:Internal medicine",
"DOAJ:Biology and Life Sciences",
"LCC:Arctic medicine. Tropical medicine",
"DOAJ:Medicine (General)",
"DOAJ:Internal medicine",
"DOAJ:Biology",
"DOAJ:Zoology",
"LCC:Special situations and conditions",
"LCC:Science"
],
"license":[
"CC by-nc"
],
"classification":[
"Internal medicine",
"Biology and Life Sciences",
"Biology",
"Health Sciences",
"Special situations and conditions",
"Science",
"Medicine (General)",
"Arctic medicine. Tropical medicine",
"Zoology",
"Medicine"
],
"classification_paths":[
"Medicine: Internal medicine: Special situations and conditions: Arctic medicine. Tropical medicine",
"Science: Zoology"
],
"country":"Brazil",
"issn":[
"1678-4936",
"0104-7930"
],
"unpunctitle":"PRELIMINARY INVESTIGATION ABOUT THE ANTICOMPLEMENT PROPERTIES OF PARTIALLY SEPARATED Vipera ammodytes VENOM",
"language":[
"English"
],
"date":"1997-01-01T00:00:00Z",
"schema_code":[
"LCC:RC955-962",
"LCC:RC952-1245",
"LCC:RC31-1245",
"LCC:R",
"LCC:Q",
"LCC:QL1-991"
],
"asciiunpunctitle":"PRELIMINARY INVESTIGATION ABOUT THE ANTICOMPLEMENT PROPERTIES OF PARTIALLY SEPARATED Vipera ammodytes VENOM",
"subject":[
"Internal medicine",
"Biology and Life Sciences",
"Biology",
"Health Sciences",
"Special situations and conditions",
"Science",
"Medicine (General)",
"Arctic medicine. Tropical medicine",
"Zoology",
"Medicine"
]
},
"last_updated":"2015-04-22T01:21:44Z",
"created_date":"2013-08-31T05:26:16Z",
"admin":{
"in_doaj":true
},
"id":"000c38480c3d4d1383444cf8b24bc85d",
"bibjson":{
"start_page":"37",
"title":"PRELIMINARY INVESTIGATION ABOUT THE ANTI-COMPLEMENT PROPERTIES OF PARTIALLY SEPARATED Vipera ammodytes VENOM",
"journal":{
"publisher":"Centro de Estudos de Venenos e Animais Pe\u00e7onhentos - CEVAP, Universidade Estadual Paulista - UNESP",
"license":[
{
"type":"CC by-nc",
"title":"CC by-nc"
}
],
"language":[
"English"
],
"title":"Journal of Venomous Animals and Toxins including Tropical Diseases",
"country":"BR",
"number":"1",
"volume":"3"
},
"author":[
{
"name":"S. STANILOVA"
},
{
"name":"Z. ZHELEV"
},
{
"name":"I. MATEV"
}
],
"link":[
{
"url":"http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-79301997000100005",
"type":"fulltext"
}
],
"year":"1997",
"keywords":[
"Anti-complement",
"venom",
"Vipera ammodytes"
],
"identifier":[
{
"type":"doi",
"id":"10.1590/S0104-79301997000100005"
},
{
"type":"pissn",
"id":"0104-7930"
},
{
"type":"eissn",
"id":"1678-4936"
}
],
"abstract":"The present study demonstrates that Vipera ammodytes venom is a potent inhibitor of human complement activation. Crude venom was fractionated by ion-exchange chromatography and each fraction was then tested for proteolytic, hemolytic and anti-complement activities. Three peaks decreased the hemolytic activity of human serum when measured using assays for the classical and alternative pathways of activation. One of the anti-complement peaks contains a protease capable of cleaving the C3 component of complement.",
"subject":[
{
"code":"RC955-962",
"term":"Arctic medicine. Tropical medicine",
"scheme":"LCC"
},
{
"code":"RC952-1245",
"term":"Special situations and conditions",
"scheme":"LCC"
},
{
"code":"RC31-1245",
"term":"Internal medicine",
"scheme":"LCC"
},
{
"code":"R",
"term":"Medicine",
"scheme":"LCC"
},
{
"term":"Internal medicine",
"scheme":"DOAJ"
},
{
"term":"Medicine (General)",
"scheme":"DOAJ"
},
{
"term":"Health Sciences",
"scheme":"DOAJ"
},
{
"code":"QL1-991",
"term":"Zoology",
"scheme":"LCC"
},
{
"code":"Q",
"term":"Science",
"scheme":"LCC"
},
{
"term":"Zoology",
"scheme":"DOAJ"
},
{
"term":"Biology",
"scheme":"DOAJ"
},
{
"term":"Biology and Life Sciences",
"scheme":"DOAJ"
},
{
"scheme":"LCC",
"term":"Arctic medicine. Tropical medicine",
"code":"RC955-962"
},
{
"scheme":"LCC",
"term":"Special situations and conditions",
"code":"RC952-1245"
},
{
"scheme":"LCC",
"term":"Internal medicine",
"code":"RC31-1245"
},
{
"scheme":"LCC",
"term":"Medicine",
"code":"R"
},
{
"term":"Internal medicine",
"scheme":"DOAJ"
},
{
"term":"Medicine (General)",
"scheme":"DOAJ"
},
{
"term":"Health Sciences",
"scheme":"DOAJ"
},
{
"scheme":"LCC",
"term":"Zoology",
"code":"QL1-991"
},
{
"scheme":"LCC",
"term":"Science",
"code":"Q"
},
{
"term":"Zoology",
"scheme":"DOAJ"
},
{
"term":"Biology",
"scheme":"DOAJ"
},
{
"term":"Biology and Life Sciences",
"scheme":"DOAJ"
},
{
"code":"RC955-962",
"term":"Arctic medicine. Tropical medicine",
"scheme":"LCC"
},
{
"code":"RC952-1245",
"term":"Special situations and conditions",
"scheme":"LCC"
},
{
"code":"RC31-1245",
"term":"Internal medicine",
"scheme":"LCC"
},
{
"code":"R",
"term":"Medicine",
"scheme":"LCC"
},
{
"term":"Internal medicine",
"scheme":"DOAJ"
},
{
"term":"Medicine (General)",
"scheme":"DOAJ"
},
{
"term":"Health Sciences",
"scheme":"DOAJ"
},
{
"code":"QL1-991",
"term":"Zoology",
"scheme":"LCC"
},
{
"code":"Q",
"term":"Science",
"scheme":"LCC"
},
{
"term":"Zoology",
"scheme":"DOAJ"
},
{
"term":"Biology",
"scheme":"DOAJ"
},
{
"term":"Biology and Life Sciences",
"scheme":"DOAJ"
},
{
"scheme":"LCC",
"term":"Arctic medicine. Tropical medicine",
"code":"RC955-962"
},
{
"scheme":"LCC",
"term":"Special situations and conditions",
"code":"RC952-1245"
},
{
"scheme":"LCC",
"term":"Internal medicine",
"code":"RC31-1245"
},
{
"scheme":"LCC",
"term":"Medicine",
"code":"R"
},
{
"term":"Internal medicine",
"scheme":"DOAJ"
},
{
"term":"Medicine (General)",
"scheme":"DOAJ"
},
{
"term":"Health Sciences",
"scheme":"DOAJ"
},
{
"scheme":"LCC",
"term":"Zoology",
"code":"QL1-991"
},
{
"scheme":"LCC",
"term":"Science",
"code":"Q"
},
{
"term":"Zoology",
"scheme":"DOAJ"
},
{
"term":"Biology",
"scheme":"DOAJ"
},
{
"term":"Biology and Life Sciences",
"scheme":"DOAJ"
},
{
"code":"RC955-962",
"term":"Arctic medicine. Tropical medicine",
"scheme":"LCC"
},
{
"code":"RC952-1245",
"term":"Special situations and conditions",
"scheme":"LCC"
},
{
"code":"RC31-1245",
"term":"Internal medicine",
"scheme":"LCC"
},
{
"code":"R",
"term":"Medicine",
"scheme":"LCC"
},
{
"code":"QL1-991",
"term":"Zoology",
"scheme":"LCC"
},
{
"code":"Q",
"term":"Science",
"scheme":"LCC"
},
{
"scheme":"LCC",
"term":"Arctic medicine. Tropical medicine",
"code":"RC955-962"
},
{
"scheme":"LCC",
"term":"Special situations and conditions",
"code":"RC952-1245"
},
{
"scheme":"LCC",
"term":"Internal medicine",
"code":"RC31-1245"
},
{
"scheme":"LCC",
"term":"Medicine",
"code":"R"
},
{
"scheme":"LCC",
"term":"Zoology",
"code":"QL1-991"
},
{
"scheme":"LCC",
"term":"Science",
"code":"Q"
}
]
}
}
}
I was also reminded of the metadata migration that you carried out for me where we standardised on the licenses. #241 #242 I think this may be related.
I've put in a PR which might well fix this...
@richard-jones did this go live?
Assuming @richard-jones meant commit https://github.com/DOAJ/doaj/commit/0165de1e872a9ca333c37b54e4f483b6159ac992, yes, his changes are live. Are you still seeing the 500 error?
Yep, no change: https://doaj.org/article/000c38480c3d4d1383444cf8b24bc85d
Error is (still)
2016-01-04 14:27:41,565 - portality.core - ERROR - Exception on /article/000c38480c3d4d1383444cf8b24bc85d [GET]
Traceback (most recent call last):
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/app.py", line 1687, in wsgi_app
response = self.full_dispatch_request()
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/app.py", line 1360, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/app.py", line 1358, in full_dispatch_request
rv = self.dispatch_request()
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/app.py", line 1344, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/cloo/repl/production/doaj/src/doaj/portality/view/doaj.py", line 278, in article_page
return render_template('doaj/article.html', article=article, journal=journal)
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/templating.py", line 125, in render_template
context, ctx.app)
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/flask/templating.py", line 107, in _render
rv = template.render(context)
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/jinja2/environment.py", line 989, in render
return self.environment.handle_exception(exc_info, True)
File "/home/cloo/repl/production/doaj/local/lib/python2.7/site-packages/jinja2/environment.py", line 754, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/cloo/repl/production/doaj/src/doaj/portality/templates/doaj/article.html", line 1, in top-level template code
{% extends "base.html" %}
File "/home/cloo/repl/production/doaj/src/doaj/portality/templates/base.html", line 21, in top-level template code
{% block extra_meta_tags %}{% endblock %}
File "/home/cloo/repl/production/doaj/src/doaj/portality/templates/doaj/article.html", line 18, in block "extra_meta_tags"
{% set jbib = journal.bibjson() %}
UndefinedError: 'None' has no attribute 'bibjson'
I believe in this case the problem might be that despite @richard-jones's relevant fix, the data on the article record is simply out of date.
The article references a journal that does not exist - see this extract from the article record I pasted above:
"issn":[
"1678-4936",
"0104-7930"
]
Neither of these ISSNs exist in the catalogue, not even in the administrative site search. What likely happened was that either:
I believe the correct journal (judging by the title, which we thankfully record on the article itself) is https://doaj.org/toc/1678-9199 . There seem to be 253 articles affected by the non-existent ISSNs.
Shall we simply mass-update them? So we take all articles with those ISSNs and that journal title and replace the ISSNs in them with "1678-9199" which seems to be the right ISSN. It'll take .. probably about 30-60 min to formulate a query to get them, update each and save, and deal with any potential problems during that process.
Shall we simply mass-update them? So we take all articles with those ISSNs and that journal title and replace the ISSNs in them with "1678-9199" which seems to be the right ISSN.
Now I see that, I recall the work is already scheduled - I think this is part of my task for #941, which I'm doing imminently.
Added this to the harvester milestone, as it'll be fixed when the journal info script is fixed.
@Steven-Eardley can this one be closed now?
I'm still seeing the issue after the release, and running the script that should fix it. Needs to be investigated some more.
These articles appear to be gone now, the link at the top leads to no results, so I'm closing this off.
Are we syncing the journal info present in article records now? (I think we are, article_cleanup_sync
type jobs) If so, that'll fix it anyway.
http://bit.ly/1OT479U
These are all article of the license type 'CC by-nc' which is an old license type and shouldn't be on the site. Note that there are 2 CC BY-NC listing under license now:
CC BY-NC (350137) CC by-nc (2619)
Any idea what is going on?