[publication_list] allow repeated bibtex entries optionally

[bishesh]

I'm using multiple bibtex files to create list of publications (issue #261 ). One use case is that we have a team where each one of us maintain our own bibtex files. Some of us are coauthors in some papers leading to the same bibtex entries in multiple files (of coauthors).

This results in pybtex error; E.g. pybtex.database.BibliographyDataError: repeated bibliograhpy entry: Khanal_2012

Can we give user the option to allow or prevent compiling with repeated bibtex entries by choosing only one from those repeated entries? Maybe by default take unique entries from the list of all entries, and issue warning during compilation if the length of unique entries do not match with the length of all entries.

[xuhdev]

Are you sure they are in different files? This error should never occur if duplicated entries are in different files. Currently there is no warning in such case, but the error should also never emit.

[bishesh]

Yes definitely. Each of these files are working fine when entered individually; combining more than one when they do not have the same bibtex entries also works.

The complete error is below, if it helps:

[2018-04-01T08:51:48Z] ERROR: auto: Scanning posts........done!
.  render_sources:public/publications/index.rst
.  render_posts:timeline_changes
.  render_posts:cache/pages/publications.html
########################################
TaskError - taskid:render_posts:cache/pages/publications.html
PythonAction Error
Traceback (most recent call last):
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/doit/action.py", line 403, in execute
    returned_value = self.py_callable(*self.args, **kwargs)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/nikola/post.py", line 579, in compile
    lang)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/nikola/plugins/compile/rest/__init__.py", line 152, in compile
    output, error_level, deps, shortcode_deps = self.compile_string(data, source, is_two_file, post, lang)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/nikola/plugins/compile/rest/__init__.py", line 131, in compile_string
    no_title_transform=self.site.config.get('NO_DOCUTILS_TITLE_TRANSFORM', False))
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/nikola/plugins/compile/rest/__init__.py", line 352, in rst2html
    pub.publish(enable_exit_status=enable_exit_status)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/core.py", line 217, in publish
    self.settings)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/readers/__init__.py", line 72, in read
    self.parse()
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/__init__.py", line 191, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 171, in run
    input_source=document['source'])
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2753, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
    node=section_node, match_titles=True)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2753, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
    node=section_node, match_titles=True)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
    node=node, match_titles=match_titles)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2326, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2338, in explicit_construct
    return method(self, expmatch)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2081, in directive
    directive_class, match, type_name, option_presets)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2130, in run_directive
    result = directive_instance.run()
  File "/home/bk17/naamii/website/naamii.gitlab.io/plugins/publication_list/publication_list.py", line 102, in run
    all_entries.extend(parser.parse_file(a).entries.items())
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/pybtex/database/input/__init__.py", line 51, in parse_file
    self.parse_stream(f)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/pybtex/database/input/bibtex.py", line 385, in parse_stream
    return self.parse_string(text)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/pybtex/database/input/bibtex.py", line 380, in parse_string
    self.process_entry(entry_type, *entry[1])
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/pybtex/database/input/bibtex.py", line 347, in process_entry
    self.data.add_entry(key, entry)
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/pybtex/database/__init__.py", line 150, in add_entry
    report_error(BibliographyDataError('repeated bibliograhpy entry: %s' % key))
  File "/home/bk17/anaconda3/envs/nikola/lib/python3.6/site-packages/pybtex/errors.py", line 77, in report_error
    raise exception
pybtex.database.BibliographyDataError: repeated bibliograhpy entry: Khanal_2012

[xuhdev]

I made a guess fix 69375fdcc090dbc558a86c84857759f7927760ee . Can you test the new version?

[bishesh]

Yes this resolves the issue. It also solves issue #265 that I had opened just a while ago. Thanks!

[bishesh]

Sorry I was too quick to close this. Now the error is indeed gone. However, the built webpage has all these repeated entries being rendered as many times as they are repeated. This is not helpful because if three co-authors have the same entry in their respective bibtex files they maintain, we will see three times the same publication on the webpage.

[xuhdev]

It should now be fixed in 5cdd332ec440e7ede4c6e4a18437d7a1b10b1b4f

[bishesh]

Yes, thanks!