plk / biblatex

biblatex is a sophisticated bibliography system for LaTeX users. It has considerably more features than traditional bibtex and supports UTF-8
518 stars 118 forks source link

Datetime. Missing frills and datetime output, authoryear style, and mergedate. Bugs. #520

Closed JohnLukeBentley closed 7 years ago

JohnLukeBentley commented 7 years ago

Some of the datetime frills (approximate, uncertain, eras) break in the authoryear style when mergedate is other than the default ("compact" or "true").

With ....

alldates=ymd, 
labeldate=year,
mergedate=compact, 

... then output is as expected (bibliographic output in document, not alphabetic, order):

  • Alberici, Emma (2016). "Why Did 61 Million Americans Vote for Trump?" In: ABC News
  • ...
  • Kennett, Jeanette (2008). "True and Proper Selves: Velleman on Love". In: Ethics 118.2, pp. 213-227. issn: 0014-1704. doi: 10.1086/523747.
  • OED Online (2016). OED Online. Oxford: Oxford University Press.
  • ...
  • SEP (2016). The Stanford Encyclopedia of Philosophy. Ed. by Edward N. Zalta
  • Aquinas, Thomas (ca. 1273?). Compendium of Theology.
  • Plato (ca. 0380 BCE). Republic. Trans. by C. D. C. Reeve. 3rd edition. Indianapolis: Hackett Publishing Company, Inc. 392 pp. isbn: 0-87220-737-4.

With

alldates=ymd, 
labeldate=year,
mergedate=basic, 

... then output breaks:

  • Alberici, Emma (2016). "Why Did 61 Million Americans Vote for Trump?" In: ABC News (2016- 11-20 21:02:28 +11:00).
  • ...
  • Kennett, Jeanette (2008). "True and Proper Selves: Velleman on Love". In: Ethics 118.2 (2008-01), pp. 213-227. issn: 0014-1704. doi: 10.1086/523747.
  • OED Online (2016). OED Online. Oxford: Oxford University Press, 2016-12. ...
  • SEP (2016). The Stanford Encyclopedia of Philosophy. Ed. by Edward N. Zalta.
  • Aquinas, Thomas (1273). Compendium of Theology.
  • Plato (0380). Republic. Trans. by C. D. C. Reeve. 3rd edition. Indianapolis: Hackett Publishing Company, Inc. 392 pp. isbn: 0-87220-737-4.

This is broken in three ways:

  1. The datetime frills (approximate, uncertain, eras) are absent from the first date; and
  2. Where a bibliographic entry has a date field with datetime precision finer than a year the fulldate time is sometimes not printed out as a second date. Here the entry for Alberici is correctly printout out, but for SEP there is no second date supplying the season. I'd expect:

    SEP (2016). The Stanford Encyclopedia of Philosophy. Ed. by Edward N. Zalta. (2016 Autumn)

    Perhaps this is only an issue with seasons.

  3. Where a bibliographic entry has a date field with datetime precision of a month the second date sometimes is surrounded by parentheses (as in Kennett, an article type), sometimes not (as in OED online, a book type).

Request fixing this up for mergedate=basic, if not all the other mergedate options (At the moment I'm personally only intrested in mergedate=basic and mergedate=compact).

In the following test files only the entries for Aquinas and Plato are directly relevant. However I supply other entries of varying datetime precision, as a guard against regression.

Gist "Biblatex-Bug-Mergedate" (Edit 04: note to self these are locally renamed Biblatex-Tester-EdtfBasics.*):

Edit01:

Edit02:

Edit03:

"Edit 04: note to self these are locally renamed Biblatex-Tester-EdtfBasics.*"

moewew commented 7 years ago

Unfortunately I'm not really well-versed in the new date-time features, so I can only make tentative suggestions.

If I understand things correctly, we need to replace (all?/most? occurrences of) the two simple lines

\printfield{labelyear}%
\printfield{extrayear}

by the much longer and harder to understand

\ifdefstring\blx@dateformat@labeldate{edtf}
  {}
  {\datecircaprint}%
\dateeraprintpre{labelyear}%
\printfield{labelyear}%
\printfield{extrayear}%
\dateuncertainprint%
\iffieldsequal{labeldateera}{labelenddateera}
  {}
  {\dateeraprint{labelyear}}%
\ifdefstring\blx@dateformat@labeldate{edtf}
  {\datecircaprintedtf}
  {}%
\iffieldundef{labelendyear}
  {}
  {\iffieldsequal{labelyear}{labelendyear}
     {}
     {\ifdefstring\blx@dateformat@labeldate{edtf}
        {\slash}% strict EDTF
        {\bibdaterangesep
         \enddatecircaprint}%
      \dateeraprintpre{labelendyear}%
      \printfield{labelendyear}%
      \enddateuncertainprint
      \ifdefstring\blx@dateformat@labeldate{edtf}
        {\enddatecircaprintedtf}
        {}%
      \dateeraprint{labelendyear}}}

@PLK Do you think it could be possible to conflate the monstrous 28 lines above that have to be repeated over and over again in all sorts of places into one more comfortable standard command? Earlier it was fairly neat to have only the two lines in all sorts of places, but having to have the 20-odd lines everywhere makes the code just look terribly complicated.

Additionally, in all \ifboolexpr we probably want to replace

      test {\iffieldundef{month}}

by

      test {\iffieldundef{season}}
      and
      test {\iffieldundef{month}}

Plus one

\iffieldundef{month}

needs to be

    \ifboolexpr{
      test {\iffieldundef{season}}
      and
      test {\iffieldundef{month}}
    }

This should address your issues (1) and (2). (3) is expected behaviour, dates for @article types always go in brackets, while those for other entry types normally go bare.

I have uploaded a amended authoryear.bbx at gist: https://gist.github.com/moewew/fe3baef052d16872ae5a5226de463064 You can use it to test my suggestions.

moewew commented 7 years ago

@JohnLukeBentley Did you have time to take a look at my suggested authoryear.bbx (https://gist.github.com/moewew/fe3baef052d16872ae5a5226de463064). Could you run some tests and tell us whether it worked for you?

JohnLukeBentley commented 7 years ago

Well there's a coincidence. Notification of your original detailed response didn't properly end up on my to do list. I happened to rediscover this thread a few days ago (I'd been away from biblatex in general). I had started testing your gist then; and have just sat down to do some more testing on it.

The short of it is:

moewew commented 7 years ago

No hurries.

JohnLukeBentley commented 7 years ago

Part way through my testing some issues arise.

New test files

I'm currently using the following testing files that are a bit more "minimal" (as in "Minimal Working Example") and hopefully easier to work with ...

The problem

With the following options (as well as the rest of the options as set in, in Biblatex-Tester-BasicExamples.tex) ...

alldates=ymd, % year, short, long, terse, comp, ymd, edtf. ymd = Year-Month-Day format
labeldate=year, % year, short, long, terse, comp, ymd, edtf. ymd = Year-Month-Day format
alldatesusetime=true,
labeldateusetime=false,

... Issue (1: eras, approximates, uncertainties) is solved for all combinations of mergedate. You mentioned issue (3: parenthesis presence around dates) is by design, so I'm happy to ignore that (at least for now). Issue (2: datetime handling, for date precision finer than a year) looks good for mergedate = false, minimum, and basic.

However, for compact, true and maximum there might be a problem.

To examine the problem note, firstly, the relevant output when mergedate=false, minimum, and basic is: identical and expected ...

2.2 References
Simpson, Lisa (2016a). Time, unspecified time zone. 2016-07-18 10:26.
Barker, Anne (2016). "Time, local time zone". In: (2016-06-06 18:20 +10:00).
Simpson, Lisa (2016b). Time, zulu time zone. 2016-07-18 20:26 Z.
Kennett, Jeanette (2008). "Month precision date". In: (2008-01).
Mendelsohn, Daniel (2010). "Day precision date." In: (2010-01-25).
SEP (2016). Season precision. Autumn 2016.

However, when mergedate=compact we get ...

2.2 References
Simpson, Lisa (2016a). Time, unspecified time zone.
Barker, Anne (2016). "Time, local time zone". In:
Simpson, Lisa (2016b). Time, zulu time zone.
Kennett, Jeanette (2008). "Month precision date". In:
Mendelsohn, Daniel (2010). "Day precision date." In:
SEP (2016). Season precision.

... which is not necessarily what we'd expect: the datetime precisions are lost entirely. (The lonely "In:"s can be ignored as merely an artifact of unrealistically spartan bib data).

I suppose for compact, true, and maximum we'd expect the same output as for false, minimum, and basic, under the previously highlighted options. That is, for the previously highlighted options, and for datetimes with precision finer than a year, mergedate should have no effect.

For further comparison, the text in 50-style-authoryear-biber.pdf goes ...

mergedate=compact merges all date specifications with the date labels. It will still treat the issue field specially:

Doe, John (2000). Book 1. Location: Publisher.
Doe, John (2003a). Book 2. Location: Publisher.
Doe, John (2003b). Book 3. Location: Publisher.
Doe, John (June 2006a). “Article 1”. In: Monthly Journal 25.6, pp. 70–85.
Doe, John (2006b). “Article 2”. In: Quarterly Journal 14.3 (Fall), pp. 5–25.

... in 50-style-authoryear-biber.pdf when mergedate=false we get ...

Doe, John (2000). Book 1. Location: Publisher, 2000

When mergedate was originally designed (by you? plk?) only years and issues had to be coped with. So it makes sense that the second instance of the string "2000" should be thrown away for mergedate=compact. But not when the label date, e.g. "(2016)", is different from the date at the end, e.g. " (2016-06-06 18:20 +10:00)".

Note, however, with the following options ...

alldates=ymd, % year, short, long, terse, comp, ymd, edtf. ymd = Year-Month-Day format
labeldate=ymd, % year, short, long, terse, comp, ymd, edtf. ymd = Year-Month-Day format
alldatesusetime=true, % print time components in non-compact date ranges
labeldateusetime=true,

... and mergedate=compact, everything is as expected ...

2.2 References
Simpson, Lisa (2016a-07-18 10:26). Time, unspecified time zone.
Barker, Anne (2016-06-06 18:20 +10:00). "Time, local time zone". In:
Simpson, Lisa (2016b-07-18 20:26 Z). Time, zulu time zone.
Kennett, Jeanette (2008-01). "Month precision date". In:
Mendelsohn, Daniel (2010-01-25). "Day precision date." In:
SEP (Autumn 2016). Season precision

Our Goal

So for datetimes finer than a year it seems that what we need is ...

My analytical powers are not with me today (I'm a bit tired) so I'd be relying on you @moewew (and/or @plk) to double check that's the best way to analyse the problem and formulate "Our goal". mergedate certainly gets tricky very quickly.

moewew commented 7 years ago

Thanks for taking time to test the suggested code. I hope to have a closer look at this over the weekend. I also noticed that one could get undesirable output if labeldate is set to year as in your example, while the actual date has higher precision. We need to find a solution that does not break backwards-compatibility and ideally does not require a major re-write of the macros involved.

JohnLukeBentley commented 7 years ago

Yeah we don't wont to break backward compatibility. I haven't thought the following through but, to avoid a major re-write, we might want to say that some combinations of options will not be supported by mergedate, for datetime precisions finer than a year.

Feel free to pass the ball back to me after having a closer look at it but before coding, if you doesn't all fall into place for you. I probably should have written the above more clearly: to properly enumerate what we expect for various options and data. Properly describe the test cases, that is, that the code needs to target. Not that I'm better able to do this than you (it's probably quite the reverse), but as a matter of properly sharing in the duties.

In any event: no hurries back at you.

moewew commented 7 years ago

Note that since mergedate=true is an alias for mergedate=compact, we only need to deal with compact and maximum.

The problem is that even currently we are breaking backwards compatibility for dates with month precision (only) and the default mergedate=compact - the month is not shown. I'm beginning to think this is really tricky. See also https://github.com/plk/biblatex/issues/148

moewew commented 7 years ago

@plk Any comments here? It is clear that mergedate is in a bit of a fix at the moment. I could more or less easily fix some of the problems here. But a few others are more delicate.

I would like to ask again about the long and complicated code

\ifdefstring\blx@dateformat@labeldate{edtf}
  {}
  {\datecircaprint}%
\dateeraprintpre{labelyear}%
\printfield{labelyear}%
\printfield{extrayear}%
\dateuncertainprint%
\iffieldsequal{labeldateera}{labelenddateera}
  {}
  {\dateeraprint{labelyear}}%
\ifdefstring\blx@dateformat@labeldate{edtf}
  {\datecircaprintedtf}
  {}%
\iffieldundef{labelendyear}
  {}
  {\iffieldsequal{labelyear}{labelendyear}
     {}
     {\ifdefstring\blx@dateformat@labeldate{edtf}
        {\slash}% strict EDTF
        {\bibdaterangesep
         \enddatecircaprint}%
      \dateeraprintpre{labelendyear}%
      \printfield{labelendyear}%
      \enddateuncertainprint
      \ifdefstring\blx@dateformat@labeldate{edtf}
        {\enddatecircaprintedtf}
        {}%
      \dateeraprint{labelendyear}}}

that replaces

\printfield{labelyear}%
\printfield{extrayear}

Would it be possible to conflate the former into a new macro? I seems to recall you argued that this is a style-specific thing and that we don't need a macro for it. But seeing that the code needs to be replicated at several places, I really think that it would be a good idea to make it into a macro.

Ideally I would like it to come from \printlabeldateextra directly.

plk commented 7 years ago

Well, I can't remember exactly but if you want to try branching and change it, I can run the test suite for the dates as that does test most of the date formats.

moewew commented 7 years ago

@plk I have #593 to fix the easy parts. As you can see the number of lines explode and we have a lot of redundancy. I'm quite unhappy about the fact that we had to copy the lines from authoryear.cbx in verbose.

The other things are much more complicate as has emerged here and requires thinking about expected output, as well as how to achieve it.

moewew commented 7 years ago

@plk I had another look at the more complicated bits of mergedate.

For false, minimum and basic it should be enough to use \printlabeldateextra directly (certainly with labeldate=year).

If labeldate is not year, however, we should probably do something about minimum and basic as we would have to merge dates if \printlabeldateextra has the same precision as \printdate - at least if we follow the documentation in 50-style-authoryear to then letter.

For compact and minimum things are a bit more complicated. mergedate really only makes sense if date is the source of labeldate. So I came up with a new check for that and worked under the assumption that mergedate should only do its thing if it is the new check is true.

Have a look at https://github.com/moewew/biblatex/commit/aa2e0a50e0e02b0fd1e1f6a86402502ad609b42c for a first attempt.

moewew commented 7 years ago

@JohnLukeBentley Finally I found some time to have a look at this. We have committed a bit of code to deal with mergedate and citations. It would be great if you could spare a bit of time, test the new stuff and tell us what you think. I don't think I was able to follow all your suggestions, but I hope that it addresses some of the concerns you have raised.

JohnLukeBentley commented 7 years ago

@moewew Thanks for doing that. Life has me busy at the moment. But I hope to get back to it shortly (leaving "shortly" undefined for the moment).

moewew commented 7 years ago

No need to hurry. It would have been nice to have another pair of eyes checking the output before we release a new version. But things should be OK now. And if you have complaints after we release then we can look at getting it into the next release.

Upon re-reading I think that what was implemented in the end is not exactly what you suggested. But it should allow one to obtain sensible output with all most combinations of options while at the same time retaining as much backwards compatibility as possible.

JohnLukeBentley commented 7 years ago

After some testing it looks like you folk have solved the original test cases. Some mighty fine work there!

However, that has enabled me to see further and to throw in another test series, for which the current code fails (at least by my lights ... you may have good reasons to not want to accomodate the new test series).

I've run an initial test with:

The original issues were:

  1. The datetime frills (approximate, uncertain, eras) are absent from the first date; and

That looks addressed.

  1. Where a bibliographic entry has a date field with datetime precision finer than a year the fulldate time is sometimes not printed out as a second date. Here the entry for Alberici is correctly printout out, but for SEP there is no second date supplying the season. I'd expect:

    SEP (2016). The Stanford Encyclopedia of Philosophy. Ed. by Edward N. Zalta. (2016 Autumn)

Perhaps this is only an issue with seasons.

I see from testing that this has been at least partially sucessfully addressed. However, it might be better, now, to understand this issue in more general terms: that for datetime precisions finer than a year the citation and bibliographic output is consistent with the spirit of the various mergedate values (false, minimum, etc.). I take that to be consistent with what you, @moewew, (and we) have been aiming at anyway. moewew wrote ...

But it should allow one to obtain sensible output with all most combinations of options while at the same time retaining as much backwards compatibility as possible.

.

  1. Where a bibliographic entry has a date field with datetime precision of a month the second date sometimes is surrounded by parentheses (as in Kennett, an article type), sometimes not (as in OED online, a book type).

3 was not tested. As you previously mentioned the original results I received were by design (based on entry type). I've just taken your word on this without much thinking it through.

I also haven't placed much attention to the nuances around the issue field (which is the only thing, for example, that differentiates mergedate=compact from mergedate=maximum, as far as I know).

For the results that follow, numerals that correspond to the above issues are listed in brackets. Where a numeral is:

Test Series 1

Using ...

%      alldates=ymd,
%      labeldate=year,
%      alldatesusetime=true,
%      labeldateusetime=false,

Varying the value for mergedate I get ...

false (1,2), minimum (1,2), basic (1,2), compact (1,2), true (1,2), maximum (1,2).

That is, everything passes.

I know "true" is an alias for "compact": I test it anyway.

Test Series 2

%      alldates=ymd,
%      labeldate=ymd,
%      alldatesusetime=true,
%      labeldateusetime=false,

Varying the value for mergedate I get ...

false (1,2), minimum (1,2), basic (1,2), compact (1,2), true (1,2), maximum (1,2)

That is, everything passes.

Intermediate conclusion

In short, for the original test cases: the absent approximate, uncertain, eras issue (1); and the issue of the proper handling of precisions finer than a year (2): that looks all fixed.

Highlights

Conventional

I'd like to highlight some significant combinations of values and demonstrate how well the current mergedate coding works for precisions finer than a year (for the origingal test cases).

%      alldates=ymd,
%      labeldate=year,
%      alldatesusetime=true,
%      labeldateusetime=false,
%      mergedate=basic,

That combination probably expresses a conventional citation/bibliography format. That yields something sensible:

2.1 Citations

Times:
Lorem (Simpson 2016a). Time, unspecifed time zone.
Lorem (Barker 2016). “Time, local time zone”.
Lorem (Simpson 2016b). Time, zulu time zone.

Precision fner than a year - Dates:
Lorem (Kennett 2008). “Month precision date”.
Lorem (Mendelsohn 2010). “Day precision date.”

Precision fner than a year - Season:
Lorem (SEP 2016). Season precision.

2.2 References

Simpson, Lisa (2016a). Time, unspecifed time zone. 2016-07-18 10:26.
Barker, Anne (2016). “Time, local time zone”. In: (2016-06-06 18:20 +10:00).
Simpson, Lisa (2016b). Time, zulu time zone. 2016-07-18 20:26 Z.
Kennett, Jeanette (2008). “Month precision date”. In: (2008-01).
Mendelsohn, Daniel (2010). “Day precision date.” In: (2010-01-25).
SEP (2016). Season precision. Autumn 2016.

All that supports a beautiful conventional format. Time details can be readily found a the end of each bibilographic entry.

Day level precision

If you wanted to have a quirky day level precision in the citation and bibliography (something I've been pushing for) you could use:

%      alldates=ymd,
%      labeldate=ymd,
%      alldatesusetime=true,
%      labeldateusetime=false,
%      mergedate=basic,

... yielding ...

2.1 Citations

Times:
Lorem (Simpson 2016a-07-18). Time, unspecifed time zone.
Lorem (Barker 2016-06-06). “Time, local time zone”.
Lorem (Simpson 2016b-07-18). Time, zulu time zone.

Precision fner than a year - Dates:
Lorem (Kennett 2008-01). “Month precision date”.
Lorem (Mendelsohn 2010-01-25). “Day precision date.”

Precision fner than a year - Season:
Lorem (SEP Autumn 2016). Season precision.

2.2 References
Simpson, Lisa (2016a-07-18). Time, unspecifed time zone. 2016-07-18 10:26.
Barker, Anne (2016-06-06). “Time, local time zone”. In: (2016-06-06 18:20
+10:00).
Simpson, Lisa (2016b-07-18). Time, zulu time zone. 2016-07-18 20:26 Z.
Kennett, Jeanette (2008-01). “Month precision date”. In: (2008-01).
Mendelsohn, Daniel (2010-01-25). “Day precision date.” In: (2010-01-25).
SEP (Autumn 2016). Season precision. Autumn 2016.

Of particular note are the "Times" entries. Under this scheme detailed information about times is available by exact matching the citation with the bibliographic entry, then scanning to the end of the line. In effect that works like the conventional year matching scheme: but this time we match on a day (if avialable). All that works well.

New case fails: Time level precision (Error)

If you wanted to have a quirky time level precision in the citation and bibliography (something I've also been pushing for) you could use the following (changing labeldateusetime to true) ...

Test Series 3

%      alldates=ymd,
%      labeldate=ymd
%      alldatesusetime=true
%      labeldateusetime=true,

Varying the value for mergedate I get ...

false (1,2), minimum (1,2:fail), basic (1,2:fail), compact (1,2), true (1,2), maximum (1,2).

That is, minimum and basic fail.

For example, for mergedate=false I get an expected result in the bibliography

Simpson, Lisa (2016a-07-18 10:26). Time, unspecifed time zone. 2016-07-1810:26.
Barker, Anne (2016-06-06 18:20 +10:00). “Time, local time zone”. In: (2016-06-06 18:20 +10:00).
Simpson, Lisa (2016b-07-18 20:26 Z). Time, zulu time zone. 2016-07-18 20:26 Z.

But for mergedate=minimum I get ...

Simpson, Lisa (2016a-07-18 10:26). Time, unspecifed time zone. 2016-07-18 10:26.
Barker, Anne (2016-06-06 18:20 +10:00). “Time, local time zone”. In: (2016-06-06 18:20 +10:00).
Simpson, Lisa (2016b-07-18 20:26 Z). Time, zulu time zone. 2016-07-18 20:26 Z.

... the same output as for mergedate=false. Given that the date and labeldate strings are identical in Anne Barker's entry I'd expect that entry to be merged (for mergedate=minimum). That is, I'd expect

Simpson, Lisa (2016a-07-18 10:26). Time, unspecifed time zone. 2016-07-18 10:26.
Barker, Anne (2016-06-06 18:20 +10:00). “Time, local time zone”. In:
Simpson, Lisa (2016b-07-18 20:26 Z). Time, zulu time zone. 2016-07-18 20:26 Z.

I think that would conform to the spirit of 50-style-authoryear-biber.pdf (which still speaks in the old terms, which assumes year precision) ....

mergedate=false strictly separates the date specification (following date) from the date label (folllowing labeldate). The year will always be printed twice:

Doe, John (2000). Book 1. Location: Publisher, 2000.
Doe, John (2003a). Book 2. Location: Publisher, 2003.
Doe, John (2003b). Book 3. Location: Publisher, 2003.

...

mergedate=minimum merges the dates whenever the full date and the date label are exactly the same string. If the date is a bare year number and there is no extrayear field, the date specification will be omitted:

Doe, John (2000). Book 1. Location: Publisher.
Doe, John (2003a). Book 2. Location: Publisher, 2003.
Doe, John (2003b). Book 3. Location: Publisher, 2003.

I note elsewhere @moewew you've mentioned that there are new draft's of ISO 8601 which deprecate, by incorporating, EDTF. I've yet to catch up on that issue but at a first glance it doesn't directly affect us on the mergedate issue (from memory @plk identified the main issue as the character for dates that were uncertain and approximate).

moewew commented 7 years ago

Thank you for taking time to look into this again.

I think it is safe to say that the whole mergedate code was written with only year-precision labeldates in mind. The updated code also still assumes year-precision for \printlabeldate and at least 'year+month/season'-precision for \printdate, but has become slightly better at dealing with other levels as you found out. Perhaps not surprisingly, mergedate=minimum also performs badly with alldates=year, labeldate=year and not only with alldates=ymd, labeldate=ymd, alldatesusetime=true, labeldateusetime=true,.

To make things work with all settings we would need something far more intricate. I might be barking up the wrong tree, but at the moment I think a new system would 'simply' need to be able to assess if \printdate would print more than \printlabeldate. At the moment this is done 'manually' under the above assumptions. For an automatic solution, the algorithm would have to pull the precision values of date and labeldate and check if the first element (there could be joint firsts: season and issue are special) not covered in labeldate's precision that would be covered by date's precision is non-empty. I don't see an easy way to implement this at the moment without resorting to a built-in 'look-up table' of these fields.

JohnLukeBentley commented 7 years ago

Perhaps not surprisingly, mergedate=minimum also performs badly with alldates=year, labeldate=year

Thanks. I missed that. But I see now. With ...

alldates=year,
labeldate=year,
alldatesusetime=false,
labeldateusetime=false,
mergedate=minimum

... I get ...

Simpson, Lisa (2016a). Time, unspecifed time zone. 2016.
Barker, Anne (2016). “Time, local time zone”. In: (2016).
Simpson, Lisa (2016b). Time, zulu time zone. 2016.

.... the "Baker, Anne" bibliographic entry does not merge when we would expect it to (under the former scheme).

To make things work with all settings we would need something far more intricate.

I suppose, then, the options would be to:

For for time level precision the following, or something like it, could be given. For ...

alldates=ymd, 
labeldate=ymd, 
alldatesusetime=true, 
labeldateusetime=true,
mergedate=maximum

... the result ...

2 Precision finer than a year

2.1 Citations
Times:
Lorem (Simpson 2016a-07-18 10:26). Time, unspecifed time zone.
Lorem (Barker 2016-06-06 18:20 +10:00). “Time, local time zone”.
Lorem (Simpson 2016b-07-18 20:26 Z). Time, zulu time zone.

Precision finer than a year - Dates:
Lorem (Kennett 2008-01). “Month precision date”.
Lorem (Mendelsohn 2010-01-25). “Day precision date.”

Precision finer than a year - Season:
Lorem (SEP Autumn 2016). Season precision.

2.2 References
Simpson, Lisa (2016a-07-18 10:26). Time, unspecifed time zone.
Barker, Anne (2016-06-06 18:20 +10:00). “Time, local time zone”. In:
Simpson, Lisa (2016b-07-18 20:26 Z). Time, zulu time zone.
Kennett, Jeanette (2008-01). “Month precision date”. In:
Mendelsohn, Daniel (2010-01-25). “Day precision date.” In:
SEP (Autumn 2016). Season precision.

So the current state of play essentially gives me what I was after: to be able to present to the reader, as an author of an essay, precisions finer than a year in a variety of ways that are not confusing (e.g. imaging I, or others, would be writing an essay about a twitter storm that happened over a 24 hours period where times might be informative).

The remainder seem to merely involve niceties around aesthetic choices in the bibliography. In short, I'm persuaded not altering the code, in terms of mergedate, leaves things in a reasonable state and would be a reasonable option to take. Although in that case updating the documentation would seem necessary.

moewew commented 7 years ago

I'm a bit torn at the moment. On the one hand I'd love to make mergedate play nice with all possible settings, but on the other hand mergedate was conceived when labeldate used to be year+extrayear only and date at most YYYY-MM-DD and uses that logic and thinking, I also doubt very much that mergedate together with all sorts of verbose date settings is a sought-after feature.

compact, maximum and false should work fine with any setting

minimum and basic would need to detect if the \printdate and \printlabeldate would give the same output. But as I mentioned above it is not trivial to detect this. I don't think it is impossible, but it is hard to get it right and make it elegant. I'm concerned about introducing lots of code and logic for something hardly anyone uses.

moewew commented 7 years ago

OK. I had a look at this in https://github.com/moewew/biblatex/commit/5c08d06e3fb29113c07d83a79d87646623d47094.

@JohnLukeBentley If you want to test this, simply put https://github.com/moewew/biblatex/blob/5c08d06e3fb29113c07d83a79d87646623d47094/tex/latex/biblatex/bbx/authoryear.bbx (this exact version) in your test folder.

@plk My reservations against this still stand. Let me know if you think this is worth it. If we decide to merge this, we need to think about the command names first. Maybe some of them should be private, or they should be moved to biblatex.def and documented?

plk commented 7 years ago

I think that making the commands public is a good idea - you can't have too many tests available and I also think that long descriptive names are fine as it makes future extensions easier. I do understand the reservations but I think that with these changes, it's a good situation and we can revisit more fundamental changes when the new ISO8601 is ratified. Will you merge in the changes and document the commands?

moewew commented 7 years ago

OK. I made the commands public. (Hopefully correctly.) Will submit a pull request.

moewew commented 7 years ago

I'd like to close this ticket since the discussion has become quite long now. We probably need to open a new issue about the EDTF/ISO transition.

Discussion here should be limited to the mergedate issue (which are hopefully solved now).

JohnLukeBentley commented 7 years ago

@moewew Just to let you know I'm working on this a little bit each day, using your authoryear.bbx. It looks like you have solved the last handful of cases. However, I'm having to wrestle with my own confusions before giving detailed feedback. I also have my eye on #540.

In short, thanks for your recent efforts on this ... I'm still on it.

moewew commented 7 years ago

If you want to test now, just get the newest development versions of biblatex and Biber from sourceforge.

540 will probably not be solved, but at the moment that it not high up my priority list, seeing that we will have to overhaul the whole EDTF/ISO thing with the advent of the new ISO anyway.

JohnLukeBentley commented 7 years ago

@moewew thanks for that coding!

And with thanks: I'm on a fresh download of the biblatex and biber dev versions from sourceforge.

ISO

We probably need to open a new issue about the EDTF/ISO transition.

@plk's suggestion, if I recall it correctly, that we stick to EDTF until ISO progresses to some firmer milestone seems right. If that is right, then is there anything for us to discuss until that milestone is reached? I mean do you have in mind a discussion around the ISO draft from a biblatex point of view to come to a collective conclusion about what ISO should look like, in order to then lobby in the ISO forum?

Merging dates

You sound a bit equivocal about whether you want to close this thread. If you'd like me to close this thread and start a new one dedicated to the mergedate issue: I'd be happy to do that if that helps keep things uncluttered.

For the moment I'll take your last prompt and continue the mergedate discussion here.

Your coding certainly solves the last set of cases I talked about. So now I have ...

% mergedate=  % false, minimum, basic, compact or true (default), maximum.
%    See \doc\latex\biblatex\examples\50-style-authoryear-biber.pdf
% mergedate bug tests:
%   Numerals in brackets refer to a pass of 1 of 3 issues mentioned 
%   at https://github.com/plk/biblatex/issues/520#issue-196903348 
%   with the following results
%     - Absent: not tested.
%     - Present: passed.
%     - Present with "fail": failed.
%     - Present with "?": unsure if passed.
% config 1:
%      alldates=ymd,  
%      labeldate=year, 
%      alldatesusetime=true, 
%      labeldateusetime=false,
%
%      false (1,2), minimum (1,2), basic (1,2), compact (1,2:?), true (1,2:?), maximum (1,2:?)
% config 2:
%      alldates=ymd,  
%      labeldate=ymd, 
%      alldatesusetime=true, 
%      labeldateusetime=false, 
%   
%      false (1,2), minimum (1,2), basic (1,2), compact (1,2:?), true (1,2:?), maximum (1,2:?)    
% config 3:
%      alldates=ymd,  
%      labeldate=ymd, 
%      alldatesusetime=true, 
%      labeldateusetime=true, 
%   
%      false (1,2), minimum (1,2), basic (1,2), compact (1,2), true (1,2), maximum (1,2)  
% config 4:
%      alldates=year,  
%      labeldate=year, 
%      alldatesusetime=false, 
%      labeldateusetime=false, 
%   
%      false (1,2), minimum (1,2), basic (1,2), compact (1,2), true (1,2), maximum (1,2) 

The results for "2" are essentially all we need to look at. Issue "2" (from the first post) is essentially the issue of the merging of dates.

To illustrate why I'm not sure about some of the results, take an entry with datetime precision and config 1 ...

%      alldates=ymd,  
%      labeldate=year, 
%      alldatesusetime=true, 
%      labeldateusetime=false,

... with mergedate=false I get something that makes sense ...

2.1 Citations
...
Lorem (Barker 2016). “Time, local time zone”
...
2.2 References
...
Barker, Anne (2016). “Time, local time zone”. In: (2016-06-06 18:20 +10:00).

... but with mergedate=compact I get ...

2.1 Citations
...
Lorem (Barker 2016). “Time, local time zone”
...
2.2 References
...
Barker, Anne (2016-06-06 18:20 +10:00). “Time, local time zone”. In:

So that might make conceptual sense given what "compact" historically was intended to achieve "merges all date specifications with the date labels". But, on the other hand, this seems to break:

At this part of my day, when my brain gets soft, I'm not sure what to suggest. But perhaps if I pass the torch back to you, you'll be able to see matters through.

But I'm might offer some more vague thoughts ...

I'm concerned about introducing lots of code and logic for something hardly anyone uses.

Yes that'd be a legitimate concern. I can't help wonder if it wouldn't make it easier, both for users and coders (i.e. you), to reduce the mergedate options. E.g. To three: none (alias false), basic (alias true), maximum. But maybe @plk has a story to tell about a prior battle to get, and the importance of, the 5 values we have today.

moewew commented 7 years ago

Re the ISO discussion: I just don't want to make this thread bigger than it already is by discussing EDTF/ISO issues here. I don't know if we need to talk about it at the moment, but if so, I'd rather not do it here.

Thanks for testing this again. I'm for obvious reasons quite eager to close this issue, but of course only if the mergedate issue is resolved.

As far as I can see you are only unhappy/unsure about compact and maximum.

The options follow my interpretation of what mergedate should do with the vast array of new features available now. When date is merged into labeldate (into the labeldate position) it retains its own settings. That means that in merged situations the date settings apply in labeldate position, and can overrule labeldate settings. If all labeldate settings were to prevail instead, we would not obtain a merged date, we would essentially be throwing away the date completely and retain only the labeldate. And if only some of them were to prevail, well, then we'd have to decide which, and that would only add confusion. So the current apparent inconsistency is something we have to live with. (Remember where we came from. The examples in 50-style-authoryear essentially assume labeldate=year, alldates=ymd, and with mergedate=compact, we'd get 'Doe, John (June 2006a). ...' for one entry. The implicit labeldate=year is not honoured there.)

I don't think we can remove some of the mergedate options, they have been with us for too long, even though I doubt they are widely used. Since some of the mergedate options are very similar, we would not even gain much from dropping some of them.

JohnLukeBentley commented 7 years ago

As far as I can see you are only unhappy/unsure about compact and maximum.

Yes, that was correct.

The options follow my interpretation of what mergedate should do with the vast array of new features available now. [With a following explication of that interpretation]

Thanks. That interpretation convinces me that the compact and maximum, and therefore all the combinations I've tested, are as they should be. Your explication is clear but my crude summary is: when merging occurs date has precedence over labeldate (in the labeldate position), in the bibliography.

On the number of mergedate options: if you are happy to preserve them then so am I.

So at least as far as the coding goes I think this mergedate issue is closed.

But might there be a documentation task, e.g., in updating 50-style-authoryear or providing another document: to exemplify mergedate working with precisions finer than a year? The sort of explanation you've just provided could go into the documentation.

With a proper sense of duty I should put up my hand to offer a draft ... but, given that in general the coders should write the docs, perhaps it something you might like to take on as a (low priority) task ... if you see value in it.

moewew commented 7 years ago

I'd already changed the docs a bit in https://github.com/plk/biblatex/commit/4b4b0fd6703c2630904f8002ccee64a169dae794, but there is #637 now. If you have time to look at the new explanations of the feature and tell me what you think, that would be really helpful. If you have suggestions for additional explanation or changes, don't hesitate to add them here.

JohnLukeBentley commented 7 years ago

Thanks! Just looking at 50-style-authoryear.tex/*.pdf in that commit I think your explanations are largely spot on and properly economic.

You might have a repeated error, however, in two of your examples. For compact and maximum you have ...

Doe, John (7th Aug. 2017). Webpage. 7th Aug. 2017.

... when it ought be ....

Doe, John (7th Aug. 2017). Webpage.

... at least if it is to conform the results I'm getting when I run with something like ...

alldates=ymd,
labeldate=year, 
alldatesusetime=true, 
labeldateusetime=true,
mergedate=compact 

... but perhaps you were trying to illustrate "even if it is printed in the position of the date label" (??) ... but I wouldn't be sure how to achieve this with a date that has day level precision.

Perhaps another example could also be thrown in with timezone level precision "Doe, John (2016-06-06 18:20 +10:00) ...".

A separate suggestion would be to explain "precision". You and I are familiar with its meaning but I suspect newcomers might be a bit thrown. So perhaps a sentence or two in the introductory paragraph. Something like ...

... The "precision" of a date or datetime is the extent to which it provides finer time intervals or finer specificity of a time interval (i.e. the provision of timezone information). So the following datetimes are ordered from lesser to greater precision: 2017-10, 2017-10-24, 2017-10-24 18:50, 2017-10-24 18:50 +05:00

All this raises the prospect of providing a separate example pdf that illustrates varying datetime precisions (an example pdf that doesn't discuss mergedate).

moewew commented 7 years ago

Thank you for the feedback.

I have fixed the obvious error you pointed to with compact and maximum in https://github.com/plk/biblatex/commit/1e3acb69364d62f88b3f81560717f1124e8d081a.

I was not too keen to add even more examples with time zone and whatnot. If people really use these things, they should experiment with the settings anyway.

I did not want to add a whole paragraph about the 'precision' issue, so I just added 'date time granularity' in brackets. Hopefully that does not make things worse. I'm not that comfortable writing these docs and I don't think they are read that often anyway. If you think a longer explanation might be helpful, feel free to submit a pull request with your suggestions.

JohnLukeBentley commented 7 years ago

'date time granularity' in brackets works well I think. I'm happy you should reject the suggestion of further examples (for this document). And I see the error fixed in the commit.

If I think that datetime precision requires further exemplification it would probably be better as its own document anyway. If I get to it I might make a case for it in a comment or pull request, as you suggest.

So the current issue is closed. Thanks very much for all that!