search

plk / biblatex

biblatex is a sophisticated bibliography system for LaTeX users. It has considerably more features than traditional bibtex and supports UTF-8
522 stars 118 forks source link

Dates and Datetimes. #461

Closed JohnLukeBentley closed 8 years ago

JohnLukeBentley commented 8 years ago

The following is a continuation of Circa dates, circa date ranges, and question marked dates. Plus Eras. 427 which was properly closed for reasons of length (at the very least it was starting to take too long to load in the browser).

I copy and paste an edited version two posts of mine that contain outstanding issues. These mostly have to do with issues around handling varying date and datetime precision. I've omitted the biblatex.pdf suggestions and the month bug, which you @plk, have already dealt with.

The style guides on dates and datetimes

I've made a review of two style guides - the Chicago Manual of Style, 16th Ed. (Chicago) and the Publication Manual of the American Psychological Association, 6th Ed. (APA) - to see the various ways dates are, and ought be, formatted.

You (@plk) will have been through all that: so I'm mostly catching up.

However my hope is that the following issues it might serve as a suitable reminder and touchstone for any further biblatex date and datetime issues. I'm only picking out those date and datetime issues from the style guides that might be biblatex relevant, rather than mention everything the style guides has to say about dates (and datetimes).

So, a style author (whether of a style manual, a biblatex standard style, a biblatex add-in style, or a biblatex user overriding/customizing a style) has to decide on the following issues:

(In the following examples bold is added, italics are original.)

[The following differs from my original post:] All the above is meant as an orientation to the issues. The following post attempts to narrow down on them, to become a specific set of recommendations.

JohnLukeBentley commented 8 years ago

Design: Origdate in standard authoryear style

The remaining design issues are details of two of those from my prior comment headed "The style guides on dates and datetimes". They probably need to be addressed first before dealing with the others.

Design: Date precision correspondence

To what extent might a style guide - e.g. the Chicago Manual of Style, 16th Ed. (Chicago) and the Publication Manual of the American Psychological Association, 6th Ed. (APA) - want the publication date precision in the reference or bibliography to correspond with the precision in the in-text citation?

Note the difference between date precision and date formatting:

Chicago

In Chicago date precision correspondence is principally an issue for the author-date-references style, for the notes-bibilography style mostly entails an in-text superscript numeral 1. So there's no citation date to correspond to the note or bibliography.

In the Chicago author-date-references style the year alone is generally taken to be relevant part of the identifying date. So only the year tends to be used in in-text citations, which correspond to the separated out year in the reference - even if more precise date information is included (e.g. a month and day).

So a common case is Chicago-Style Citation Quick Guide, Author-Date, Ex "Article in a newspaper or popular magazine":

(In all the following examples bold is added, italics is original).

Mendelsohn, Daniel. 2010. "But Enough about Me." New Yorker, January 25.

(Mendelsohn 2010, 68)

That is, Chicago sometimes specifies an in-text to reference date precision mismatch (here 2010-01-25 versus 2010).

Elsewhere in Chicago a day precision date might be included in a citation 14.206 Citing in text rather than in a bibliography:

.... that when it comes to the aging quarterback's uncertain prospects for yet another season, "there is final, and there is Favre" (New York Times, January 25, 2010).

In which case a reference entry would be something like (going off the "bibliography" example from 14.206 and rearranging it to conform with the general author-date-reference pattern).

If that was the case there'd be an in-text to reference entry date precision match (even though the reference entry has its date separated by other publication facts).

So sometimes Chicago has a in-text to reference correspondence of the date precision; at other times it doesn't.

APA

APA, CH 7, Ex 11 Online newspaper article:

  • Brody, J. E. (2007, December 11). Mental reserves keep brains agile. The New York Times. Retrieved from http://www.nytimes.com

I can't find a corresponding in-text citation example for newspapers. However, APA, CH 7, Sec 6.20 "Personal Communications" has:

T. K. Lutes (personal communication, April 18, 2001)

So APA permits (I don't have evidence to say it requires) a day precision date both in the reference and in-text citation. That's a precision match, even though the format is mismatched.

For a reference entry APA, unlike Chicago, doesn't separate date parts with other publication facts. That makes the APA simpler.

Biblatex recommendations

In the biblatex standard style authoryear I recommend we enable the user to make the following choices, given something like ...

@article{mendelsohn_2010_enough,
    author = {Mendelsohn, Daniel},
    date = {2010-01-25},
    title = {But {{Enough About Me}}},
    urldate = {2016-08-03},
    journaltitle = {The New Yorker},
    url = {http://www.newyorker.com/magazine/2010/01/25/but-enough-about-me-2}
}

(In the following example bold added, italics original).

To achieve the above combination of formats [excepting those marked "Don't do this"] I recommend ...

Design: If dates have corresponding precision, should the formatting correspond?

Although I can't find anything explicit, the APA seems to encourage a possible scheme like ...

APA, CH 7, Ex 11 Online newspaper article:

  • Brody, J. E. (2007, December 11). Mental reserves keep brains agile. The New York Times. Retrieved from http://www.nytimes.com

An APA in-text citation example I derive:

Lorem (Brody, The New York Times, December 11, 2007)

... that is, where the date precision corresponds but the date formatting is inconsistent.

One could choose to insist upon formatting consistency by changing the in-text citation spec to result in something like:

Lorem (Brody, The New York Times, 2007, December 11)

Generally I don't see any good reason for any style to have inconsistent output formats between the in-text citation and bibliographic entry.

In biblatex, presumably, if the "date precision correspondence" issues above are sorted out then date formatting correspondence will generally apply, without having to do extra work. At least in the standard biblatex styles like authoryear.

njbart commented 8 years ago

So APA permits (I don't have evidence to say it requires) a day precision date both in the reference and in-text citation.

I don’t think so. APA does not seem to permit month or day precision in text (with one exception). The general rule is: “Even if the reference includes month and year, include only the year in the text citation” (APA Manual 6e, 6.11).

The only exception are personal communications, which do not appear in the reference list; these may have month or day precision in text:

“[…] personal communications are not included in the reference list. Cite personal communications in text only. Give the initials as well as the surname of the communicator, and provide as exact a date as possible” (6.20).

Also, I don’t think APA or Chicago ever recommend or permit giving both author and title in text, so I doubt your proposal “(Brody, The New York Times, 2007, December 11)” is correct in either style.

Generally I don't see any good reason for any style to have inconsistent output formats between the in-text citation and bibliographic entry.

The way I understand APA and Chicago, both do indeed opt for different output formats between the in-text citation and reference list entry if an item’s date has month or day precision.

JohnLukeBentley commented 8 years ago

Thanks Nick.

You address three issues on what the Style Guides specify.

1. An in-text citation with author and title.

You are correct that my derived example, "(Brody, The New York Times, 2007, December 11)", in showing author and title in text, is probably not supported by APA. I regret constructing the example that way to the extent it distracts us with this off-topic issue. I was sloppy there. I should have, when talking about APA, used "(Brody, 2007, December 11)".

But in my having triggered this off-topic thanks for flagging it.

Chicago might support title and author in text under some circumstances. But given that is an off topic issue I won 't expand on the reasons why I think so.

In any case for the purposes of thinking about dates and datetimes we can let my derived APA in text citation example be "(Brody, 2007, December 11)". (I'll leave the mistake in my original post).

2. Design: Date precision correspondence.

{

Which evokes two issues in my topmost post "The style guides on dates and datetimes"

Date precision correspondence. That is, if relevant, an in-text to reference correspondence of the date precision.

... and implicitly ..

Outputting date precision finer than year level. Whether to forbid it, permit, or mandate it; and whether to set that permission globally, on a type basis, or entry basis."

}

We are clear, I think, that both APA and Chicago sometimes permit dates at precisions finer than a year in the reference/bibliography. Indeed when the circumstances apply they encourage it.

However, you seem to be right that with respect to APA in-text citations:

Chicago in-text citations appear to follow an analogous scheme:

So as a broad understanding of Chicago and APA on date precision correspondence:

3. Design: If dates have corresponding precision, should the formatting correspond?

You wrote:

The way I understand APA and Chicago, both do indeed opt for different output formats between the in-text citation and reference list entry if an item’s date has month or day precision. [emphasis original].

In the light of my further illustration of issue 2 above (on your prompting) it might be clearer that according to the Style guides, a reference to in-text citation date precision correspondence is rare, for dates finer than a year. That is, generally if a reference entry has a fine precision (e.g. day level) then the in-text citation has precision only to a year.

But there is at least one case, in Chicago, for a date precision finer than a year, where the reference (or bibliography) and in-text citation have corresponding precisions. It's an example I've quoted above which I repeat now ...

Chicago > 14.206 Citing in text rather than in a bibliography:

[In-text citation:]

another season, "there is final, and there is Favre" (New York Times, January 25, 2010).

If, for some reason, a bibliography entry were included, it would appear as follows:

[Bibliography:]

Borzi, Pat. “Retirement Discussion Begins Anew for Favre.” New York Times, January 25, 2010. http://www.nytimes.com/2010/01/26/sports/football/26vikings.html?emc=eta1.

... in that case, as exemplified by Chicago themselves, for the same output precision they have the same output format.

The Style Guides and biblatex

Having traversed the three issue you raise, none of them alters the value of my prior recommendations (Under "Biblatex recommendations" in my second post), to my mind.

Those recommendations essentially amount to ensuring:

By "some other scheme through minor option setting" I'll note Philip (@plk) correctly observes in biblatex-apa.pdf "The APA style manual is ... not entirely consistent". One might like to make one's style more consistent.

For example one might like to reject the Chicago/APA idea that you can have an in-text citation without a concomitant reference entry.

And with respect to the quoting of electronic sources I suspect Chicago/APA might both be still behind the times here. Namely in not thinking too much about instances where one is writing about electronic sources where the day level dates or time level dates are significant. Think of an article, say, analyzing a newscycle. In that case you may want to quote even time level dates - both in the in-text citations and in the reference entry.

In short: there are circumstances where an author would want to include date precisions finer than a year, in the in-text citation. And they might, therefore, want a set of consistent biblatex options around the presentation of that precision (in-text to reference precision and formatting options).

Do you have a view on the value of my recommendations (under "Biblatex recommendations" in my second post)?

Edit: Spelling.

JohnLukeBentley commented 8 years ago

@plk What view are you taking toward my recommendations (under "Biblatex recommendations" in my second post)?

Perhaps the length of these posts make these issues difficult to digest. In which case perhaps I could try breaking it down to one issue at a time (without the contextualizing explanations) - in the attempt to make sharper whether any of these issues are something that ought be addressed in the core rather than left to style authors. ??

plk commented 8 years ago

This issue of ORIGDATE in core has been brought up before and I think we decided that it complicated the core styles a bit too much. Such issues can be dealt with via a sourcemap usually so that you don't need two .bib files. Also, we tend to go by weight of user requests and this is rarely requested ...

JohnLukeBentley commented 8 years ago

Thanks for addressing the Origdate issue, but that issue was peripheral. The focus of my post that mentions the Origdate issue, is on other issues.

plk commented 8 years ago

I'm afraid I've lost track of the issues - perhaps you can summarise?

JohnLukeBentley commented 8 years ago

There are seven issues summarized (as bullet points) in my first post, under the heading: "The style guides on dates and datetimes".

My second post focuses on two issues: "Design: Date precision correspondence" and "Design: If dates have corresponding precision, should the formatting correspond?" (apart from the origdate issue).

The first issue to consider would be "Design: Date precision correspondence". Under that heading I have a specific set of "Biblatex recommendations".

Anything in the present post between quote marks are verbatim heading strings (so a page search should get you to the right spots).

Thanks again for the sourcemap tip off ... I'll look into that.

plk commented 8 years ago

I did look through these but they are all style issues for specific styles to decide on and nothing really for the core or core styles since none of these have ever been requested. The core styles are really "reference" styles which shouldn't be complex or full-featured in any particular direction. The point of custom styles is exactly to address these questions.

JohnLukeBentley commented 8 years ago

Thanks Philip. Now that you've made that call I'll carry on with general testing (after implementing a sourcemap for my origdate stuff) of the biblatex core.

I see you've already released biblatex 3.6 (with all the datetime code) to production, with 3.7 the current dev version. Will it be OK if I test datetime stuff against the production 3.6? In other words have you touched any datetime code since the release of 3.6?

Did any of my issues above shed any light on your implementation of (the style) biblatex-apa?

plk commented 8 years ago

There are no changes in datetime code in 3.7 (I expect there may be later as people start to play with it). The APA specification is a bit of a pain in places as it is not that consistent and it is constantly being revised by APA blog posts. The style now uses the new datetime features internally. The style changes only incrementally at the moment as I don't have the time for a major rewrite, which it probably needs ...

JohnLukeBentley commented 8 years ago

Thanks.

Yeah any style, APA or otherwise, is necessarily fiddly business. It's probably difficult to come up with a consistent set of rules even in principle. In any case I certainly understand if you don't want to give it major attention at the moment. I'm unlikely to give the APA style further specific attention (in that I currently favour Chicago).

In the fullness of time I'll probably need to become a style author myself ... and probably implementing my own desires based off Chicago or APA. And, to that end, it does appear that you've given me everything in the core to enable any datetime format I should wish.

I'll resume core testing against 3.6 and let you know of my results, even if I find no issues/bugs.

BTW the production biblatex.pdf needs its version number updated from "3.5" to "3.6" (looking at https://www.ctan.org/tex-archive/macros/latex/contrib/biblatex/doc)

JohnLukeBentley commented 8 years ago

Hey Philip, there might be a backward compatibility issue.

Using a MWE like ... 

\documentclass{article}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
  @book{plato_2004_republic,
    author = {{Plato}},
    date = {-0379~},
    title = {Republic},
    translator = {Reeve, C. D. C.},
    publisher = {{Hackett Publishing Company, Inc.}},
    timestamp = {2016-10-05T08:55:13Z},
    pagetotal = {392},
    isbn = {0-87220-737-4},
    edition = {3rd edition},
    location = {{Indianapolis}}
  }

  @incollection{hume_1975_enquiry,
    author = {Hume, David},
    date = {1975-06-12},
    origdate = {1751},
    title = {An {{Enquiry Concerning}} the {{Principles}} of {{Morals}}},
    editor = {Selby-Bigge, L. A. and Nidditch, P. H.},
    publisher = {{Oxford University Press}},
    booktitle = {Enquiries {{Concerning Human Understanding}} and {{Concerning}} the {{Principles}} of {{Morals}}},
    timestamp = {2016-06-05T20:08:47Z},
    isbn = {978-0-19-824536-0},
    edition = {3},
    location = {{New York}},
    note={Note},
    addendum={Addendum},
    annotation={Annotation}
  }
\end{filecontents}

\usepackage[
  style=authoryear,
  backend=biber
]{biblatex}

\addbibresource{\jobname.bib}

\begin{document}
  \nocite{*}
  \printbibliography
\end{document}

Trying to run this with the Plato entry, date = {-0379~}, gives "Undefined control sequence. \end". Commenting out the Plato entry, so Hume's regular date = {1975-06-12} is used, works fine.

If I throw in a basic mix of the recently created options to support the new datetime formats, everything is fixed up.

\usepackage[
  style=authoryear,
  alldates=ymd,
  dateera=secular,
  datecirca=true,
  datezeros=true,
  dateuncertain=true,
  backend=biber,
  backref=true,
  sorting=nyt%
]{biblatex}

I seem to recall that precisely for backward compatibility reasons you wanted the user to deliberately set these new options. If that's right, and defaults can't be set to handle the first example, perhaps a more informative error message could be given (??).

plk commented 8 years ago

Hmm, using 2.7/3.7 dev versions, I have no problems with this?

JohnLukeBentley commented 8 years ago

Indeed updating to the dev versions gets rid of the problems.

So I can now use ..

\usepackage[
  style=authoryear,
  backend=biber
]{biblatex}

... without error messages and Plato's date = {-0379~} passes through without processing, as expected.

Thanks Philip!

JohnLukeBentley commented 8 years ago

For the sake of others that might be landing in this thread from Google: plk has helped me with code for data source mapping (e.g. using DeclareSourcemap) that allows one to use origdate with the biblatex native authoryear style.

One can therefore readily switch between styles that don't support origdate (like the biblatex native style like authoryear) and those that do (e.g. biblatex-chicago) without having to change one's biblatex source file (of bibliographic entries).

A full explanation and solution is at Stackexchange > Biblatex DeclareSourcemap: how do we conditionally prefix a space when appending fields?

JohnLukeBentley commented 8 years ago

Biblatex.pdf error, P 37

Also supported are 'uncertain’ dates (edtf 5.2.2) which are automatically expanded ...

Should be ...

Also supported are ‘unspecified’ dates (edtf 5.2.2) which are automatically expanded ...

That is, change 'uncertain' to 'unspecified'.

JohnLukeBentley commented 8 years ago

Recalling that in biblatex.pdf, p 36 it is claimed:

[Biblatex date fields] adhere to EDTF (Extended Date/Time Format) specification levels 0 and 1

Part of EDTF level 1 is 5.2.4 Year Exceeding Four Digits (L1).

I'd suggest we'd want to implement and document this for the sake of completeness (the alternative is to document that this is not implemented).

Quoting the section in it's entirety ...

'y' may be used at the beginning of the date string to signify that the date is a year, when (and only when) the year exceeds four digits, i.e. for years later than 9999 or earlier than -9999. (An alternative, exponential form, with optional precision, is allowed in level 2.)

Examples

y170000002
the year 170000002
y-170000002
the year -170000002

They appear to have written "may" in error: I think they mean "must". That is, given the BNF ...

longYearSimple = "y" "-"? positiveDigit digit digit digit digit+

... and the level 0 "Definition for year" just confines itself to four digits ...

year = positiveYear | negativeYear | "0000"

positiveYear = positiveDigit digit digit digit | digit positiveDigit digit digit | digit digit positiveDigit digit | digit digit digit positiveDigit

negativeYear = "-" positiveYear

In short, if the year has more than four digits it seems that EDTF requires that it must be prefixed by a "y".

Edit: "for digits" to "four digits".

plk commented 8 years ago

Did you test this? I thought I did implement this ...

JohnLukeBentley commented 8 years ago

I hadn't tested it. But I've just tried, "12345" and "y12345", in 96-dates.tex: and they fail.

And there is nothing in the documentation about it.

I, too, thought you had implemented it.

JohnLukeBentley commented 8 years ago

... and, note, when a year more than four digits is permitted then only the year part is permitted, not other date components (month, day, etc).

% Legal
y170000002
y-170000002

% Illegal
y170000002-03-22
y-170000002-03-22

Based on my looking at the EDTF BNF and the Online EDTF Validator.

plk commented 8 years ago

There was a minor parsing bug biber - biber 2.7 should fix this - can you try? If it doesn't work, a tiny MWE would be appreciated.

JohnLukeBentley commented 8 years ago

Thanks, it looks good now. You might like to throw in the following to 96-dates.tex ...

@misc{jlbdate160,
  note      = {Year exceeding four digits},        
  date      = {y12345},
  origdate  = {y-12345},
  % eventdate = {y12345-10-11}  % Fails, as expected
  % urldate = {12345} % Fails, as expected
}
JohnLukeBentley commented 8 years ago

And a heads up that the working draft of 8601-2 uses alternative characters for some EDTF formats.

So far I know of:

This came to my attention from @retorquere in zotero-better-biblatex > EDTF dates in BibLaTeX #590 > A comment in virtue of his use of a third party javascript EDTF parser called EDTF.js (github link). From that EDTF.js documentation ...

EDTF.js fully implements EDTF levels 0, 1, and 2 as specified by WD 2016-02-16 of ISO 8601-2 with the following exceptions (as raised by the EDTF community): ....

I'm not fully across the status of the relationship between between EDTF and WD 2016-02-16 of ISO 8601-2. It sounds like ISO 8601-2 (which I don't yet have access to) is positioning itself to do what EDTF does. And given an ISO standard might have more clout, ISO 8601-2 might eventually eclipse EDTF as THE bibliographic standard. Note to @nickbart1980.

(Incidentally, I had intended in not "publicizing" biblatex's support for EDTF until I'd come back to you with further testing results, which I still intend to carry out. But, I do already know your implementation is fairly robust; getting BBT to handle EDTF sort of helps with my testing of biblatex; and, as it turns out, testing BBT has already thrown up a biblatex relevant issue (the year beyond four digits issue, just solved)).

However, for the moment it sounds like the best thing for you to do is to not make any changes in virtue of whatever is in WD 2016-02-16 of ISO 8601-2. That is, given that our standard is EDTF, not WD 2016-02-16 of ISO 8601-2. It'd be a matter of letting the standards stabilize first, I'd suggest.

While working on making zotero-better-biblatex (BBT) cope with passing through EDTF dates from Zotero to biblatex we've (this is the royal "we") been targeting biblatex's close tracking of EDTF (and including a biblatex exception "1997/"). So there's no BBT reason for you to start thinking about coping with ISO 8601-2 (in addition to EDTF).

In short, there's nothing you need do here for the time being, in my view.

njbart commented 8 years ago

In short, there's nothing you need do here for the time being, in my view.

I’d agree.

For future reference: WD 2016-02-16 of ISO 8601-2 is available from – and being discussed on – the LoC server; see http://listserv.loc.gov/cgi-bin/wa?A2=ind1602&L=datetime&T=0&X=1A10F96E1DF2376B79&Y=rden%40loc.gov&P=1351.

JohnLukeBentley commented 8 years ago

@nickbart1980, thanks for the ISO 8601-2. In summary it's good that they are, among other things, solidifying EDTF (with some small changes) as a "profile".

@plk "unknown/2006" fails (unexpectedly) ...

@book{simpson_cool_book_0094,
  author = {Simpson, Lisa},
  title = {Cool Book 0094},
  date = {unknown/2006}
}

By contrast my "2004-06-01/unknown" passes (as expected).

There's this strange asymmetry with respect to "unknown" and "open" in EDTF ...

EDTF, 5.2.3. Extended Interval (L1)

  • 'unknown' may be used for the start or end date when it is unknown.
  • 'open' may be used when no end date is specified, ....

EDTF, Table of examples

unknown/2006 beginning unknown, end 2006

2004-06-01/unknown beginning unknown, end 2006

2004-01-01/open beginning January 1 2004 with no end date

That is, "open" never comes first, as in "open/2004-01-01". I have no idea why, if EDTF spec authors have " beginning January 1 2004 with no end date", they wouldn't also want "no start date, ending January 1 2004".

This is changed in WD of ISO 8601-2. The symbols for "unknown" and "open" have changed (asterisk "*" and nothing before or after a forward slash, respectively); And both unknown and open can be at the start or end ...

  • Unknown. Start or end date unknown. The character ‘*’ (asterisk) may be used for the start or end date to indicate “unknown”.
  • Open Start or date open. The start or end date may be left blank, either because there is none or for any other reason.

(4.5 Enhanced time interval, P 5)

So perhaps you'd want to ensure "open" and "unknown" work either at the beginning or end of the interval (keeping the current string literals), even though "open" at the start (as in "open/2004-01-01") is illegal under the (current version) of EDTF.

Edit:

So perhaps you'd want to add these tests in 96-dates.tex:

@misc{jlbdate170,
  note      = {Intervals - unknown},
  date      = {unknown/2006},
  origdate  = {2004-06-01/unknown},
}

@misc{jlbdate180,
  note      = {Intervals - open},
  date      = {2004-01-01/open}, 
  origdate  = {open/2004},
  eventdate = {/2006},
  urldate   = {2004-06-01/},
}
plk commented 8 years ago

Please try 2.7/3.7 versions. open/unknown ranges should now be possible as either start or end, also with the ''/'*' abbreviations.

JohnLukeBentley commented 8 years ago

Open/unknown

Good idea to throw in support for "*".

My version of 96-dates.tex now works fine. Specifically I get, ...

% 96-dates-jlb.tex
@misc{jlbdate170,
  note      = {Intervals - unknown},
  date      = {unknown/2006},
  origdate  = {2004-06-01/unknown},
  eventdate = {*/2006},
  urldate =  {2004-06-01/*},
}

@misc{jlbdate180,
  note      = {Intervals - open},
  date      = {2004-01-01/open}, 
  origdate  = {open/2004},
  eventdate = {/2006},
  urldate   = {2004-06-01/},
}

% PDF result
Intervals - unknown
labeldate = 2004-06-01/
date = /2006
origdate = 2004-06-01/
eventdate = /2006
urldate = 2004-06-01/

Intervals - open
labeldate = 2004-01-01/
date = 2004-01-01/
origdate = /2004
eventdate = /2006
urldate = 2004-06-01/

In my view:

And so, I can't see much use for unknown and open date ranges.

Is this the reason why you don't, in biblatex, bother supporting the difference between unknown and open date ranges (apart from being able to parse their input correctly) and just map them to the same sort of output (E.g. "2005/unknown" and "2005/open" will be output the same way, perhaps "2005/")?

One issue is that while all the basic combinations of unknown and open date ranges work in 96-dates.tex, in one of my tests that is more production-like (using the biblatex native authoryear style) I get output that chokes for unknown or open start dates. E.g. Something input as "unknown/2006" will output as a citation like "(Simpson n.d.)" and a reference like "Simpson, Lisa (n.d.). Cool Book 0092".

Perhaps the output should be "n.d./2006". I'm not clear if this is a mere style issue (particular to authoryear) or has core consequences.

My BiblatexOpenUnknownMwe.tex (which I hope justly violates your request for a "tiny" Mwe):

\documentclass{article}
\usepackage{a4wide}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
% Unknown
@book{simpson_cool_book_0092,
  author = {Simpson, Lisa},
  title = {Cool Book 0092},
  date = {unknown/2006},
  timestamp = {2016-10-31T05:55:17Z}
}

@book{simpson_cool_book_0093,
  author = {Simpson, Lisa},
  date = {2004-06-01/unknown},
  title = {Cool Book 0093},
  timestamp = {2016-10-31T05:55:37Z}
}

@book{simpson_cool_book_0094,
  author = {Simpson, Lisa},
  title = {Cool Book 0094},
  date = {*/2006},
  timestamp = {2016-10-31T05:56:44Z}
}

@book{simpson_cool_book_0095,
  author = {Simpson, Lisa},
  date = {2004-06-01/*},
  title = {Cool Book 0095},
  timestamp = {2016-10-31T05:57:12Z}
}

% Open
@book{simpson_cool_book_0096,
  author = {Simpson, Lisa},
  date = {2004-01-01/open},
  title = {Cool Book 0096},
  timestamp = {2016-10-31T05:57:31Z}
}

@book{simpson_cool_book_0097,
  author = {Simpson, Lisa},
  date = {open/2004-01-01},
  title = {Cool Book 0097},
  timestamp = {2016-10-31T05:57:56Z}
}

@book{simpson_cool_book_0098,
  author = {Simpson, Lisa},
  date = {2004-01-01/},
  title = {Cool Book 0098},
  timestamp = {2016-10-31T05:58:30Z}
}

@book{simpson_cool_book_0099,
  author = {Simpson, Lisa},
  date = {/2004-01-01},
  title = {Cool Book 0099},
  timestamp = {2016-10-31T05:58:56Z}
}
\end{filecontents}

\usepackage[%
  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  % Dates and Datetimes
  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  alldates=ymd,
  alldatesusetime=true,
  alltimes=24h,
  seconds=false,
  timezones=true,
  datezeros=true,
  dateera=secular,
  dateeraauto=501,% Sets the max year ceiling for automatic printing of era
  abbreviate=true, % prints full localisation strings
  dateabbrev=false, % prints full date localisation strings
  dateuncertain=true,%
  datecirca=true,%
  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  % Style
  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  style=authoryear,
  sorting=none,
  %sorting=nyt,
  dashed=true,
  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  % Other
  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  backend=biber,%
]{biblatex}

\addbibresource{\jobname.bib}

% Space between table rows
\renewcommand{\arraystretch}{1.5}

\begin{document}

Note \verb|sorting=none| is set.

\begin{refsection}
\section{Unknown}
\begin{tabular}{|p{180pt}|l|l|}

  \hline
  \textbf{Description} & \textbf{Input} & \textbf{In text citation result}\\
  \hline     
  Start date unknown, end date specified. ``Unknown". & \verb|unknown/2006| & \autocite{simpson_cool_book_0092}\\
  \hline     
  Start date specified, end date unknown. ``Unknown". & \verb|2004-06-01/unknown| & \autocite{simpson_cool_book_0093}\\
  \hline  
  Start date unknown, end date specified. ``*". & \verb|*/2006| & \autocite{simpson_cool_book_0094}\\
  \hline     
  Start date specified, end date unknown.``*". & \verb|2004-06-01/*| & \autocite{simpson_cool_book_0095}\\
  \hline    
  \noalign{\bigskip}  

\end{tabular}
\printbibliography 
\end{refsection}

\begin{refsection}
  \section{Open}
  \begin{tabular}{|p{180pt}|l|l|}

    \hline
    \textbf{Description} & \textbf{Input} & \textbf{In text citation result}\\
    \hline           
    Start specified, no end date. ``open". & \verb|2004-01-01/open| & \autocite{simpson_cool_book_0096}\\
    \hline   
    No start date, end date specified. ``open". & \verb|open/2004-01-01| & \autocite{simpson_cool_book_0097}\\
    \hline 
    Start specified, no end date. [Nothing] & \verb|2004-01-01/| & \autocite{simpson_cool_book_0098}\\
    \hline   
    No start date, end date specified. [Nothing] & \verb|/2004-01-01| & \autocite{simpson_cool_book_0099}\\
    \hline                 
  \end{tabular}
  \printbibliography 
\end{refsection}

\end{document}

Biblatex.pdf errors

biblatex.pdf, p37, error: double "in" ...

the localisation strings in in § 4.9.2.21

biblatex.pdf, p1, version is "3.5", should be "3.7".

plk commented 8 years ago

Doc errors corrected. The issue you mention is a style issue as "nodate" is when the startyear is empty by default. I might add a marker for "unknown" and make this a bibstring, by default the empty string, the same as "open" so that the distinction is there if anyone wants it.

JohnLukeBentley commented 8 years ago

OK, just a style issue.

But if you are going to correct the authoryear style (which is a good idea) for something like "unknown/2006" or "open/2006" then the main thing to do would be ensure the end date is not suppressed in the output.

% Bad
"n.d" % The current output
"" % What seems to be your idea

% Good
"/2006" % A marriage of your idea with not suppressing the ending date

But I agree that providing internal markers that distinguish between "unknown" and "open" is going to help style authors who might want it; and that, for native biblatex styles, mapping these both to a single output, the empty string, is probably the way to go.

That way "n.d." could be understood as "No date, nor dates" (edit: because it would only pop up with respect to an empty date field).

Edit: And it might be these sort of changes occur in the core and flow through to the native styles for free?

JohnLukeBentley commented 8 years ago

Bug: In the bibliography the season is output when alldates=year.

Biblatex-Bug-SeasonShownWhenDateOptionIsYear-Mwe.tex

\documentclass{article}

\begin{filecontents}{\jobname.bib}
@article{barker_2016_swiss,
  author = {Barker, Anne},
  title = {Swiss Voters Say No to Guaranteed Free Money},
  date = {2016-06-06T18:20:00+10:00},
  journaltitle = {ABC News},
}

@book{sep_2016_stanford,
  author = {{SEP}},
  date = {2016-23},
  title = {The Stanford Encyclopedia of Philosophy},
  editor = {Zalta, Edward N.},
}
\end{filecontents}

\usepackage[%
  alldates=year, %year, short, long, terse, comp, ymd, edtf
%  alldatesusetime=false,
  alltimes=24h,
  seconds=true,
  timezones=true,
  datezeros=true,
  dateabbrev=false,
  style=authoryear,
  sorting=none,
  backend=biber,
]{biblatex}

\addbibresource{\jobname.bib}

\begin{document}
\autocite[see][under `We already have social services']{barker_2016_swiss} \\  
\autocite{sep_2016_stanford} \\
\printbibliography
\end{document}

Output (For the bibliography)

Barker, Anne (2016). "Swiss Voters Say No to Guaranteed Free Money". In: ABC News. SEP (Autumn 2016). The Stanford Encyclopedia of Philosophy. Ed. by Edward N. Zalta

The Barker entry is as expected. For the SEP entry I expect "SEP (2016) ...", rather than "SEP (Autumn 2016) ..."

plk commented 8 years ago

Fixed in 3.7 dev.

JohnLukeBentley commented 8 years ago

Looks good at my end. Thanks.

My testing continues. This mainly consists of my building my custom style to move closer to the goal of "Getting everything I want in terms of date output". The idea is that this might throw up exotic and bug revealing cases in the core, as it did above.

I remain committed to report back when this is all complete.

JohnLukeBentley commented 8 years ago

Bug? \citefield does not work with the date field.

Biblatex-Bug-Citefield-Mwe.tex

\documentclass{article}

\begin{filecontents}{\jobname.bib}
@article{barker_2016_swiss,
  author = {Barker, Anne},
  title = {Swiss Voters Say No to Guaranteed Free Money},
  date = {2016-06-06T18:20:00+10:00},
  journaltitle = {ABC News},
}

\end{filecontents}

\usepackage[%
  alldates=ymd, % year, short, long, terse, comp, ymd, edtf
  alldatesusetime=true,
  alltimes=24h,
  seconds=true,
  timezones=true,
  datezeros=true,
  dateabbrev=false,
  style=authoryear,
  sorting=none,
]{biblatex}

\addbibresource{\jobname.bib}

\begin{document}
\autocite{barker_2016_swiss} \\  
\citefield{barker_2016_swiss}{title}\\
\citefield{barker_2016_swiss}{date}\\
\printbibliography
\end{document}

Output of document body (citation stuff)

[leading space] (Barker 2016) Swiss Voters Say No to Guaranteed Free Money date

Expect

[leading space] (Barker 2016) Swiss Voters Say No to Guaranteed Free Money 2016-06-06T18:20:00+10:00

... or a formated date (?).

My motive for using \citefield{key}{date} was to output the raw date field to compare that against formatted results, as when using \autocite and \printbibliography (edit: for testing purposes).

I suppose if you where to create, in the Biblatex core, (or there already are) \DeclareFieldFormat commands for date formatting, I'd have to use \citefield{key}[citefield]{date} (or similar?) to get the raw format. That is, after getting \citefield{key}{date} to work at all.

plk commented 8 years ago

date isn't a biblatex field - biber splits this into the correct year, month etc. fields which are available in biblatex.

JohnLukeBentley commented 8 years ago

Would it be a pain for you to make date available as a biblatex field, in addition to what you are currently doing to split these up up into year, month, etc? (I don't think I'd be after origdate, eventdate, urldate, enddate although consistency might require you implement all or none).

The only use I can think of for this is in testing. That is, if the implementation would be non trivial I can live without it.

plk commented 8 years ago

It is easy to do but it would cause all sorts of confusion I think.

JohnLukeBentley commented 8 years ago

Well I remain confused why it's not there. I mean I would have thought that \citefield would be able reference all the fields listed under biblatex.pdf, "2.2.2 Data Fields" (starting page 16), whatever else it might also be able to reference (e.g. derived fields like year, endyear, etc). Especially since it's listed as a "low-level" command.

Conceptually I had in mind that any legal field you might put in a .bib file would be a "biblatex field" (or historically a "bibtex field"). And that conceptualization seems consistent with "The name on the left is the default data model name of the field as used by biblatex and its backend" (biblatex.pdf, "2.2.2 Data Fields", p. 16). ... and the description for \citefield contains "The last argument is the name of a <field>, in the sense explained in § 2.2 [2.2 Entry Fields, p 13]."

I'm guessing that you are concerned about confusion in user's minds between date as it's normally output (in all sorts of strictly controlled and specially optioned ways, e.g "26th July 2016 3:00 pm") and date output in raw form (e.g. "2016-07-26T15:00"). ?

Or perhaps you are thinking of the risk that style authors might try something like \printfield{date} (which won't work) rather than \printlabeldateextra\bibdatetimesep\printlabeltime or just \printdate ?

That is, perhaps you are wanting to promote component date parts and other specialized date commands, and that having an outputable date field, where output reflects the raw input, corrupts that promotion. ?

plk commented 8 years ago

Yes, that's it - people will start to use it and bypass all of the date code. Since in the most obvious cases, it splits into a year field at the very least, it's the same as it's always been for bibtex users. I am loathe to make it available just for tests when it could easily be copied to a literal custom field like usera with a sourcemap and printed with \printfield for tests.

JohnLukeBentley commented 8 years ago

Plk, that's a brilliant workaround. I wouldn't have thought of it myself.

What follows is the implementation of it. This shows it working in the general case. However, it throws up a quirky issue around the display of tildes ~ (EDTF circa's).

\documentclass{article}

\begin{filecontents}{\jobname.bib}
@article{barker_2016_swiss,
  author = {Barker, Anne},
  title = {Swiss Voters Say No to Guaranteed Free Money},
  date = {2016-06-06T18:20:00+10:00},
  origdate = {1322},
  journaltitle = {ABC News}
}
@book{plato_2004_republic,
  author = {{Plato}},
  date = {-0379~},
  title = {Republic},
  translator = {Reeve, C. D. C.},
  publisher = {{Hackett Publishing Company, Inc.}},
}
\end{filecontents}

\usepackage[%
  alldates=ymd, % year, short, long, terse, comp, ymd, edtf
  alldatesusetime=true,
  alltimes=24h,
  seconds=true,
  timezones=true,
  datezeros=true,
  dateabbrev=false,
  dateera=secular,
  datecirca=true,
  dateuncertain=true,
  style=authoryear,
  sorting=none,
]{biblatex}

\addbibresource{\jobname.bib}

\DeclareSourcemap{
  \maps[datatype=bibtex]{
    \map[overwrite]{
      % Set usera to date so 
      % the raw input date can be accessed with \citefield{key}{usera}
      \step[fieldsource=date]
      \step[fieldset=usera, origfieldval]
    }
  }
}

\newcommand{\mytilde}{\raise.17ex\hbox{$\scriptstyle\mathtt{\sim}$}}

\begin{document}

\section{Date to usera mapping in general}
\autocite{barker_2016_swiss} \\  
\citefield{barker_2016_swiss}{usera}\\

\section{Tilde issue}
Lorem \mytilde \\
\autocite{plato_2004_republic}\\
\citefield{plato_2004_republic}{usera}\\

\end{document}

Output

1 Date to usera mapping in general (Barker 2016) 2016-06-06T18:20:00+10:00

2 Tilde issue Lorem ∼ (Plato ca. 0380 BCE) -0379

I gather that outputting tilde's in latex is a problem general to latex. Above I demonstrate one of the (random) solutions I could find on the web to solve it for latex in general.

The problem when using \citefield{plato_2004_republic}{usera} is that it displays "-0379" when we'd expect "-0379~".

I raise the issue not as something to fix for my case, my testing of circa dates with `\citefield{plato_2004_republic}{usera} is so obscure that it is unlikely to repeat itself: at this point, in my larger test file, I just copy and paste the input date. I raise the issue, rather, in case you can think of any other unintended consequences, where the display of a tilde might be required (I can't think of such cases).

njbart commented 8 years ago

You could map date to a “verbatim field” such as verba instead.

JohnLukeBentley commented 8 years ago

Yes, as a matter of choosing the right name for things "verbatim fields", looking at the biblatex.pdf, seems to specially target this purpose.

Incidentally, did you change your username or am I imagining things?