Attribute "Date" fields should be defined as a "string" type

[benloh]

When defining a custom attribute type you have the option of three types:

• boolean
• number
• string

There is no built-in support for Dates. So if you define a custom attribute date field, e.g. "Date of Birth" or "Date or Printing" etc, you need to use a string. And any filtering options or table sorting options will be string based.

Using a number does not work because the input fields use validation and only allow numbers, e.g. you can't enter "4/2/24".

Do we need to add support for a native date field? Time also?
This starts to get complicated. e.g. sometimes you want only date, sometimes only time, and sometimes you want date-time. So if we were going to do this, we probably would need three types of fields.

Or we can keep things simple and make sure templates use "strings"?

[jdanish]

The two key concerns are:

1: We want consistent input across users for easier reading (e.g., if one kid writes 8/14 and one writes Aug, 14 it's hard to see them as the same)

2: We want filters to allow: before, after, and maybe "between" (and by extension, sort up or down)

So, that makes me think that dates will be ideal, rather than try and layer that on top of strings?

Everything I come up with to not have all 3 formats is a hack like "have the time, set it to 12:00, but don't display it." Which makes me think that you are right about needing all 3.

I glanced at the gephi documentation online and they seem to use date, datetime and then if you want time on its own you use number. I'm guessing it's not a major time savings (no pun intended) so we might as well be robust if you are doing the work anyhow?

[jdanish]

@benloh I just confirmed this with Kalani. One minor addition / thought: for Historians, it is apparently REALLY annoying to use a calendar selection tool for cases where they may not have a specific detail (e.g., might know a year and not the date). Therefore, I think the easiest compromise on user interface would be to have a drop-down / text field for each component where there is an option for not setting one. So:

• Month: drop-down with 12 entries taken from template (some templates might be 1-12, others January-December). Default to January- December for now because kids.
• Day: number field with range 1-31 if set
• Year: 4 digit number and a drop-down that takes it's 2 values and default from the template (normally CE as default and BCE as an option, but could be AD, or default to BCE for some cases). I suppose if it is easy, having an option for kid-friendly that just leaves off the drop-down but assumes some template based choice (CE) is cool too. As noted before, any time we need to filter or sort, treat unspecified as 0 (so it'll be before something specified as 1).

For export purposes, let's use UTC.

Actually, from this perspective, we only need one date type, and all of the options could be set to unspecified, but if we don't need them we may not want them appearing in the choices. That is, if we want kids to add only a year, we can store it as being YYYY0000 etc., but we wouldn't want to show the kids anything beyond the year box. So it might be that one date format is all we need, but that we want some display / choice options?

[benloh]

@jdanish Alright. I think if you want to do this, we need to go all in. This is complicated from an internal data representation perspective, and a flexible UI perspective, as well as template definition perspective, not to mention filtering and sorting.

There are whole libraries dedicated to this, but I think we'll have to roll our own version to capture all the nuances.

I'm guessing it's probably about 2 to 3 weeks of calendar time to implement this.

[jdanish]

OK. I think it's worth it. Does that throw off our budget / timeline? I don't have a sense of how 2-3 weeks fits or not.

[benloh]

[!CAUTION] Working notes. Please ignore

Review date frameworks

• Temporal Spec
  • Temporal Polyfill
• moment.js -- legacy project, in maintenenace mode. Medium article about moment.js
• luxon -- replacement for moment.js, can handle math
• AstroTime
• Chrono -- natural language date parser
• ISO 8601 String

Design

A flexible date format tha allows for:

• implied times (e.g. 1900s)
  • optional no timezones -- don't imply time zones if time zones were not specified in the source data
• handles eras (e.g. BCE, CE) and math

How this would work

1. Users open a node for editing and click on a date field.
2. The date field is an open text field: you can type (almost) anything, e.g.:

   1921
   4/1921
   apr 1921
   4000 bce

3. chrono will interpret the value and preview the result for you. This way you get feedback about what the interpretation is. Ideally the interpretation allows for non-specified values, e.g. 4/1921 will be interpreted as April 1921, NOT April 1, 1921.
4. The input string will be displayed as-is.
5. You have the option of selecting a display format via a ... or view as... menu. e.g. entering 4/1921 will show

   4/1921 (as entered)
   4/1921 (DATE_SHORT)
   Apr 1921 (DATE_MED)
   April 1921 (DATE_FULL)
   ...

   The format is sensitive to the values that have and have NOT been specified. e.g. since "DAY" is not specified, the selected format will now unnecessarily interpet an implied day of "1". This menu might get rather long.
   We can also use historical era notation, e.g. jun 1921 bce will be interpreted as:

   jun 1921 bce (as entered)
   6/1921 BCE (HISTORICAL_DATE_SHORT)
   Jun 1921 BCE (HISTORICAL_DATE_MED)
   June 1921 BCE (HISTORICAL_DATE_LONG)
   ...

   We only add the historical date as an option here because bce was entered in the text input field.

See this codepen chrono demo to get a feel for how it interprets dates. Note that I believe we can have chrono deliberately ignore implied dates (e.g. 4/1921 does not have to be interpreted as 4/1/1921)

Handling Common Era Notation

BCE dates for filtering/sorting can be supported by using negative dates.

Internal Data Representation

HistoricalDateTimeClass

• era
• year
• month
• day
• hour
• minute
• second
• timezone

Setters

• setEra
• setYear

Internally, the HistoricalDateTime representation is a class with 8 properties (listed above), any of which may be undefined.

HistoricalDateTimeVObject

• displayFormat -- e.g. DATE_SHORT vs DATE_MED

Input Style Selector?

In addition to being able to select any of the internal data fields individually, combinations are also useful:

• HistoricalDateTime -- PlainDateTime + era
• HistoricalYear -- PlainYear + era
• HistoricalYearMonth -- PlainYearMonth + era
• ZonedDateTime -- PlainDateTime + offset + TimeZone
• PlainDateTime
• PlainDate
• PlainTime
• PlainMonthDay
• PlainYearMonth
• PlainYear

See temporal for inspiration and language.

Formatting Output

Support luxon outputs? e.g.

• DATE_SHORT -- "10/14/1983"
• DATE_MED -- "Oct 14, 1983"
• DATE_FULL -- "October 14, 1983"

Since the data representation is internal, you can easily select different formats for the ouptut, e.g. change from DATE_SHORT to DATE_MED without changing the data.

UI: Selecting a HistoricalDateTime

Selecting a date

Other use cases

Fuzzy Dates

• before <date>
• after <date>
• around <date>

Intervals

• between <date1> and <date2>

Caveats

• Assume Gregorian calendar -- we're not planning supporting non-standard/non-western calendars? Or do we need to support stuff like Mayan calendar time (perhaps simply using a plain year without era would suffice)?

---

TO DO

• [ ] See ChatGPT demos of extending chrono with eras
• [ ] Chrono Custom Plugins
  • [ ] Add support for plain year
  • [ ] Add support for eras

[benloh]

@jdanish @kalanicraig In working through the historical dates, I have a few questions about how those dates are used.

I'm assuming that most historical dates (e.g. dates that require an eras designation) would follow these rules (from [BC and AD, BCE and CE: What’s the Difference?](https://www.antidote.info/en/blog/reports/bc-and-ad-bce-and-ce-whats-difference#:~:text=CE%20stands%20for%20%E2%80%9Ccommon%20(or,at%20least%20the%20early%201700s.)

• BC should appear after the numerical year, while AD should appear before it. 1100 BC, AD 1066

• BCE and CE should both appear after the numerical year. 1100 BCE, 1066 CE

• As is the case with most initialisms, periods may be used after each letter. 1100 B.C., A.D. 1066, 1100 B.C.E., 1066 C.E.

A few questions:

1. Other than BCE/CE and BC/AD, are there other eras designations that we need to support?
2. Is it important to follow the "AD" comes before the year, but "BC" after the year formatting?
3. Are months or days ever included? And if so, what format would the be? e.g. would you ever use:

   Mar 1100 BC
   Mar 2 1100 BC
   Mar 2 1100 B.C.
   2 Mar 1100 BC
   Mar AD 1100
   AD Mar 1100
   Mar 2 AD 1100
   AD Mar 2 1100
   2 Mar AD 1100
   AD 2 Mar AD 1100

Any other common formats that we need to support?

[kalanicraig]

Dates are complex, and it just programmatically

Let's assume the historical need is driving the software first. With that assumption in mind, we'll want to accommodate a variety of historic dates, which means ISO and Gregorian but also Islamic, Jewish, Chinese, etc. However: almost all dating systems have a year, month, day.

What I'd suggest is that rather than trying to validate every date type and display every date type, we:

1. Assume the data is saved in an ISO-style date: YYYY MM DD
2. Ask if each piece of the date is approximate (e.g. if YYYY is approximate, no need for MM or DD value; same for MM and DD)
3. Allow negative values to accommodate BC

Does that make sense?

—k

On Wed, Jun 12, 2024 at 2:17 PM benloh @.***> wrote:

@jdanish https://github.com/jdanish @kalanicraig https://github.com/kalanicraig In working through the historical dates, I have a few questions about how those dates are used.

I'm assuming that most historical dates (e.g. dates that require an eras designation) would follow these rules (from [BC and AD, BCE and CE: What’s the Difference?]( https://www.antidote.info/en/blog/reports/bc-and-ad-bce-and-ce-whats-difference#:~:text=CE%20stands%20for%20%E2%80%9Ccommon%20(or,at%20least%20the%20early%201700s.)

-

BC should appear after the numerical year, while AD should appear before it. 1100 BC, AD 1066

BCE and CE should both appear after the numerical year. 1100 BCE, 1066 CE

As is the case with most initialisms, periods may be used after each letter. 1100 B.C., A.D. 1066, 1100 B.C.E., 1066 C.E.

A few questions:

1. Other than BCE/CE and BC/AD, are there other eras designations that we need to support?
2. Is it important to follow the "AD" comes before the year, but "BC" after the year formatting?
3. Are months or days ever included? And if so, what format would the be? e.g. would you ever use:

Mar 1100 BC Mar 2 1100 BC Mar 2 1100 B.C. 2 Mar 1100 BC Mar AD 1100 AD Mar 1100 Mar 2 AD 1100 AD Mar 2 1100 2 Mar AD 1100 AD 2 Mar AD 1100

Any other common formats that we need to support?

— Reply to this email directly, view it on GitHub https://github.com/netcreateorg/netcreate-itest/issues/207#issuecomment-2163644906, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACKL4NF47WGJID3P2KZGM5TZHCGCPAVCNFSM6AAAAABHZNKF6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRTGY2DIOJQGY . You are receiving this because you were mentioned.Message ID: @.***>

[benloh]

We actually have a nifty library called chronos that can let us input times in a variety of formats and then have the system interpret and validate the date as a date object, and then display that date in a variety of formats. For example:

1. Type in apr 1
2. The system validates the date as month:4 day:1
3. You can then select the output to be 04/01 or April 1 or any number of different options.

Similarly:

1. Type in 10000 bc
2. The system validates the date as year:10000
3. Select output to be 10,000 B.C.E or 10000 BC etc.

And it can potentially fall back to an interpreted date, e.g.

1. Type in Around 10000 bc
2. The system validates the date as year:10000, ignoring the 'Around' text
3. Select output to be 10,000 B.C.E or 10000 BC etc.

You can then decide whether you want the date display in the node to use the date as entered, or as some specific formatted string (defined in the project template), or perhaps even a flexible value that you can select dynamically.

Advantages:

• This would give you the maximum amount of flexibility and ensure that we can do math on the values.
• This would also give you more flexibility when entering dates instead of being limited strictly to a YYYY/MM/DD format (and you can enter years beyond 4 digits).
• And, the validation helps disambiguate whether 24 is day or a year.
• It supports interpretation of the text (e.g. your string could be "year of our lord 1742" would be interpreted as "AD 1740") and can potentially support different locales, e.g. 10/12/2024 is Dec 10 in GB.

It does make choosing dates require a little extra work.

---

But I might be overthinking this and a simple date field that has separate five dimensions is more appropriate?

• year (does it need to support more than 4 digits?)
• month (selectable as either numeric or alphabetical)
• day
• era (BC, BCE, B.C.E)
• calendar (ISO, Gregorian, Jewish, Chinese)

[jdanish]

I don’t mind the extra work to edit the template if Kalani thinks it’d be helpful for historians…. For kids I could see arguments both ways. Is it a major difference in effort required?

[kalanicraig]

I lean toward the library that lets us actually do date sorting without choosing a dominant system. For instance the standard Islamic calendar doesn’t have a BC.

On Wed, Jun 12, 2024 at 6:04 PM Joshua Danish @.***> wrote:

I don’t mind the extra work to edit the template if Kalani thinks it’d be helpful for historians…. For kids I could see arguments both ways. Is it a major difference in effort required?

— Reply to this email directly, view it on GitHub https://github.com/netcreateorg/netcreate-itest/issues/207#issuecomment-2163977085, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACKL4NDB645CFP5HJISXWTLZHDAWDAVCNFSM6AAAAABHZNKF6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRTHE3TOMBYGU . You are receiving this because you were mentioned.Message ID: @.***>

[benloh]

Partly implemented. See #228