Built in checks for Time series dataframes

[vignkri]

Is your feature request related to a problem? Please describe.

DataFrames indexed by either DateTimeIndex or derived indexes need to have specialised tests verifications. There are 3 different types of time based indexes supported in Pandas.

• DateTimeIndex
• PeriodIndex
• TimedeltaIndex

They have different internal type structure depending on the type. Adding built-in checks on top of the different indexes could be useful.

Describe the solution you'd like

• Tests on the sampling frequency of the data based on the frequency property available on the DateTimeIndex
  • A DateTimeIndex with frequency can be implicitly converted to a PeriodIndex and then the index is transformed to an explicit PeriodIndex
• Tests on length of data based on frequency
  • A time-series data-frame for one-day at an hourly frequency should ideally have a length of 24 elements, in case of missing elements, the value is internally represented as NaN.
• Tests on timezone on the time-span
  • There are edge cases when DateTime objects span 2 days when data is from midnight to midnight with DST enabled
  • For example, with Central European Summer Time, the time is +02:00 from UTC, so midnight to midnight on the local time would be as follows:
  • FROM: 2022-04-29T00:00:00+02:00 - in UTC 2022-04-28T22:00:00+00:00
  • TO: 2022-04-29T23:00:00+02:00 - in UTC 2022-04-29T21:00:00+00:00`.

Describe alternatives you've considered

• Manual verifications on the data using .shape property based on the property of the data being accessed
• Manual timezone verifications on the data
• Assertions on timezone changes

Additional context Add any other context or screenshots about the feature request here.

Edits

• typos and errors

[cosmicBboy]

Amazing @vignkri !

Super excited to get built-in support for this :)

So for each of the tests you list above, it would help if you could provide code snippets that implement the checks using the vanilla pandas API (ideally on a pd.Series of the DateTime, Period, and Timedelta dtypes)... and once we have those it'll be super straight-forward to add them to the pa.Check class namespace.

[tkaraouzene]

Hello !

Any update on this topic I would be very interested to use this kind of feature. I would add some additional tests for DatetimeIndex:

1. check no duplicates
2. check index is monotonic increasing

[jonmather]

I would also love to get this feature added! Anything that's blocking getting this done? I can provide snippets for some of these tests if helpful

[cosmicBboy]

Contributions welcome 🤗