ENH: date_range not working as it intuitively should when specifying start, end, and periods

[onnoeberhard]

Code Sample, a copy-pastable example if possible

>>> start = pd.Timestamp('2008-01-02 07:51:37.999477')
>>> end = start + pd.Timedelta('2 hours')
>>> pd.date_range(start, end, periods=1000)    # Intuitively a linearly spaced time series
Traceback (most recent call last):

  File "<ipython-input-69-2304a28824c6>", line 1, in <module>
    pd.date_range(start, end, periods=1000)

  File "E:\Anaconda3\lib\site-packages\pandas\core\indexes\datetimes.py", line 2057, in date_range
    closed=closed, **kwargs)

  File "E:\Anaconda3\lib\site-packages\pandas\util\_decorators.py", line 118, in wrapper
    return func(*args, **kwargs)

  File "E:\Anaconda3\lib\site-packages\pandas\core\indexes\datetimes.py", line 324, in __new__
    ambiguous=ambiguous)

  File "E:\Anaconda3\lib\site-packages\pandas\core\indexes\datetimes.py", line 421, in _generate
    raise ValueError('Of the three parameters: start, end, and '

ValueError: Of the three parameters: start, end, and periods, exactly two must be specified

Problem description

I need a DatetimeIndex object to later use as index in a Series. DatetimeIndex should start at start, end at end and have a fixed number of elements (1000). Intuitively, this should work with pd.date_range, but it doesn't, and I haven't found a good explanation about why this is the case. I have found a workaround on Stackoverflow (https://stackoverflow.com/questions/25796030/how-can-i-use-pandas-date-range-to-obtain-a-time-series-with-n-specified-perio) that does work:

>>> start = pd.Timestamp('2008-01-02 07:51:37.999477')
>>> end = start + pd.Timedelta('2 hours')
>>> pd.to_datetime(np.linspace(start.value, end.value, 1000))
DatetimeIndex(['2008-01-02 07:51:37.999476992',
               '2008-01-02 07:51:45.206684160',
               '2008-01-02 07:51:52.413891328',
               '2008-01-02 07:51:59.621098496',
               '2008-01-02 07:52:06.828305920',
               '2008-01-02 07:52:14.035513088',
               '2008-01-02 07:52:21.242720256',
               '2008-01-02 07:52:28.449927424',
               '2008-01-02 07:52:35.657134592',
               '2008-01-02 07:52:42.864341760',
               ...
               '2008-01-02 09:50:33.134612224',
               '2008-01-02 09:50:40.341819392',
               '2008-01-02 09:50:47.549026560',
               '2008-01-02 09:50:54.756233728',
               '2008-01-02 09:51:01.963440896',
               '2008-01-02 09:51:09.170648064',
               '2008-01-02 09:51:16.377855488',
               '2008-01-02 09:51:23.585062656',
               '2008-01-02 09:51:30.792269824',
               '2008-01-02 09:51:37.999476992'],
              dtype='datetime64[ns]', length=1000, freq=None)

Expected Output

>>> start = pd.Timestamp('2008-01-02 07:51:37.999477')
>>> end = start + pd.Timedelta('2 hours')
>>> pd.date_range(start, end, periods=1000)
DatetimeIndex(['2008-01-02 07:51:37.999476992',
               '2008-01-02 07:51:45.206684160',
               '2008-01-02 07:51:52.413891328',
               '2008-01-02 07:51:59.621098496',
               '2008-01-02 07:52:06.828305920',
               '2008-01-02 07:52:14.035513088',
               '2008-01-02 07:52:21.242720256',
               '2008-01-02 07:52:28.449927424',
               '2008-01-02 07:52:35.657134592',
               '2008-01-02 07:52:42.864341760',
               ...
               '2008-01-02 09:50:33.134612224',
               '2008-01-02 09:50:40.341819392',
               '2008-01-02 09:50:47.549026560',
               '2008-01-02 09:50:54.756233728',
               '2008-01-02 09:51:01.963440896',
               '2008-01-02 09:51:09.170648064',
               '2008-01-02 09:51:16.377855488',
               '2008-01-02 09:51:23.585062656',
               '2008-01-02 09:51:30.792269824',
               '2008-01-02 09:51:37.999476992'],
              dtype='datetime64[ns]', length=1000, freq=None)

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.4.final.0 python-bits: 64 OS: Windows OS-release: 7 machine: AMD64 processor: Intel64 Family 6 Model 44 Stepping 2, GenuineIntel byteorder: little LC_ALL: None LANG: en LOCALE: None.None pandas: 0.22.0 pytest: 3.3.2 pip: 9.0.1 setuptools: 38.4.0 Cython: 0.27.3 numpy: 1.14.2 scipy: 1.0.1 pyarrow: None xarray: None IPython: 6.2.1 sphinx: 1.6.6 patsy: 0.5.0 dateutil: 2.6.1 pytz: 2017.3 blosc: None bottleneck: 1.2.1 tables: 3.4.2 numexpr: 2.6.4 feather: None matplotlib: 2.1.2 openpyxl: 2.4.10 xlrd: 1.1.0 xlwt: 1.3.0 xlsxwriter: 1.0.2 lxml: 4.1.1 bs4: 4.6.0 html5lib: 0.9999999 sqlalchemy: 1.2.1 pymysql: None psycopg2: None jinja2: 2.10 s3fs: None fastparquet: None pandas_gbq: None pandas_datareader: None

[jreback]

I suppose if freq is NOT specified then you could accept all three and give a linspace repr. What breaks if you do that?

If freq is specified then this should for sure raise. The idea of the error is simply to be helpful in telling you that you may have overspecified the args.

[chris-b1]

Your workaround from SO is a pretty reasonable solution, not sure we should support this. I definitely would not want to change the default behavior, but suppose could with with something like freq=None or freq='interpolate'

[onnoeberhard]

I agree to definetely don't change the default behaviour. As I see, the default freq is 'D', so this:

pd.date_range(start, end, periods=1000)

would not work, because it is the same as

pd.date_range(start, end, periods=1000, freq='D')

which really should not work. However, if the user explicitly sets freq=None, the linspace behaviour would be practical:

pd.date_range(start, end, periods=1000, freq=None)

I suggest a simple change in the date_range function like so:

def date_range(start=None, end=None, periods=None, freq='D', tz=None,
               normalize=False, name=None, closed=None, **kwargs):
    """
    ...
    """
    # Return a linearly spaced DatetimeIndex if `freq` is not set, but `start`, `end`, and `periods` are
    if start and end and periods and not freq:
        di = tools.to_datetime(np.linspace(start, end, periods), **kwargs)
        if name:
            di.name = name
        return di

    return DatetimeIndex(start=start, end=end, periods=periods,
                         freq=freq, tz=tz, normalize=normalize, name=name,
                         closed=closed, **kwargs)

together with appropriate changes in the docstring and test functions. This would provide the desired behaviour, while not changing anything else.

I am not yet a contributor, so I cannot implement this myself (unless I am made one :upside_down_face:)

Because the change is really just a convenience feature of the date_range function, I don't think it would be wise to implement this directly in the DatetimeIndex constructor, or in another function, like bdate_range.

[TomAugspurger]

FWI, we could change the default freq to None, and document that it's 'D' when only two of star, end, and freq are specified. That way pd.date_range(start, end, periods=100) will work.

I am not yet a contributor, so I cannot implement this myself (unless I am made one 🙃)

Anyone can make a pull request: http://pandas-docs.github.io/pandas-docs-travis/contributing.html

[onnoeberhard]

That's a good idea, I like it much better.

Anyone can make a pull request

Oh okay I didn't know that (never done this before), then I am going to try to implement it :)