[ENH] Implement Histogram Conditional Density Estimation

[ShreeshaM07]

Describe the solution you'd like

Histogram estimation is not present in skpro. Implement them from scratch using the conditional density estimate finding the optimal binwidth(h) and find the function that fits the histograms most aptly without over smoothing nor undersmoothing.

Additional context

Useful resources

• Wasserman - All of statistics(chapter 20.2)
• https://www.youtube.com/watch?v=SUvPJ4URYGA

[ShreeshaM07]

@fkiraly Do you recommend any other resources to refer for implementing this?

Also do I have to implement the Kernel Density Estimation for Gaussian, tophat for this histogram?

[fkiraly]

Sure!

Some classical ones:

• Hyndman et al - Estimating and visualizing conditional densities
• Efromovich - Conditional density estimation in a regression setting
• Bashtannyk, Hyndman - Bandwidth selection for kernel conditional density estimation

This is mostly kernel based.

Also, what is tophat?

[ShreeshaM07]

Also, what is tophat?

Its a type of kernel in sklearn KDE the K(x,h) is proportional to 1 for x<h.

It mostly resembles a bin itself.

[fkiraly]

Oh, I see, the "top-hat kernel", which is the same as a box kernel. It corresponds to a uniform distribution.

Here are a few things I noticed:

• for histogram CDE, we need a histogram distribution - these are not available in skpro yet. That might be a simpler implementation item, as it does not hinge on design decisions? Issue here: https://github.com/sktime/skpro/issues/323
• there is also a subtask of representing kernel mixtures, not all of these are distribution mixtures. There are a few design decisions to think about, this is a more difficult issue. Issue opened here: https://github.com/sktime/skpro/issues/324
• Given that CDE literature is a bit dense and not entirely straightforward to get a grip on - is it perhaps easier to start with unconditional density estimation, e.g., looking at the sklearn and similar estimators for DE and the "right interfaces"? That would be a bit of a pivot, but might help arriving at the right design and implementation choices step-by-step.