search

scikit-hep / awkward

Manipulate JSON-like data with NumPy-like idioms.
https://awkward-array.org
BSD 3-Clause "New" or "Revised" License
827 stars 85 forks source link

`Indexed._mergeable_next` does not consider `self.parameters` when merging against a non-indexed type #2537

Open agoose77 opened 1 year ago

agoose77 commented 1 year ago

Description of new feature

TL;DR — if an indexed type has type parameters, then we can't just push mergeable checks to the content.

There is some ambiguity here, but I think this is incorrect. We only allow non-union merging when arrays have the same nominal type, so even if IndexedArray.content shares the same value of __array__ as other, the externally visible types don't agree.

Currently, this isn't too visible; we end up introducing unions that simplify away after parameters are merged. But, this involves multiple unnecessary allocations and kernel calls.

jpivarski commented 1 week ago

Currently, this isn't too visible; we end up introducing unions that simplify away after parameters are merged. But, this involves multiple unnecessary allocations and kernel calls.

It sounds like that means this is a (potential/unmeasured) performance issue, rather than a bug, right? If I understand what you're saying, the outputs are what they ought to be, given the inputs.