Open gdementen opened 1 year ago
@alixdamman : what do you think?
- raise an error if fewer or too many separators detected. Safest (zero chance to have bad labels pass unnoticed) but not practical (it leaves the problem to the user)
- use maxsplit=len(names) - 1 in np.char.split. This would solve the extra separators issue but not the fewer separators issues. Also it could mask a problem if there are extra seps in some labels that the user does not know about. I can live with the later problem though.
- or configurable maxsplit, defaulting to the above. More flexible, but same problems as option 1.
- or force splitting using the maximum number of separators (i.e. complete the sequences with as many '' as necessary). In the presence of too many labels, it will probably not be the result the user wants in most cases.
- force splitting using len(names) - 1 (i.e use maxsplit but then complete the too short sequences)
- configurable force splitting (e.g. nsplit argument), using len(names) - 1 as default. My preferred solution at this point.
- configurable force splitting , but with a default which raises if not all sequence have the same length. I hesitate with this option.
Honestly, without an example of the resulting axes for each option, it is difficult to me to evaluate what would be the best solution. I am usually in favor of somewhat "forcing" the users to clean their code (i.e. check the labels, dimensions of arrays, ...), so, by default, my answer is option 1. But if you think that there are potentially many cases in which option 1 could not be practical, I am OK to choose the option 6.
In that case, I'd go for option 7, which is basically option 1 with an easy way to deal with the exception when it happens. I will try to implement that.
Implementing nsplit is not that urgent. Avoiding the silent failure is the urgent part so I extracted #1089 out of this and scaled down the priority.
Besides, to be complete, we should implement a way to split right (rsplit boolean argument)?
When the number of separators is not the same in all labels, labels which have more separators have their last part dropped silently and any label which becomes a duplicate because of this is dropped silently too (it seems like the last duplicate wins).
Also, when there is a label with fewer separators than expected, it raises a very weird error.
The two issues are because the zip(*sequences) idiom that we use happily ignores sequences longer than the shortest of the sequences.
The code of Axis.split is roughly like this:
Our options are :
nsplit
argument), using len(names) - 1 as default.