search

hpcflow / hpcflow-new

Mozilla Public License 2.0
0 stars 5 forks source link

Sets aren't ordered #691

Open dkfellows opened 1 month ago

dkfellows commented 1 month ago

Fact: in no version of Python are sets guaranteed to be insertion order preserving. (I think they ought to give that guarantee, but they never have done.)

In hpcflow/sdk/persistence/pending.py, in CommitResourceMap.group_by_resource(), this line appears:

https://github.com/hpcflow/hpcflow-new/blob/50fd0010fecd2bdd2fc2abe2aaceddeec39015b7/hpcflow/sdk/persistence/pending.py#L617

That is then used as a dictionary key (after conversion to a tuple) in groups. Unfortunately, this means that the elements of the tuple used as the key can appear in any order, making the dictionary not work right. It can appear to work right for a long time too (as it's totally dependent on the internal implementation of sets, which is actually quite weird in Python). Yes, I've been bitten by this in the past!

If the key of that dictionary really is a set of things, use a frozenset as the key collection type, not a tuple.

aplowman commented 1 month ago

Good catch!

dkfellows commented 1 week ago

An alternative I've found/remembered for doing most of what you'd want an ordered set for: use dict.fromkeys(iterable) (the values will be None). It doesn't support exactly the same API (another project I was on had a wrapper to fix that that I could dig up if required) but for many of the uses that's not important.