Add AutoNumberedEnum to stdlib

[2df14aea-354c-46a6-8f5c-342c9f465901]

BPO	26988
Nosy	@warsaw, @rhettinger, @vstinner, @ethanfurman, @vedgar, @Vgr255, @johnthagen, @kennethreitz
Files	issue26988.stoneleaf.01.patch issue26988.stoneleaf.02.patch issue26988.stoneleaf.03.patch issue26988.stoneleaf.05.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/ethanfurman' closed_at = created_at = labels = ['type-feature', 'library'] title = 'Add AutoNumberedEnum to stdlib' updated_at = user = 'https://github.com/johnthagen' ``` bugs.python.org fields: ```python activity = actor = 'ethan.furman' assignee = 'ethan.furman' closed = True closed_date = closer = 'ethan.furman' components = ['Library (Lib)'] creation = creator = 'John Hagen' dependencies = [] files = ['43694', '43743', '43972', '43986'] hgrepos = [] issue_num = 26988 keywords = ['patch'] message_count = 62.0 messages = ['265214', '265215', '265216', '265218', '265237', '269476', '269521', '270039', '270048', '270049', '270051', '270054', '270056', '270071', '270088', '270107', '270117', '270137', '270142', '270148', '270151', '270199', '270201', '270212', '270219', '270238', '270523', '270524', '271811', '271855', '271868', '272071', '272874', '272876', '272885', '272901', '272932', '272941', '272951', '272952', '272971', '272982', '273006', '273039', '273040', '273064', '273065', '273076', '273084', '273092', '273103', '273105', '273109', '273118', '273125', '273126', '273127', '273131', '273134', '273145', '273154', '273188'] nosy_count = 11.0 nosy_names = ['barry', 'rhettinger', 'vstinner', 'eli.bendersky', 'ethan.furman', 'python-dev', 'veky', 'abarry', 'John Hagen', 'David Hagen', 'kennethreitz'] pr_nums = [] priority = 'normal' resolution = 'rejected' stage = 'resolved' status = 'closed' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue26988' versions = ['Python 3.6'] ```

[2df14aea-354c-46a6-8f5c-342c9f465901]

I suggest that the AutoNumberedEnum be added to the standard library for the following reasons:

1) Provides a fundamental tool for defining a unique, abstract set of coupled members 2) Avoids boilerplate @enum.unique for a very common use case of enumerations 3) The code already exists in the Python documentation, so it has been vetted at some level

The AutoNumberedEnum also allows the developer to make a clearer distinction between enumerations whose values have special meaning and those that do not.

Consider:

@enum.unique
class Color(enum.Enum):
    red = 1
    blue = 2
    green = 3

@enum.unique
class Shape(enum.Enum):
    """Member values denote number of sides."""
    circle = 1
    triangle = 3
    square = 4

With AutoNumberedEnum it's possible to better express the intent that the value of Color members does not hold special meaning, while Shape members do:

class Color(enum.AutoNumberedEnum):
    red = ()
    blue = ()
    green = ()

@enum.unique
class Shape(enum.Enum):
    """Member values denote number of sides."""
    circle = 1
    triangle = 3
    square = 4

For enumerations with many members (10s), there becomes a maintenance issue when inserting new enumerations into the list:

@enum.unique
class Color(enum.Enum):
    aquamarine = 1
    blue = 2
    fushia = 3
    # inserting a member here (perhaps because it's clearest to keep these in alphabetic order)
    # results in having to increment all following members
    ...
    green = 40
    red = 41

Most other languages have support for naming enumerations without explicitly declaring their values (albeit with specialized syntax that makes it cleaner):

C++: http://en.cppreference.com/w/cpp/language/enum C#: https://msdn.microsoft.com/en-us/library/sbbt4032.aspx Java: https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html Rust: https://doc.rust-lang.org/book/enums.html

Currently, a developer can copy the code from the Python docs into his or her project, or add a dependency on aenum. I would argue that it belongs in the standard library.

If there are objections to it being too magical, I would argue it's already spelled out in the docs, so it's being prescribed as a solution already. I think it should be very clear when you derive from "AutoNumberedEnum" what is going on.

The code is very simple, so I would hope maintenance would not be difficult.

[ethanfurman]

On the other hand, if you use aenum you can do:

class Color(AutoNumber):
    red
    green
    blue

And isn't that better*? ;)

• For those in danger of swallowing their tongue over the magic involved: The magic is turned off as soon as any descriptor is defined (property, function, etc.).

[warsaw]

Wow Ethan, that's quite interesting. I'm not sure if I like it or not. :)

It's better than having to assign a dummy value to the enum items, and if we did require that, I'd suggest just throwing away the values for autonumbering.

But not having to specify any values at all is probably better, although it does look weird!

/me goes to look at the magic you're using...

[ethanfurman]

It's in _EnumDict.__getitem; there's some duplication in __setitem for supporting Python 2 (although with 2 you have to use either the empty tuple, or some other handy thing that may go in the __doc__ attribute, for example).

[2df14aea-354c-46a6-8f5c-342c9f465901]

@Ethan, I hadn't tried to use the aenum AutoNumberEnum that way, but I agree with Barry that I like it. To me that is the ideal case we should shoot for as I think it's the best* long term and deviate only if practical concerns prevent it.

So I am +1 for empty member assignment and if that is rejected, +1 for = () assignment as at least it is a big step forward. I feel both solutions already have some "magic", so would lean toward the one that leads to the least amount of boilerplate.

As for the empty assignment, I have played around with something similar before and will throw out one con for it: static analyzers get really confused. PyCharm, for example, thinks this is has unresolved references in it:

class Color(AutoNumberEnum):
    red
    green
    blue

But the counter point is that if this is in the stdlib, static analyzer authors are much more likely to add a special case for it than if in a "non-official" third party package (PyCharm example: https://youtrack.jetbrains.com/issue/PY-19150).

Another piece of evidence to support inclusion is that Python already provides specialized Enum subclasses (like IntEnum). I've written a lot of Python code that uses Enums and haven't personally needed IntEnum yet, but would have used an AutoEnum many, many times.

• I am assuming here that a true "enum" keyword is out of the question at this point for Python, which would probably be even better.

[2df14aea-354c-46a6-8f5c-342c9f465901]

@Ethan/Barry what needs to be done now to accept or reject this for Python 3.6? Should I propose it onto python-dev? If accepted, would be nice to get it in before alpha 3 (~2 weeks). What's nice about this proposal is Ethan has already written the code, so it's just a matter of getting consensus to add it.

[warsaw]

I don't really think much more needs to be done other than agree on this tracker issue that it's something we want, and that Ethan's implementation is good enough. It's been a while since I looked that the code but it seemed good to me before, so if Ethan is happy with it going into the stdlib, so am I. As for the feature, it *is* a little magical, but not so much that it would be confusing or ambiguous, so I'm +1 on it.

I'll defer to Ethan for the final decision, but if we have tests and documentation, I'd be very happy to see this in 3.6.

[2df14aea-354c-46a6-8f5c-342c9f465901]

Is this something we want to get in before the next alpha in two days? Just wanted to bring up the deadline since this may be a feature people want to play around with during the alpha phase.

Ethan, I'm happy to help with documentation or anything else.

[ethanfurman]

There is one wrinkle with auto-numbering. Consider the following code:

class Color(AutoNumberEnum):
  red
  green
  blue
  @property
  def cap_name(self):
    return self.name.upper()

What happens with property?

• property is looked up in the class namespace
• it's not found, so is given the number 4
• it's then called to handle cap_name
• an exception is raised

As far as I can tell there is only one "good" way around this:

• store any names you don't want auto-numbered into an _ignore_ attribute (this only needs to be global and built-in names that are used before the first method is defined)

another option is to build a proxy around any found global/built-in objects and decide what to do based on whether those objects are immediately called, but that fails when the object is simply assigned for later use.

So, barring any other ideas to handle this problem, the above example should look like this:

class Color(AutoNumberEnum):
  _ignore_ = 'property'
  red
  green
  blue
  @property
  def cap_name(self):
    return self.name.upper()

Another advantage of using ignore is the ability to have temporary variables automatically discarded:

  class Period(timedelta, Enum):
    '''
    different lengths of time
    '''
    _ignore_ = 'Period i'
    Period = vars()
    for i in range(31):
      Period['day_%d' % i] = i, 'day'
    for i in range(15):
      Period['week_%d' % i] = i*7, 'week'
    for i in range(12):
      Period['month_%d' % i] = i*30, 'month'
    OneDay = day_1
    OneWeek = week_1
    def __new__(self, value, period):
      ...

and the final enumeration does not have the temp variables Period nor i.

Thoughts?

[warsaw]

On Jul 09, 2016, at 03:20 PM, Ethan Furman wrote:

As far as I can tell there is only one "good" way around this:

• store any names you don't want auto-numbered into an _ignore_ attribute (this only needs to be global and built-in names that are used before the first method is defined)

Seems reasonable, though why not a dunder, e.g. __ignore__?

_ignore_ = 'Period i'

Why space separated names inside a string instead of a sequence of strings?

[ethanfurman]

A standard feature of Enum is that either space separated strings or a list of strings is accepted any where either is.

_sunder names are the ones reserved for Enum use (such as _value, _name, and soon _order (for py2/py3 compatible code).

[2df14aea-354c-46a6-8f5c-342c9f465901]

What happens with property?

• property is looked up in the class namespace

Perhaps you've already considered this, I'm not intimately familiar with how classes are parsed and constructed but is it possible to determine if the object is a decorator? It already determines to stop auto-numbering when it hits the first method, could it stop when it hits the first decorator or method?

Being able to use temporaries is an interesting side effect, but I feel like that would be used less often than a @staticmethod, @property, or @classmethod over a method, in which case it becomes a little more complex.

That being said, I think either solution is valid.

[ethanfurman]

The problem with testing the type of object a name refers to outside the class is it then becomes more difficult to make that an Enum member:

class AddressType(Enum):
    pobox
    mailbox  # third-party po box
    property

Having to assign a value to property pretty much negates the value of the magic.

I'll go with _ignore_.

[ethanfurman]

I need names. aenum already has an AutoNumberEnum (the one from the docs, no magic) so I hate to use the same name for the stdlib version with different behavior.

So I either need a different name for the stdlib version, or a different name for the aenum version.

Any ideas?

Hmmm... maybe SequentialEnum for the aenum version...

[2df14aea-354c-46a6-8f5c-342c9f465901]

Some ideas for the new stdlib class:

BasicEnum - This helps emphasize that it is a simpler version of Enum that doesn't support all of the Enum features (like assigning values). It also helps communicate that if you don't need values this is a better fit.

AutoEnum - This new version (compared with AutoNumberEnum in the docs) does more than just auto number, since it does even the value assignment. Auto helps communicate that this is automatically creating much of the class internals for you.

[ethanfurman]

I like AutoEnum.

Another auto-related thought: one of the more common Enum questions on StackOverflow is how to automatically have the value be the stringified version of the name:

class Huh(Enum):
  this
  that
  those

Huh.this.name == Huh.this.value # True

So the question then becomes: is there a way to easily support both auto-number and auto-string values?

While I don't have the auto-string feature yet in aenum, the system I am using to specify optional settings looks like this:

------

class Color(Enum, settings=AutoNumber):
  red
  ...
------

------
class Color(IntEnum, settings=AutoNumber):
  red
  ...
------

------
class Color(Enum, settings=AutoName):
  red
  ...

---

The other option, of course, is to just stick with the prebuilt method of doing things:

class Color(AutoEnum):
  ...

class Color(AutoEnum, IntEnum):
  ...

class Color(AutoNameEnum):
  ...

[warsaw]

On Jul 10, 2016, at 05:42 PM, Ethan Furman wrote:

class Color(Enum, settings=AutoNumber): [...] class Color(Enum, settings=AutoName):

I guess settings would take an AutoType enum. But that can't also be autonumbered or it would be autos all the way down. :)

[2df14aea-354c-46a6-8f5c-342c9f465901]

To me, class Color(AutoEnum) and class Color(AutoEnum, IntEnum) is a little more straightforward. It makes usage of AutoEnum similar to IntEnum, and I would expect it to be at least as popular as it.

A enum-related side question, since the plan is for this to go into the stdlib, would it also go into enum34 since that is the official back port of the stdlib Enum?

[ethanfurman]

Yeah, I think the public interface will just be the AutoEnum and AutoNameEnum style.

---

Will these features go into enum34?

Not sure. At this point I have the stdlib enum, enum34 enum, and aenum enum.

In terms of capability, aenum is the most advanced, followed by the stdlib enum, and finally enum34 (really the only difference between stdlib and enum34 is the automatic definition order).

The only advantage of enum34 over aenum is if it works in enum34 it will definitely work in the stdlib, whilst aenum has features not in the stdlib (speaking from a user point of view).

So I haven't decided, but at this moment I'm not excited about the prospect. :(

What I'll probably do is put enum34 in bug-fix only mode.

[warsaw]

On Jul 11, 2016, at 12:27 AM, Ethan Furman wrote:

Not sure. At this point I have the stdlib enum, enum34 enum, and aenum enum.

In terms of capability, aenum is the most advanced, followed by the stdlib enum, and finally enum34 (really the only difference between stdlib and enum34 is the automatic definition order).

The only advantage of enum34 over aenum is if it works in enum34 it will definitely work in the stdlib, whilst aenum has features not in the stdlib (speaking from a user point of view).

So I haven't decided, but at this moment I'm not excited about the prospect. :(

What I'll probably do is put enum34 in bug-fix only mode.

It's been useful to have a standalone version of the stdlib module, and in fact, I maintain the enum34 package in Debian. However, we only support that for Python 2 since we don't have to worry about any Python 3 versions before 3.4 (and even there, 3.5 is the default for Stretch and Ubuntu 16.04 LTS).

We do have reverse dependencies for python-enum34, but given that we *really* want people to port to Python 3, I'm not sure I really care too much any more about enum34 in Debian.

[ethanfurman]

That brings up a really good point -- this feature requires the __prepare__ method of the metaclass, so it won't work in Python 2 any way.

So, yeah, bug-fix-mostly mode for enum34. :)

[ethanfurman]

If you are constructing your own base Enum type, which would you rather do?

class BaseZeroEnum(Enum):
   "initial integer is 0"
   _start_ = 0
   ...

or

class BaseZeroEnum(Enum, start=0):
   "initial integer is 0"
   ...

? Oh, and yes if you specify a starting number you also activate the AutoNumber feature.

[warsaw]

On Jul 11, 2016, at 07:05 PM, Ethan Furman wrote:

class BaseZeroEnum(Enum, start=0): "initial integer is 0" ...

? Oh, and yes if you specify a starting number you also activate the AutoNumber feature.

I like the way this one looks.

[2df14aea-354c-46a6-8f5c-342c9f465901]

class BaseZeroEnum(Enum, start=0): "initial integer is 0" ...

I also think this looks better.

[ethanfurman]

Here's the code. I'll do the doc changes next.

[2df14aea-354c-46a6-8f5c-342c9f465901]

I like the addition of UniqueEnum. It's the default use case often.

[ethanfurman]

Okay, here's an updated patch with the doc changes.

Once the main patch is committed I'm going to reorganize the docs a bit, but that's later.

[ethanfurman]

Oh, and yes, I'll fix the whole 80-column thing. ;)

[ethanfurman]

The 80-column thing isn't fixed yet, but I redid the patch:

• removed AutoNameEnum since there was some question of casing, etc
• redid the methodology of auto-generating values: instead of adding it all to the prepared namespace or the metaclass, I now look for a special method in the Enum class, and if present use that
• removed settings (no AutoNumber, AutoName, nor Unique)
• redid AutoEnum to use the _generate_nextvalue technique
• updated docs

Let me know what you think!

[2df14aea-354c-46a6-8f5c-342c9f465901]

@Ethan

I reviewed your latest patch. I think it's a good step forward in terms of simplicity. Most of my comments were not major.

Even before this patch, I was mulling around how enum.Unique, enum.UniqueEnum, and enum.unique seemed to violate the "There should be one obvious way to do it" principle, so I like that you omitted those in the latest patch.

Looks good to me, thanks for all of your work!

[ethanfurman]

Okay, I think I'm done making changes unless any more mistakes are found.

Thanks everyone for the pushing, prodding, and feedback!

[1762cc99-3127-4a62-9baf-30c3d0f51ef7]

New changeset 7ed7d7f58fcd by Ethan Furman in branch 'default': Add AutoEnum: automatically provides next value if missing. bpo-26988. https://hg.python.org/cpython/rev/7ed7d7f58fcd

[2df14aea-354c-46a6-8f5c-342c9f465901]

I think there is a small typo in the Changelog / What's New. The Issue isn't hyper-linked:

https://docs.python.org/3.6/whatsnew/changelog.html#id2

[ethanfurman]

added missing '#'

[vstinner]

FYI Raymond Hettinger started a discussion on Twitter about this feature, and the feedback may not be as good as you expected: https://twitter.com/raymondh/status/765652816849285120

(I dislike this new magic thing, but I also never used the enum module, so I'm not sure that my opinion matters ;-))

[rhettinger]

The use of a bare identifier as a self-assigning statement is unprecedented in the world of Python. PyFlakes flags it as an error (because prior to now, a bare identifier in a class statement was almost always an error). I suspect this will cause issues with other static analysis and refactoring tools as well:

--- tmp_enum_example.py \-----

    from enum import AutoEnum

    class Color(AutoEnum):
        red
        green = 5
        blue

    print(list(Color))

--- bash session \------------

    $ py -m pyflakes tmp_enum_example.py
    tmp_enum_example.py:4: undefined name 'red'
    tmp_enum_example.py:6: undefined name 'blue'
    tmp_enum_example.py:11: undefined name 'yellow'

Also, the usual technique of commenting out blocks with triple quotes introduces unexpected breakage:

--- tmp_enum_example2.py \-----

    from enum import AutoEnum

    class Color(AutoEnum):
        red
        green = 5
        blue
        ''' XXX temporarily comment these out
        brown
        orange
        '''
        yellow

    print(list(Color))

--- bash session \------------

    $ py -m tmp_enum_example.py
    [<Color.red: 1>, <Color.green: 5>, <Color.blue: 6>, <Color.yellow: 7>]
    /Users/raymond/cpython/python.exe: Error while finding spec for
    'tmp_enum_example.py' (AttributeError: module 'tmp_enum_example'
    has no attribute '__path__')

I worry that this proposal is worse than just being non-idiomatic Python. In a way, it undermines pillars of the language and conflict everyone's mental model of how the language works. Historically, a bare variable name raised a NameError if undefined and would otherwise act as a expression who's result was discarded. However, as used here, it fiats an attribute into existence and assigns it a value. That to my eyes looks like a new language that isn't Python. This is really weird and undermines my expectations.

The need for the "ignore" parameter for the "shielded set" is a hint that the patch is working against the grain of the language and is not in harmony with Python as a coherent whole. It is a harbinger of problems to come.

Lastly, I question whether there is any real problem being solved. You already have "Color = IntEnum('Animal', 'red green blue')" that works well enough, doesn't mess with language norms, that works nicely with triple quotes for multiline entries, and that extends easily to hundreds of constants.

It seems to me that too much magic and unidiomatic weirdness are being leveled at too small of a problem. Plus we already have one way to do it.

In teaching people to make effective use of the language, a key learning point is learning the portents of trouble to come and recognizing that that not everything that can be made to work should actually be done.

Please reconsider whether you really want to open this Pandora's box. Right now, it's not too late. No doubt that you will find some people who like this (it reminds them of C), but you will also find some very experienced developers who are made queasy by the bare identifier transforming from an expression into an assigning statement. This more than an "if you don't like it, don't use it" decision, I think an important and invisible line is being crossed that we will regret.

P.S. A lesson I learned from maintaining the itertools module is that adding more variants of a single idea tends to make the overall toolkit harder to learn and impairs usability. Users suffer when given too many choices for closely related tasks. The "one way to do it" line in the Zen of Python is there for a reason.

[2df14aea-354c-46a6-8f5c-342c9f465901]

@Raymond, you raise valid concerns to be sure. Hoping we can work something out.

@Ethan, what are your thoughts?

It's not just C that has enums where you can define a unique group of names and omit the values for clarity when they are not significant:

C++: http://en.cppreference.com/w/cpp/language/enum C#: https://msdn.microsoft.com/en-us/library/sbbt4032.aspx Java: https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html Rust: https://doc.rust-lang.org/book/enums.html

In my experience this is the most common and simple use case for enums.

Raymond, what are your thoughts about the version of AutoEnum that requires that a bare tuple be used as the value. It has been in the Python docs since 3.4 and was actually the original request of this issue: https://docs.python.org/library/enum.html#autonumber

It avoids many of the concerns that you've raised while still providing a way to create Enums in the normal class declaration method users would expect with a minimum amount of boilerplate. Note that normally you want to use @enum.unique with a normal Enum, but AutoEnum also allows you to omit that boilerplate as you can't accidentally alias the values.

@enum.unique
class Color(enum.Enum):
    aquamarine = 1
    blue = 2
    fushia = 3
    # inserting a member here (perhaps because it's clearest to keep these in alphabetic order)
    # results in having to increment all following members
    ...
    green = 40
    red = 41

vs.

class Color(enum.AutoEnum):
    aquamarine = ()
    blue = ()
    fushia = ()
    # inserting a member here (perhaps because it's clearest to keep these in alphabetic order)
    # results in no refactoring
    ... (30+ more)
    green = ()
    red = ()

A big advantage of the class based Enums compared to the functional API is that you can clearly document an Enum and its members in way Sphinx can take advantage of and developers would be used to.

# Assuming tuple assignment version for this example.
class ClientOperationMode(enum.AutoEnum):
    """Modes of operations of the network client."""

    push = ()
    """The client pushes data to the server automatically."""

    pull = ()
    """The client pulls commands from the server."""

    hybrid = ()
    """The client both pushes data and pulls for commands from the server."""

Sphinx will document this AutoEnum like a normal class, pulling in the class docstring, and attribute docstrings.

I don't see an obvious way to do this in the functional API docs: https://docs.python.org/library/enum.html#functional-api

[cd293b3e-6d38-412b-8370-a46a9aaee518]

I tend to like all things magic, but the more I think about it, and the less I like it being a part of the standard library. I've had a use for this feature before, and when I did, I cooked my own 12-lines subclass of EnumMeta and _EnumDict. Raymond's points are pretty spot-on, and I also think that this shouldn't go in the stdlib. There's still time.

While this particular flavour of magic sounds too, well, magic, for Python or even myself, I think a less magical approach can be taken. Namely, something like AutoNumberedEnum which requires values to be empty tuples is being explicit about what you want and not allowing typos in the class definition. Raymond's point about (temporarily) commenting out enum members breaking the order highlights this, and while this approach doesn't solve it, it makes it obvious that there is *something* that changed.

Another approach, which doesn't exclude the above, is to make EnumMeta and _EnumDict public and documented classes (!), thus allowing people like me to create their own blueberry-flavoured magic enumerations without any impact on the people who don't use my code. Or if I don't feel like reinventing the wheel, I can just pip install the module and use that instead.

I think that the whole idea of making enums in Python work like they do in C is looking at the problem from the wrong angle. It's true that Python takes some of its design from C, but naively trying to replicate C-like behaviour with C-like syntax doesn't work all the time. How often do beginners think 'x ^ y' means 'x to the power of y', and are then surprised by the behaviour?

I think this version of Enum raises the barrier to entry for new programmers. Enum is a nice feature, and it helps new and old programmers alike write clean(er) code for various purposes. When a programmer sees this use, they won't think "oh this must call __getitem and then assign an automatic value", they'll wonder "why is this code even running?". And then it's up to the Raymonds of this world to explain that Enum uses a metaclass (which, I'm sure, is not a topic they'll want to tackle by the time these programmers reach Enum) and that the mapping overloads __getitem.

All in all, this magical approach is just too magical for Python. I understand that magic is fun to have and play with, but the standard libary isn't where you should keep your toys. I use a throwaway repo for all my magic this-is-not-a-good-idea-but-I-wanna-do-it-anyway ideas, and this is where I think such magic goes. It definitely doesn't belong in the standard library, within an arm's reach of the first curious programmer to wander there.

[vstinner]

It's not just C that has enums where you can define a unique group of names and omit the values ...

Yes, Python 3.4 too: Animal = Enum('Animal', 'ant bee cat dog')

https://docs.python.org/dev/library/enum.html#functional-api

[vstinner]

Raymond, what are your thoughts about the version of AutoEnum that requires that a bare tuple be used as the value. It has been in the Python docs since 3.4 and was actually the original request of this issue: https://docs.python.org/library/enum.html#autonumber

Well, I suggest to keep it as a recipe in the doc ;-)

[ethanfurman]

Raymond, I appreciate your review and your poll. I am open to removing AutoEnum, but would like to give it a couple more weeks of review. (I'll post on py-dev.)

The only point you made that I will firmly refute is the "unexpected breakage": you ran your test script with the -m "run module as script" flag, which is what caused the problem.

[rhettinger]

you ran your test script with the -m "run module as script" flag

Right, that was a mistest, so it looks like triple quotes do work.

I did notice that there's also an issue if one line reads, "red='hello'".

But really, the big issue is using a bare-identifier to fiat an attribute into existence. That's a door that really shouldn't be opened.

Secondarily, the doesn't seem to be any use case that can't be readily covered by the existing classes. There is no real need for the witchcraft and the departure from Python norms.

[3fa19196-1e8a-4473-b524-6181d40d40a9]

Secondarily, the doesn't seem to be any use case that can't be readily covered by the existing classes.

The use case that doesn't have a clean interface in 3.5 at the moment is the most common use case of enums: just a collection of named objects of given type; I don't care about the values because they don't really have values apart from their object identities.

When writing enums in Rust, Swift, C#, etc., the bare identifier not only saves typing--it allows the developer to explicitly indicate that the underlying value has no meaning. (It may be better to use "object()" rather than an integer on AutoEnum, but that is not very important.)

It was said that Python has this feature already:

Yes, Python 3.4 too: Animal = Enum('Animal', 'ant bee cat dog')

I will concede that this can do what I want. I hope others will concede that this is not a clean interface. The class name is duplicated and the members are regexed out of a space-delimited string. This same argument could be made to deprecate the unnecessary "class" keyword in favor of the "type" function.

I will also concede that there is some deep magic going on in AutoEnum and that magic should be avoided when it obscures. I personally think the only people who will be truly surprised will be those who already know Python at a deep enough level to know that deep magic must be required here. Everyone else will see "Enum" and a list of bare identifiers, and correctly conclude that this is your basic enum from everyone other language.

Perhaps an ideal solution would be an enum keyword:

enum PrimaryColor: red blue green

But that's not happening ever.

The next best solution is the current implementation:

class PrimaryColor(AutoEnum):
    red
    blue
    green

But because of the magic, it only barely beats out what I think is the other great solution already mentioned here:

class PrimaryColor(AutoEnum):
    red = ()
    blue = ()
    green = ()

These two solutions are isomorphic. Both save the developer from having to provide a (possibly meaningless) value. Both let docstrings be added. Both provide the ability to reorganize without renumbering. The last one trades magic for boilerplate.

I'll keep using them from the aenum package if they don't make it into 3.6, but I think this is a fundamental enough construct that it belongs in the standard library. It is hard to convince tool maintainers to fully support these until they are blessed here.

[rhettinger]

Temporarily, marking this as open so that more people can see the new comments.

For John, I'm not sure what I can say that will help you. The goal of the standard libraries modules is to provide tools the use language we have, not to invent a new language the isn't really Python or Pythonic. In the absence of an "enum" keyword, you have a number of ways to work within the language.

In this case (wanting auto numbering but not caring what the values are), you already have several ways to do it. I your issue is not where the use case is met; instead, you just don't like how their spelled (because it isn't exactly how it it looks in C).

This already works: Animal = Enum('Animal', ['ant', 'bee', 'cat', 'dog']). This is very flexible and lets you read the constants from many possible sources.

If you're attracted to multiline input, that is already possible as well:

    Animal = Enum('Animal', '''
    ant
    bee
    cat
    dog
    ''')

It is clear that you're bugged by writing Animal twice, but that is how Python normally works and it is a very minor nuisance (it only occurs once when the enum is defined). Note, you already are rewriting "Animal" every time you use the enum value (presumably many times): board_ark(Animal.ant, Animal.bee)

This whole feature request boils down to wanting a currently existing feature to be spelled a little differently, in a way that doesn't look like normal Python. Elsewhere, we resisted the temptation to alter the language look-and-feel to accommodate small spelling tweaks for various modules (i.e. we pass in normal strings to the regex module even though that sometimes requires double escaping, we pass in the class name to namedtuples even though that uses the class name twice, the xpath notation in passed into XML tools as strings even though parts of it look like regular langauge, we don't abuse the | operator to link together chains of itertools, etc)

Since the enum module already provides one way to do it, I recommend that we stick with that one way. Creating too many variants does not help users. Using the core language in odd ways also does not help users.

[warsaw]

Ethan, the suggestion has come up several times about using a dummy value such as the empty tuple to do autonumbering, thus looking more Pythonic. I'm not a huge fan of the empty tuple, and I'm still not sure whether we need this, but I wonder if it would be possible to not have a new base class, but to put the smarts in the value to which the enums were assigned. E.g. is this possible (a separate question than whether it's good :):

from enum import Enum, auto

class Color(Enum):
    red = auto
    green = auto
    blue = auto

Apologies if this has already been suggested; this tracker thread is too long to read the whole thing. :(

[ethanfurman]

Thank you, Raymond, David, Barry, John, etc. for your feedback.

While I really like AutoEnum I can see that that much magic doesn't need to exist in the stdlib.

Unfortunately, the alternatives aren't very pretty, so I'll leave the AutoNumberEnum as a recipe in the docs, and not worry about an 'auto' special value.

For those that love AutoEnum, it will be in the aenum third-party package.

[2df14aea-354c-46a6-8f5c-342c9f465901]

Ethan, thank you so much for all of your work. Looking forward to any future collaboration.

[fe5a23f9-4d47-49f8-9fb5-d6fbad5d9e38]

I don't think the problem is with (Auto)Enum at all. Problem is in a different place, and this is just a symptom.

Problem is that people make too many assumptions about class suite. Yes, type is default metaclass. But not every class should be constructed by type's rules - Python provides a very detailed and powerful mechanism for specifying a different set of rules, should you need it. This is what metaclass keyword argument should be all about. And that is the true reason why metaclass conflict is an issue: every metaclass is a DSL specification, and the same suite cannot be interpreted according to two different DSLs at the same time.

But since there's a widespread myth that users don't like to type, implementors use a trick, "class Spam(metaclass=SpamMeta): pass", and say to their users: "there, just inherit from Spam like you inherit from ordinary classes". That way, we spare them a few keystrokes, sparing them also of opportunity to learn that metaclasses _do_ change the semantics of what they see indented after the colon.

I wonder what Raymond's poll result would be if a normal way to write such code would expose the metaclass

    class Color(metaclass=AutoEnum):
        blue
        yellow

? Raymond has many valid objections, but they all apply to "ordinary" classes, instances of type. Color is not an instance of type, or at least conceptually it isn't. type (as a metaclass in Python) means a blueprint, a way to construct new instances via __call, and every instance of it has that behavior, some of which even customize it by defining __new and/or __init__. type is also special because its power of creation is transitive: its instances know how to produce their instances in almost the same way it does.

But nothing of it is mandatory, and in fact it just stands in the way when we try to define Enum (or singletons, or database models, or... whatever that is not really a Python type). We do inherit from type because it's easier, and then usurp and override its __call behavior to do something almost entirely different. (Don't you feel something is wrong when your __new method doesn't construct a new object at all?:) It's completely possible, but it's not the only way.

Maybe class is just too tainted a keyword. There was a thread on python-ideas, that we should have a new keyword, "make". "make m n(...)" would be a clearer way to write what's currently written "class n(..., metaclass=m)", with a much more prominent position for the metaclass, obviating the need for "just inherit from Spam" trick, and dissociating in people's minds the connections of arbitrary metaobjects with type. ("class" keyword could be just a shortcut for "make type" in that syntax.) But of course, a new keyword is probably too much to ask. (A mild effect can be gained by reusing keywords "class" and "from", but it looks much less nice.)

However, there is _another_ underused way metaclasses can communicate with the external world of users of their instances: keyword arguments. I wonder if Raymond's objections would be as strong if the autovivification was explicitly documented?

    class Color(Enum, style='declarative'):
        blue
        yellow

Of course we can bikeshed about exact keyword argument names and values, but my point is that there is an unused communication channel for converting typical Python user's surprise "why does this code even work" into "hey, what does that style='declarative' mean?"

[kennethreitz]

This addition to Python, specifically the bare assignment-free red syntax would be a huge mistake for the language. It looks like a syntax error to me, and completely destroys my mental model for how Python code either does work or should work.

A simple assignment to None, (), or imported static AUTO_ENUM (or similar, akin to subprocess.PIPE) would be much preferred.

[vstinner]

Would it be possible to use a auto or AUTO_ENUM constant from the enum in the Enum class? (And drop AutoEnum class.)