Rethinking Pyomo component string representations

[whart222]

Yesterday, I discussed a preliminary extension of VarList to include per-variable labels. This would allow me to label variables as I add them to the list.

This conversation raised some concerns that the users would not be able to associate labels with the location of the variable in a model. I noted that this extension really isn't logically different from Pyomo capabilities that already exist, where we can assign variables on a model with an arbitrary name. For example:

from pyomo.environ import *

m = ConcreteModel()
setattr(m, 'some wierd.name', Var())
setattr(m, ' ', Var())

m.pprint()

Which produces output like:

2 Var Declarations
      : Size=1, Index=None
        Key  : Lower : Value : Upper : Fixed : Stale : Domain
        None :  None :  None :  None : False :  True :  Reals
    some wierd.name : Size=1, Index=None
        Key  : Lower : Value : Upper : Fixed : Stale : Domain
        None :  None :  None :  None : False :  True :  Reals

2 Declarations: some wierd.name  

Note that the setattr() method is being used in a manner that makes it impossible to identify the original variable component. In fact, I didn't even know that I could use a space as an attribute name until I tried this. But more generally, we can add arbitrary symbols in attribute names.

I think this is generally bad, but I'd like to assess everyone's thoughts on this. Should we change setattr() semantics? Should we require that string representations provide exact correspondence to the location of components in models? If not, then what names are reasonable and appropriate? Or are any allowable?

[jsiirola]

I agree that "unusual" names are in general a bad thing, and I would be OK with performing some "sanity checks" on incoming names. Personally, allowing "." is crazy useful in transformations. I can tolerate internal spaces, too -- but leading and trailing spaces seem very bad.

That said, the examples you mention all can be retrieved by their label (name):

>>> from pyomo.environ import *
>>> m = ConcreteModel()
>>> v = Var()
>>> m.add_component('   ', v)
>>> m.pprint()
1 Var Declarations
        : Size=1, Index=None
        Key  : Lower : Value : Upper : Fixed : Stale : Domain
        None :  None :  None :  None : False :  True :  Reals

1 Declarations:    
>>> m.component('   ')
<pyomo.core.base.var.SimpleVar object at 0x7f80171d98c0>
>>> m.component(v.name)
<pyomo.core.base.var.SimpleVar object at 0x7f80171d98c0>
>>> hex(id(v))
'0x7f80171d98c0'
>>>

Instead of creating a new ComponentList-like component that allows mangling the names, @blnicho and I suggest using a Component indexed by Any and then using the name you want as the key:

>>> m = ConcreteModel()
>>> v = Var()
>>> m.add_component('   ', v)
>>> m.yVar = Var([1,2])
>>> m.b = Block()
>>> m.b.x = Var(Any, dense=False)
>>> for _ in m.component_data_objects(Var, descend_into=False):
...     m.b.x[_.name] = 0
... 
>>> m.pprint()
1 Set Declarations
    yVar_index : Dim=0, Dimen=1, Size=2, Domain=None, Ordered=False, Bounds=(1, 2)
        [1, 2]

2 Var Declarations
        : Size=1, Index=None
        Key  : Lower : Value : Upper : Fixed : Stale : Domain
        None :  None :  None :  None : False :  True :  Reals
    yVar : Size=2, Index=yVar_index
        Key : Lower : Value : Upper : Fixed : Stale : Domain
          1 :  None :  None :  None : False :  True :  Reals
          2 :  None :  None :  None : False :  True :  Reals

1 Block Declarations
    b : Size=1, Index=None, Active=True
        1 Var Declarations
            x : Size=3, Index=Any
                Key     : Lower : Value : Upper : Fixed : Stale : Domain
                        :  None :     0 :  None : False : False :  Reals
                yVar[1] :  None :     0 :  None : False : False :  Reals
                yVar[2] :  None :     0 :  None : False : False :  Reals

        1 Declarations: x

4 Declarations:     yVar_index yVar b

[whart222]

@jsiirola Can you provide context for why using "."s is so useful? I don't see why you couldn't use the same sort of trick within transformations: component names are used as indices.

[jsiirola]

I am pretty sure you could use the same "Any index" trick. The reason is almost completely historical (i.e., dates from before Any was a valid index), and mostly arose when I was maintaining some (but not all) of the model structure.

[whart222]

So then if you could use the same "Any index" trick ... then we should be able to deprecate the use of string representations with "."'s. That seems counter to your recommendation above.

[jsiirola]

Apart from the effort to rework some transformations that currently allow/encourage the use of "." in component names, I have no reason to veto the removal of "." from the list of allowable characters in component names.

More generally, I would not oppose forbidding the following characters: .,'", as well as leading/trailing white space.

[carldlaird]

If we are going this far, why not only allow names that are python addressable as attributes? This would make the rules clear.

[whart222]

@carldlaird I agree. I think that Python limits attributes to strings that start with an underscore or letter, and which only contain letters, numbers or underscores.

[qtothec]

I think I would be happier if variable names were restricted in this way too.

[whart222]

I drafted a branch of Pyomo to restrict attribute names a while back, and we haven't done anything with it. I've moved the branch here:

https://github.com/whart222/pyomo/tree/component_setattr