deepcopy in py3k evaluates interpolated strings

[WeatherGod]

Consider the following code example:

from __future__ import print_function
from copy import deepcopy
from configobj import ConfigObj
conf = ConfigObj()
conf['foo'] = 'bar'
conf['baz'] = '%(foo)s.txt'
conf_new = deepcopy(conf)
conf_new['foo'] = 'abc'
print(conf_new['bar'])

In python 2.7, the output is "abc.txt". In python 3.7, the output is 'bar.txt', which is the value of the 'bar' key in the original config object.

[WeatherGod]

Continuing the investigate this. I compared the execution paths between py2.7 and py3.7. The point where it diverges is in def __reduce__(self): when it calls dict(self). From that, I was able to get a slightly simplified example that demonstrates the problem:

from __future__ import print_function
from configobj import ConfigObj
conf = ConfigObj()
conf['foo'] = 'bar'
conf['baz'] = '%(foo)s.txt'
print(dict(conf))

In py2.7, this prints out {'foo': 'bar', 'baz': '%(foo)s.txt'}

In py3.7, this prints out: {'foo': 'bar', 'baz': 'bar.txt'}

[WeatherGod]

having a hard time debugging past this point. It seems like the behavior of dict() has changed in subtle ways. I am going to see if earlier py3k releases had this problem.

[WeatherGod]

Ah ha! this all works as expected in python 3.5! So, a change introduced in py3.6 broke things. Perhaps the reworked dictionary?

[WeatherGod]

Apparently matplotlib ran into this problem last year with their rcparams implementation and I completely missed this: https://github.com/matplotlib/matplotlib/pull/12604

So, this would have been broken on a minor release from cpython. Looking into seeing how configobj can be adapted to fix this problem.

[WeatherGod]

Alright, I figured out a solution to the following cases: copy.copy(conf), copy.deepcopy(conf), and conf.copy(), but I haven't figured out dict(conf). Basically, dict() will now use conf.items(), which has always triggered string interpolation. Previously, it used dict.items(), which didn't.

[WeatherGod]

while writing up unit tests, I discovered an interesting inconsistency in the existing code in pre-py3.6 versions.

conf = ConfigObj(['[a]', 'foo = bar', 'baz = $foo'], interpolation=True)
assert dict(conf) == {'a': {'foo': 'bar', 'baz': 'bar'}}
assert dict(conf['a']) == {'foo': 'bar', 'baz': '$foo'}

So, it seems that calling dict() on a ConfigObj is different than calling dict() on a Section? Not exactly sure why this is happening.

[robdennis]

I appreciate the investigation here and I'm inclined to get this into 5.1.0