Copying a Resource object does not work with dict().copy and copy.deepcopy()

[bkranendonk]

Issue: I tried to copy a Resource, so that I dont have to specify the same object over and over again

What I ran: self.base_instance = osr.openstack_compute_instance_v2( _name="Instance", name="Instance", count="1", availability_zone=self.provider_az[0], image_name="CentOS 7 (LTS)", flavor_id="2004", key_pair="SchoolDocker", network={"name": "noice"}, user_data="", security_groups=[""] )

instance_2 = copy.deepcopy(self.base_instance)

Traceback: Traceback (most recent call last): File "./main.py", line 78, in <module> config.add_docker() File "./main.py", line 47, in add_docker instance_2 = copy.deepcopy(self.base_instance) File "/usr/lib/python3.6/copy.py", line 96, in copy rv = reductor(4) TypeError: 'Attribute' object is not callable

What I also tried: instance_2 = self.base_instance.copy()

Does also not work

[mjuenema]

Hi @bbramms, sorry for the long wait. I had a closer look at the problem you described. Please check https://github.com/mjuenema/python-terrascript/blob/issue/issue99b/docs/issues/099.ipynb for details.

The Python-Terrascript script code contains some hacks that make its classes "un-pickable". The copy.copy() function actually uses pickle to copy objects (see the very end of https://docs.python.org/3.5/library/copy.html). I fear that...

a) I am going to make the Python-Terrascript classes raise PicklingError, and b) Python-Terrascript classes cannot be copied (with some small exceptions).

Sorry!

[mjuenema]

@bbramms Good news. I think I fixed the problem. Please try yourself using the develop branch and provide feedback so I can close the issue.

[bkranendonk]

Thanks for your reply!

I tried out this piece of code with the cloned develop branch:

config = Terrascript()
config += openstack()
def gen_instances(c=10):
    base_instance = openstack_compute_instance_v2('Instance', name="Base Instance")
    result = []
    for i in range(c):
        ins = base_instance.copy()
        ins['name'] = "Instance {}".format(str(c))
        ins['flavor_id'] = str(c)
        result.append(ins)
    return result

print(gen_instances())

Unfortunately that snippet still returns 10 times the last created resource:

[{'name': 'Instance 10', 'flavor_id': '10'}, {'name': 'Instance 10', 'flavor_id': '10'}, {'name': 'Instance 10', 'flavor_id': '10'}, {'name': 'Instance 10', 'flavor_id': '10'}, {'name': 'Instance 10', 'flavor_id': '10'}, {'name': 'Instance 10', 'flavor_id': '10'}, {'name': 'Instance 10', 'flavor_id': '10'}, {'name': 'Instance 10', 'flavor_id': '10'}, {'name': 'Instance 10', 'flavor_id': '10'}, {'name': 'Instance 10', 'flavor_id': '10'}]

Are there any best practices for copying resources within Terrascript? I really want to know because it's a wonderful module. Thanks! :)

[mjuenema]

Hi, there are no "official" best practice for copying a resource, or any other Terrascript object for that matter. It's not something I had particularly envisaged when I started work on the project but it is of course a perfectly valid approach.

What you encountered is quite simply a bug. The difficulty I have in fixing this bug stems from the fact that Terrascript kind of "abuses" Python dictionaries for its own purposes. That works well when it works but fails miserably when it doesn't.

Can I suggest an alternative to your implementation? You could create a function that returns instances of openstack_compute_instance_v2 instead of using copy.copy().

I slept over below and it and decided that it is rubbish. So please ignore. Here's some example code which I haven't actually tested but I hope you get the idea.

def create_openstack_compute_instance_v2(*args, **kwargs):
    return openstack_compute_instance_v2(*args, **kwargs)

That's something I definitely had in mind when I started Terrascript because it makes good use of the Python language.

If you wanted, you could expand that to be a generic function for creating "anything".

def create_something(cls, *args, **kwargs):
    return cls(*args, **kwargs)

[mjuenema]

On a side note, I had a look at your PyIOS project. I am interested because that's what i am really dealing with on a day-to-day basis as a Systems and Network Engineer. And I also can appreciate how much effort is involved in maintaining even a small project on Github.

As you probably have noticed, I have very limited time for working on Terrascript as I don't deal with "the cloud" at all these days.

[jbscare]

This may be a dumb question, but could you add a terrascript.copy function in the code itself? (Or terrascript.deepcopy, or whatever.)

[bkranendonk]

On a side note, I had a look at your PyIOS project. I am interested because that's what i am really dealing with on a day-to-day basis as a Systems and Network Engineer. And I also can appreciate how much effort is involved in maintaining even a small project on Github.

As you probably have noticed, I have very limited time for working on Terrascript as I don't deal with "the cloud" at all these days.

Developed that for a Software Defined Networking grade actually, didnt touch it since then. Feel free to create some issues and I will look into them when i have some time.

[mjuenema]

@bbramms I rewrote your code a bit.

import terrascript
import terrascript.resource
import terrascript.provider

config = terrascript.Terrascript()
config += terrascript.provider.openstack(...)

def gen_instances(c=10):
    result = []
    for i in range(c):
        name = "Instance {}".format(str(c))    # Should that be str(i)
        flavour_id = strc(c)
        ins = terrascript.resource.openstack_compute_instance_v2(name, name=name, flavour_id=flavour_id)
        result.append(ins)
    return result

print(gen_instances())

[mjuenema]

@jbscare I'd really want to avoid adding .copy() or .deepcopy() methods. copy.copy() should work although I think there are better alternatives.

[bkranendonk]

Thanks for sorting this out!