search

subutai-io / agent

Subutai Agent is a tool that provides CLI to control Subutai infrastructure, and is a daemon that receives and performs Management commands through secured channels
https://subutai.io
15 stars 15 forks source link

Template Version Numbers ignored. #274

Closed lbthomsen closed 6 years ago

lbthomsen commented 6 years ago

Asis at the moment it is not really possible to operate multiple templates with same name but different version numbers.

Say a container is deployed based on debian-stretch@0.0.1 and later a new container is deployed using debian-stretch@0.0.2 - that container will still be using the old template already cached on the peer.

Workaround at the moment is to embed the version numbers in the template name - so debian-stretch-0.0.1 etc.

dilshat commented 6 years ago

Shortcoming of lxc library used by agent https://github.com/lxc/go-lxc/blob/v2/lxc-binding.go#L27 namely can not clone template by name and version, only by name

lbthomsen commented 6 years ago

The template should be "debian-stretch" not "debian-stretch-0.0.5". I understand the lxc limitation being the reason here, but I would propose the following instead if you think that's possible. How about always using the latest version no matter what that is and include a check for that. In other words - if a peer have already used debian-stretch 0.0.4 for earlier deployment and a newer version - say debian-stretch 0.0.5 is available - next time someone deploy a container using debian-stretch, agent should check and realize a newer version is available - download and import that template and use that for the new deployment. Or seen from template dev point of view - if I upload a template with a higher number I can be sure that will be used rather than the old one for future deployments. IF that idea is technically possible/viable I think it's a better approach than having to embed the version number in the template name.

akarasulu commented 6 years ago

The template should be "debian-stretch" not "debian-stretch-0.0.5"

I'm 100% with you on this.

I think we can do this with a minimal impact footprint keeping changes non-invasive to the Agent. Really this is a little bit of logic to search by name into Gorjun and pick the latest version of the template. We've actually started coding the same logic into the vagrant plugin. It is needed as a short term solution.

@Dilshat can you please look into a scheme we can use to minimize the invasive changes so all the agent has to do is look up and find the latest template from the CDN and use the freshest one. Here's the logic we can use:

  1. If no version of the template is installed on the RH always use the latest
  2. If an older version of the template is installed, but not being used, remove and use the latest
  3. If an older version of the template is installed, and being used, well nothing to do just keep using it

NOTE: We use a single file name for the template i.e. debian-stretch, debian-jessie, ubuntu-xenial etc, but we can upload new versions of these to the CDN. @Dilshat @emli the CDN seems to support versions and tags for templates and maybe you can explain how perhaps we can leverage this aspect too, maybe on image uploads and on downloads: guess without using that the CDN defaults to just file upload times to order templates. Need clarification on this aspect of Kurjun @emli.

I think this is the least invasive approach and also protects our users with a chance to get the latest templates with security patches. @lbthomsen pointed out how this is a major issue and I totally agree.

akarasulu commented 6 years ago

@Dilshat @emli Lars has put in a significant and great effort to keep us fresh and updated on the base images. I want to make sure the product and our users benefit from the effort instead of this getting buried for months.

lbthomsen commented 6 years ago

But @akarasulu - in your model as you describe no. 3 will fail too.

But actually - I think the inheritance is only done at the clone point in time doesn't it? So even if a cloned container is using an old version and a newer one is installed - it won't change anything on the running container. Or am I completely wrong there?

My guess would be a read/write btrfs snapshop is created and that will survive even if the original subvolume is nuked:

root@ncpws04:/var/snap/subutai-master/common/lxc# 
root@ncpws04:/var/snap/subutai-master/common/lxc# btrfs subvolume create foo
Create subvolume './foo'
root@ncpws04:/var/snap/subutai-master/common/lxc# echo "123">foo/test.txt
root@ncpws04:/var/snap/subutai-master/common/lxc# btrfs subvolume snapshot foo bar
Create a snapshot of 'foo' in './bar'
root@ncpws04:/var/snap/subutai-master/common/lxc# btrfs subvolume delete foo
Delete subvolume (no-commit): '/var/snap/subutai-master/common/lxc/foo'
root@ncpws04:/var/snap/subutai-master/common/lxc# cat bar/test.txt 
123

So your 1 2 3 from above can as far as I see it be simplified.

  1. Check what is the latest version of the template by name
  2. If a newer one exist - download and replace existing
  3. Then do the clone etc.
lbthomsen commented 6 years ago

The above from Subutai pov:

root@ncpws06:~# subutai list -t
TEMPLATE
--------
debian
debian-stretch-0.0.4
debian-stretch-0.0.5
debian-stretch-0.0.7
master
oraclejava816
ubuntu16
root@ncpws06:~# subutai clone debian deb123
INFO[2018-01-21 19:46:57] deb123 started                               
INFO[2018-01-21 19:46:57] deb123 with ID 1041F16A289A3EBFFD9CF1A89E7892898BAD88F1 successfully cloned 
root@ncpws06:~# subutai destroy debian
WARN[2018-01-21 19:47:14] Deleting container metadata entry, bucket not found 
INFO[2018-01-21 19:47:14] debian not found. Please check if a container name is correct. 
root@ncpws06:~# subutai list -t
TEMPLATE
--------
debian-stretch-0.0.4
debian-stretch-0.0.5
debian-stretch-0.0.7
master
oraclejava816
ubuntu16
root@ncpws06:~# subutai list -c
CONTAINER
---------
Container-1-EgX-iwv-2-3
Container-2-Hpp-hwa-1-4
deb123
management

Container survives fine with the template gone missing. So - if newer template exist - nuke old - and proceed as if the template didn't exist in the first place.

This approach is obviously not ideal - much better would be proper versioning of templates but I reckon that would be hard with lxc and well - this approach is an acceptable compromise.

dilshat commented 6 years ago

There might be child templates that depend on this particular template and its contents. Also some apps may depend on a particular template and its contents. The replace mechanism can bork the existing environments. Currently the only way I see is to integrate version into path. Also we have an author of template. This must be used too, so that the samely named container of different authors can coexist on the same machine. We should place version along with author into template's config file and use it in agent to address template/container by name+author+version like $container/$author/$version e.g.: debian/lars/0.0.1 This way lxc can address the path to figure out a concrete version. But we need @lbthomsen to prove this by doing tests and preparing all necessary commands to integrate into agent. This task is a long lasting one since support for this must be implemented on all layers : Console, Hub, Gorjun/CDN if missing.

lbthomsen commented 6 years ago

First of all - we are not disagreeing on the fact that version numbers (and possibly author) should be included and handled correctly and that this issue should probably remain until that is the case.

What we should do however is to find the proper way to deal with this right now in the least disruptive manner.

Embedding version numbers in the template name works obviously, BUT aside from being butt ugly that model also introduce some issues. Blueprints will include templates - so we will be looking at a situation where blueprints are almost perpetually out of date.

The compromise that I suggested is exactly that - a compromise - not ideal and most certainly not without issues. Unless I missed something very fundamental however, I don't think the issues are as severe as you suggest.

Let me start by your suggestion that "The replace mechanism can bork the existing environments.". Hmmm, unless I misunderstood the mechanism entirely I can't see how that can happen. When a template is "cloned" it looks like the underlying lxc creates a btrfs snapshot of the template and that btrfs snapshot then becomes the root/var/opt/home of the instantiated container. That snapshot will not be borked or changed by anything that happens to the subvolume that it was snapshot from - even if that subvolume is removed entirely - nothing happens. Let me illustrate:

root@ncpws06:~# subutai import debian-stretch
INFO[2018-01-22 18:44:04] Importing debian-stretch                     
INFO[2018-01-22 18:44:05] Version: 0.1.0                               
INFO[2018-01-22 18:44:06] Template's owner signature verified          
INFO[2018-01-22 18:44:07] Unpacking template debian-stretch            
INFO[2018-01-22 18:44:08] Installing template debian-stretch           
root@ncpws06:~# btrfs subvolume list /var/snap/
core/    subutai/                                                                                                                                                                                                    
root@ncpws06:~# btrfs subvolume list /var/snap/                                                                                                                                                                      
core/    subutai/                                                                                                                                                                                                    
root@ncpws06:~# btrfs subvolume list /var/snap/subutai/                                                                                                                                                              
64/      66/      67/      common/  current/                                                                                                                                                                         
root@ncpws06:~# btrfs subvolume list /var/snap/subutai/common/lxc/                                                                                                                                                   
ID 258 gen 172642 top level 5 path ubuntu16                                                                                                                                                                          
ID 259 gen 59965 top level 258 path ubuntu16/rootfs                                                                                                                                                                  
ID 260 gen 59966 top level 258 path ubuntu16/home                                                                                                                                                                    
ID 261 gen 59967 top level 258 path ubuntu16/opt                                                                                                                                                                     
ID 262 gen 59968 top level 258 path ubuntu16/var                                                                                                                                                                     
ID 268 gen 173829 top level 5 path management                                                                                                                                                                        
ID 269 gen 173847 top level 268 path management/rootfs                                                                                                                                                               
ID 270 gen 163216 top level 268 path management/home                                                                                                                                                                 
ID 271 gen 173848 top level 268 path management/opt                                                                                                                                                                  
ID 272 gen 173848 top level 268 path management/var                                                                                                                                                                  
ID 290 gen 172642 top level 5 path master                                                                                                                                                                            
ID 291 gen 7471 top level 290 path master/rootfs                                                                                                                                                                     
ID 292 gen 7474 top level 290 path master/home                                                                                                                                                                       
ID 293 gen 7477 top level 290 path master/opt                                                                                                                                                                        
ID 294 gen 7480 top level 290 path master/var                                                                                                                                                                        
ID 336 gen 172642 top level 5 path oraclejava816                                                                                                                                                                     
ID 337 gen 12448 top level 336 path oraclejava816/rootfs                                                                                                                                                             
ID 338 gen 12451 top level 336 path oraclejava816/home                                                                                                                                                               
ID 339 gen 12442 top level 336 path oraclejava816/opt                                                                                                                                                                
ID 340 gen 12445 top level 336 path oraclejava816/var                                                                                                                                                                
ID 544 gen 149991 top level 5 path tmpdir/export743727454/rootfs                                                                                                                                                     
ID 662 gen 172595 top level 5 path Container-1-EgX-iwv-2-3                                                                                                                                                           
ID 663 gen 173826 top level 662 path Container-1-EgX-iwv-2-3/rootfs                                                                                                                                                  
ID 664 gen 166558 top level 662 path Container-1-EgX-iwv-2-3/home                                                                                                                                                    
ID 665 gen 166558 top level 662 path Container-1-EgX-iwv-2-3/opt                                                                                                                                                     
ID 666 gen 173826 top level 662 path Container-1-EgX-iwv-2-3/var                                                                                                                                                     
ID 677 gen 173365 top level 5 path debian-stretch-0.0.7                                                                                                                                                              
ID 678 gen 167628 top level 677 path debian-stretch-0.0.7/var                                                                                                                                                        
ID 679 gen 167625 top level 677 path debian-stretch-0.0.7/rootfs                                                                                                                                                     
ID 680 gen 167626 top level 677 path debian-stretch-0.0.7/home                                                                                                                                                       
ID 681 gen 167627 top level 677 path debian-stretch-0.0.7/opt                                                                                                                                                        
ID 682 gen 173829 top level 5 path Container-2-Hpp-hwa-1-4                                                                                                                                                           
ID 683 gen 173834 top level 682 path Container-2-Hpp-hwa-1-4/rootfs                                                                                                                                                  
ID 684 gen 167629 top level 682 path Container-2-Hpp-hwa-1-4/home                                                                                                                                                    
ID 685 gen 167629 top level 682 path Container-2-Hpp-hwa-1-4/opt                                                                                                                                                     
ID 686 gen 173829 top level 682 path Container-2-Hpp-hwa-1-4/var
ID 687 gen 171083 top level 5 path deb123
ID 688 gen 173541 top level 687 path deb123/rootfs
ID 689 gen 171082 top level 687 path deb123/home
ID 690 gen 171082 top level 687 path deb123/opt
ID 691 gen 173827 top level 687 path deb123/var
ID 702 gen 173858 top level 5 path debian-stretch
ID 703 gen 173850 top level 702 path debian-stretch/home
ID 704 gen 173853 top level 702 path debian-stretch/opt
ID 705 gen 173856 top level 702 path debian-stretch/var
ID 706 gen 173859 top level 702 path debian-stretch/rootfs
root@ncpws06:~# subutai clone debian-stretch deb0
INFO[2018-01-22 18:45:05] deb0 started                                 
INFO[2018-01-22 18:45:05] deb0 with ID E22258E9C78AA9937E5AFC49987709A1881DB024 successfully cloned 
root@ncpws06:~# btrfs subvolume list /var/snap/subutai/common/lxc/
ID 258 gen 172642 top level 5 path ubuntu16
ID 259 gen 59965 top level 258 path ubuntu16/rootfs
ID 260 gen 59966 top level 258 path ubuntu16/home
ID 261 gen 59967 top level 258 path ubuntu16/opt
ID 262 gen 59968 top level 258 path ubuntu16/var
ID 268 gen 173829 top level 5 path management
ID 269 gen 173862 top level 268 path management/rootfs
ID 270 gen 163216 top level 268 path management/home
ID 271 gen 173862 top level 268 path management/opt
ID 272 gen 173862 top level 268 path management/var
ID 290 gen 172642 top level 5 path master
ID 291 gen 7471 top level 290 path master/rootfs
ID 292 gen 7474 top level 290 path master/home
ID 293 gen 7477 top level 290 path master/opt
ID 294 gen 7480 top level 290 path master/var
ID 336 gen 172642 top level 5 path oraclejava816
ID 337 gen 12448 top level 336 path oraclejava816/rootfs
ID 338 gen 12451 top level 336 path oraclejava816/home
ID 339 gen 12442 top level 336 path oraclejava816/opt
ID 340 gen 12445 top level 336 path oraclejava816/var
ID 544 gen 149991 top level 5 path tmpdir/export743727454/rootfs
ID 662 gen 172595 top level 5 path Container-1-EgX-iwv-2-3
ID 663 gen 173826 top level 662 path Container-1-EgX-iwv-2-3/rootfs
ID 664 gen 166558 top level 662 path Container-1-EgX-iwv-2-3/home
ID 665 gen 166558 top level 662 path Container-1-EgX-iwv-2-3/opt
ID 666 gen 173826 top level 662 path Container-1-EgX-iwv-2-3/var
ID 677 gen 173365 top level 5 path debian-stretch-0.0.7
ID 678 gen 167628 top level 677 path debian-stretch-0.0.7/var
ID 679 gen 167625 top level 677 path debian-stretch-0.0.7/rootfs
ID 680 gen 167626 top level 677 path debian-stretch-0.0.7/home
ID 681 gen 167627 top level 677 path debian-stretch-0.0.7/opt
ID 682 gen 173829 top level 5 path Container-2-Hpp-hwa-1-4
ID 683 gen 173834 top level 682 path Container-2-Hpp-hwa-1-4/rootfs
ID 684 gen 167629 top level 682 path Container-2-Hpp-hwa-1-4/home
ID 685 gen 167629 top level 682 path Container-2-Hpp-hwa-1-4/opt
ID 686 gen 173829 top level 682 path Container-2-Hpp-hwa-1-4/var
ID 687 gen 171083 top level 5 path deb123
ID 688 gen 173541 top level 687 path deb123/rootfs
ID 689 gen 171082 top level 687 path deb123/home
ID 690 gen 171082 top level 687 path deb123/opt
ID 691 gen 173827 top level 687 path deb123/var
ID 702 gen 173864 top level 5 path debian-stretch
ID 703 gen 173864 top level 702 path debian-stretch/home
ID 704 gen 173865 top level 702 path debian-stretch/opt
ID 705 gen 173866 top level 702 path debian-stretch/var
ID 706 gen 173863 top level 702 path debian-stretch/rootfs
ID 707 gen 173866 top level 5 path deb0
ID 708 gen 173864 top level 707 path deb0/rootfs
ID 709 gen 173864 top level 707 path deb0/home
ID 710 gen 173865 top level 707 path deb0/opt
ID 711 gen 173866 top level 707 path deb0/var
root@ncpws06:~# subutai destroy debian-stretch
WARN[2018-01-22 18:45:48] Deleting container metadata entry, bucket not found 
INFO[2018-01-22 18:45:48] debian-stretch not found. Please check if a container name is correct. 
root@ncpws06:~# btrfs subvolume list /var/snap/subutai/common/lxc/
ID 258 gen 172642 top level 5 path ubuntu16
ID 259 gen 59965 top level 258 path ubuntu16/rootfs
ID 260 gen 59966 top level 258 path ubuntu16/home
ID 261 gen 59967 top level 258 path ubuntu16/opt
ID 262 gen 59968 top level 258 path ubuntu16/var
ID 268 gen 173829 top level 5 path management
ID 269 gen 173867 top level 268 path management/rootfs
ID 270 gen 163216 top level 268 path management/home
ID 271 gen 173867 top level 268 path management/opt
ID 272 gen 173867 top level 268 path management/var
ID 290 gen 172642 top level 5 path master
ID 291 gen 7471 top level 290 path master/rootfs
ID 292 gen 7474 top level 290 path master/home
ID 293 gen 7477 top level 290 path master/opt
ID 294 gen 7480 top level 290 path master/var
ID 336 gen 172642 top level 5 path oraclejava816
ID 337 gen 12448 top level 336 path oraclejava816/rootfs
ID 338 gen 12451 top level 336 path oraclejava816/home
ID 339 gen 12442 top level 336 path oraclejava816/opt
ID 340 gen 12445 top level 336 path oraclejava816/var
ID 544 gen 149991 top level 5 path tmpdir/export743727454/rootfs
ID 662 gen 172595 top level 5 path Container-1-EgX-iwv-2-3
ID 663 gen 173826 top level 662 path Container-1-EgX-iwv-2-3/rootfs
ID 664 gen 166558 top level 662 path Container-1-EgX-iwv-2-3/home
ID 665 gen 166558 top level 662 path Container-1-EgX-iwv-2-3/opt
ID 666 gen 173826 top level 662 path Container-1-EgX-iwv-2-3/var
ID 677 gen 173365 top level 5 path debian-stretch-0.0.7
ID 678 gen 167628 top level 677 path debian-stretch-0.0.7/var
ID 679 gen 167625 top level 677 path debian-stretch-0.0.7/rootfs
ID 680 gen 167626 top level 677 path debian-stretch-0.0.7/home
ID 681 gen 167627 top level 677 path debian-stretch-0.0.7/opt
ID 682 gen 173829 top level 5 path Container-2-Hpp-hwa-1-4
ID 683 gen 173834 top level 682 path Container-2-Hpp-hwa-1-4/rootfs
ID 684 gen 167629 top level 682 path Container-2-Hpp-hwa-1-4/home
ID 685 gen 167629 top level 682 path Container-2-Hpp-hwa-1-4/opt
ID 686 gen 173829 top level 682 path Container-2-Hpp-hwa-1-4/var
ID 687 gen 171083 top level 5 path deb123
ID 688 gen 173541 top level 687 path deb123/rootfs
ID 689 gen 171082 top level 687 path deb123/home
ID 690 gen 171082 top level 687 path deb123/opt
ID 691 gen 173827 top level 687 path deb123/var
ID 707 gen 173867 top level 5 path deb0
ID 708 gen 173867 top level 707 path deb0/rootfs
ID 709 gen 173867 top level 707 path deb0/home
ID 710 gen 173867 top level 707 path deb0/opt
ID 711 gen 173867 top level 707 path deb0/var

So, the way it looks to me is that templates are ONLY referenced at the exact point in time where a container is cloned from said template and future changes to that template will not alter or change the cloned container in any way shape or form.

Now, obviously a blueprint depending on a template could get screwed if big changes are introduced in a template. But that is no difference than for example Debian in itself. Future versions of Debian does not guarantee that everything in existence right now will work smoothly.

That leaves your first statement: "There might be child templates that depend on this particular template and its contents." but I would like to see an example of this. The way it appears subutai/lxc operates by using btrfs snapshot and these are in their very nature a point in time snapshot and their starting point will remain whatever it is at that exact point in time and no matter what gets changed to the template from thereon it doesn't change the snapshot. Am I wrong about this? If I am you have a valid point.

Let me finally walk through what I imagine. So right now there's an existing debian-stretch template version 0.1.0. I import that:

root@ncpws06:~# subutai import debian-stretch
INFO[2018-01-22 18:53:25] Importing debian-stretch                     
INFO[2018-01-22 18:53:26] Version: 0.1.0                               
INFO[2018-01-22 18:53:27] Template's owner signature verified          
INFO[2018-01-22 18:53:27] Downloading debian-stretch                   
 82.16 MiB / 82.16 MiB [==============================================================================================================================================================================] 100.00% 2m30s
INFO[2018-01-22 18:56:00] File integrity verified                      
INFO[2018-01-22 18:56:00] Unpacking template debian-stretch            
INFO[2018-01-22 18:56:02] Installing template debian-stretch           
root@ncpws06:~# subutai clone debian-stretch deb0
INFO[2018-01-22 18:57:17] deb0 started                                 
INFO[2018-01-22 18:57:17] deb0 with ID E4170E2D47625A60C99944691FF833AF5F4D37C4 successfully cloned 
root@ncpws06:~# subutai attach deb0
root@deb0:/# 
root@deb0:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@deb0:/# exit

Now I create a new version of debian-stretch 0.1.1.

image

On the peer I have to clean out the old template:

root@ncpws06:~# subutai destroy debian-stretch 
WARN[2018-01-22 19:05:43] Deleting container metadata entry, bucket not found 
INFO[2018-01-22 19:05:43] debian-stretch not found. Please check if a container name is correct. 
root@ncpws06:~# rm /var/snap/subutai/common/lxc/tmpdir/debian-stretch-subutai-template_0.1.0_amd64.tar.gz

This above is kind of the "extra step" that I am proposing!

root@ncpws06:~# subutai import debian-stretch
INFO[2018-01-22 19:16:20] Importing debian-stretch                                                                                                                                                                   
INFO[2018-01-22 19:16:21] Version: 0.1.1                                                                                                                                                                             
INFO[2018-01-22 19:16:22] Template's owner signature verified                                                                                                                                                        
INFO[2018-01-22 19:16:22] Downloading debian-stretch                                                                                                                                                                 
 82.16 MiB / 82.16 MiB [==============================================================================================================================================================================] 100.00% 5m18s
INFO[2018-01-22 19:21:44] File integrity verified                                                                                                                                                                    
INFO[2018-01-22 19:21:44] Unpacking template debian-stretch                                                                                                                                                          
INFO[2018-01-22 19:21:45] Installing template debian-stretch                                                                                                                                                         
root@ncpws06:~# subutai clone debian-stretch deb1                                                                                                                                                                    
INFO[2018-01-22 19:22:25] deb1 started                                                                                                                                                                               
INFO[2018-01-22 19:22:25] deb1 with ID 4816AA7BCF5C3DB0FE4D6A11D845B7F6554797B3 successfully cloned                                                                                                                  
root@ncpws06:~# subutai attach deb1                                                                                                                                                                                  
root@deb1:/# ls                                                                                                                                                                                                      
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var                                                                                                               
root@deb1:/# exit

So now we have two containers, both based on debian-stretch:

root@ncpws06:~# subutai list -c -p
CONTAINER               PARENT                                                                                                                                                                                       
---------               ------                                                                                                                                                                                       
Container-1-EgX-iwv-2-3 debian-stretch-0.0.4                                                                                                                                                                         
Container-2-Hpp-hwa-1-4 debian-stretch-0.0.7                                                                                                                                                                         
deb0                    debian-stretch                                                                                                                                                                               
deb1                    debian-stretch                                                                                                                                                                               
deb123                  debian                                                                                                                                                                                       
management              openjre16

But having been snapshot'ed at different points in time - they are not based on the same version.

ID 258 gen 172642 top level 5 path ubuntu16                                                                                                                                                                          
ID 259 gen 59965 top level 258 path ubuntu16/rootfs                                                                                                                                                                  
ID 260 gen 59966 top level 258 path ubuntu16/home                                                                                                                                                                    
ID 261 gen 59967 top level 258 path ubuntu16/opt                                                                                                                                                                     
ID 262 gen 59968 top level 258 path ubuntu16/var                                                                                                                                                                     
ID 268 gen 173829 top level 5 path management                                                                                                                                                                        
ID 269 gen 173975 top level 268 path management/rootfs
ID 270 gen 163216 top level 268 path management/home
ID 271 gen 173975 top level 268 path management/opt
ID 272 gen 173975 top level 268 path management/var
ID 290 gen 172642 top level 5 path master
ID 291 gen 7471 top level 290 path master/rootfs
ID 292 gen 7474 top level 290 path master/home
ID 293 gen 7477 top level 290 path master/opt
ID 294 gen 7480 top level 290 path master/var
ID 336 gen 172642 top level 5 path oraclejava816
ID 337 gen 12448 top level 336 path oraclejava816/rootfs
ID 338 gen 12451 top level 336 path oraclejava816/home
ID 339 gen 12442 top level 336 path oraclejava816/opt
ID 340 gen 12445 top level 336 path oraclejava816/var
ID 544 gen 149991 top level 5 path tmpdir/export743727454/rootfs
ID 662 gen 172595 top level 5 path Container-1-EgX-iwv-2-3
ID 663 gen 173826 top level 662 path Container-1-EgX-iwv-2-3/rootfs
ID 664 gen 166558 top level 662 path Container-1-EgX-iwv-2-3/home
ID 665 gen 166558 top level 662 path Container-1-EgX-iwv-2-3/opt
ID 666 gen 173826 top level 662 path Container-1-EgX-iwv-2-3/var
ID 677 gen 173365 top level 5 path debian-stretch-0.0.7
ID 678 gen 167628 top level 677 path debian-stretch-0.0.7/var
ID 679 gen 167625 top level 677 path debian-stretch-0.0.7/rootfs
ID 680 gen 167626 top level 677 path debian-stretch-0.0.7/home
ID 681 gen 167627 top level 677 path debian-stretch-0.0.7/opt
ID 682 gen 173829 top level 5 path Container-2-Hpp-hwa-1-4
ID 683 gen 173945 top level 682 path Container-2-Hpp-hwa-1-4/rootfs
ID 684 gen 167629 top level 682 path Container-2-Hpp-hwa-1-4/home
ID 685 gen 167629 top level 682 path Container-2-Hpp-hwa-1-4/opt
ID 686 gen 173946 top level 682 path Container-2-Hpp-hwa-1-4/var
ID 687 gen 171083 top level 5 path deb123
ID 688 gen 173541 top level 687 path deb123/rootfs
ID 689 gen 171082 top level 687 path deb123/home
ID 690 gen 171082 top level 687 path deb123/opt
ID 691 gen 173827 top level 687 path deb123/var
ID 717 gen 173907 top level 5 path deb0
ID 718 gen 173944 top level 717 path deb0/rootfs
ID 719 gen 173907 top level 717 path deb0/home
ID 720 gen 173907 top level 717 path deb0/opt
ID 721 gen 173946 top level 717 path deb0/var
ID 722 gen 173970 top level 5 path debian-stretch
ID 723 gen 173970 top level 722 path debian-stretch/home
ID 724 gen 173971 top level 722 path debian-stretch/opt
ID 725 gen 173972 top level 722 path debian-stretch/var
ID 726 gen 173969 top level 722 path debian-stretch/rootfs
ID 727 gen 173973 top level 5 path deb1
ID 728 gen 173974 top level 727 path deb1/rootfs
ID 729 gen 173973 top level 727 path deb1/home
ID 730 gen 173973 top level 727 path deb1/opt
ID 731 gen 173974 top level 727 path deb1/var

What have I misunderstood?

dilshat commented 6 years ago

Hmm, so we can replace a parent template without affecting neither child templates nor containers. We can go this minimal way then without big changes and just replace the template if a different version is specified than the existing one. Otherwise use what we already have. Is this what you suggest?

These are the cases where this can fail us: 1) Hub has a grow environment operation and blueprints that can rely on a specific version to be present and this operation can fail due to changed template contents. 2) At least one operation will fail if there are several concurrent import operations specifying different versions of the same template (which one to replace ?)

lbthomsen commented 6 years ago

Well - I am asking you dilshat :) As far as I can see it makes no difference at all. I am fishing here - you got way more experience with this than I do, so if you can think of a scenario where it would cause problems I would really like to know.

Regarding 1: Let me try this! Regarding 2: If we ignore the version (as we pretty much do right now anyway) this becomes simple - the "latest" always!

lbthomsen commented 6 years ago

Ah no need to test - yes - number 1 is a VERY strong argument for doing it right with proper version check! But even worse case you would simply end up having two containers in the same environment based on different base templates. The moment a container is started it is on it's own anyway - so while not ideal I still think it's better than risking deployment with an old and potentially insecure container.

dilshat commented 6 years ago

Fixed in 6.4.2