Instances keep running for deleted version

[GoogleCodeExporter]

What steps will reproduce the problem?

I deployed my Managed VMs version 794 with:

gcloud preview app deploy app.yaml --docker-build=remote --version 794 --force 
--promote 

Then I deleted version 792 with:

gcloud preview app modules delete default --version 792

What is the expected output? What do you see instead?

Both operations were successful and the App Engine Console reports 794 as the 
only current version.  However, the v792 instances are still running over 4 
hours after the version was deleted.  Manually deleting them is ineffective, as 
expected.

From https://code.google.com/p/google-cloud-sdk/issues/detail?id=284 I 
understand that the failsafe instance deletion is performed every 2 hours so it 
looks like something has gone more wrong than usual.  In the last month, I've 
*never* seen instances be deleted at the same time as the version, so the 
failsafe process is more like the normal process at this point...

Could you please delete my v792 instances, and make the version switch process 
more robust overall?  Thanks.

What is the output of 'gcloud info'?

Google Cloud SDK [0.9.87]

Platform: [Linux, x86_64]
Python Version: [2.7.6 (default, Mar 22 2014, 22:59:56)  [GCC 4.8.2]]
Python Location: [/root/shippable_ve/bin/python2]
Site Packages: [Enabled]

Installation Root: [/root/google-cloud-sdk]
Installed Components:
  core: [2015.11.18]
  core-nix: [2015.11.16]
  app: []
  gcloud: []
  gsutil-nix: [4.15]
  gsutil: [4.15]
  bq: [2.0.18]
  bq-nix: [2.0.18]
System PATH: 
[/root/bin:/root/shippable_ve/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/
bin:/sbin:/bin:/root/usr/local/bin:/root/google-cloud-sdk/bin]
Cloud SDK on PATH: [True]

Installation Properties: [/root/google-cloud-sdk/properties]
User Config Directory: [/root/.config/gcloud]
User Properties: [/root/.config/gcloud/properties]
Current Workspace: [None]
Workspace Config Directory: [None]
Workspace Properties: [None]

Account: [piotr@reviewable.io]
Project: [reviewable-prod]

Current Properties:
  [core]
    project: [reviewable-prod]
    account: [piotr@reviewable.io]
    disable_usage_reporting: [False]
  [app]
    use_appengine_api: [True]

Logs Directory: [/root/.config/gcloud/logs]
Last Log File: [/root/.config/gcloud/logs/2015.11.20/01.05.50.993856.log]

Original issue reported on code.google.com by pi...@ideanest.com on 20 Nov 2015 at 5:13

[GoogleCodeExporter]

Looks like the v792 instances got deleted around 11am PST, roughly 18 hours 
after the version was deleted.  (This is a best guess based on CPU usage data, 
since there doesn't appear to be a way to chart the number of instances over 
time in the console.)

Original comment by pi...@ideanest.com on 21 Nov 2015 at 12:12

[GoogleCodeExporter]

And now after deploying v796, App Engine somehow managed to spin up *four* 
instances of v794 (where under normal load there would be just two).  This is 
getting slightly ridiculous (and costing me money).

Original comment by pi...@ideanest.com on 22 Nov 2015 at 8:49

Attachments:

• v794.PNG

[GoogleCodeExporter]

Thanks for reporting this. I've alerted the appropriate folks internally.

Expect an update on this bug soon.

Original comment by z...@google.com on 24 Nov 2015 at 9:10

• Changed state: Accepted

[GoogleCodeExporter]

I've heard from an internal team that this issue should be fixed as of 
yesterday. Do you still see it occurring?

Original comment by z...@google.com on 24 Nov 2015 at 9:57

• Changed state: NeedsMoreInfo

[GoogleCodeExporter]

The old version's instances got deleted within a minute after my latest 
deployment so looks like this is fixed.  Thanks!

Original comment by pi...@ideanest.com on 1 Dec 2015 at 1:26

[GoogleCodeExporter]

Thanks for reporting! Glad we could fix it.

Original comment by z...@google.com on 1 Dec 2015 at 2:17

• Changed state: Verified