Service instance delete event does not get recorded when space is deleted

[jenspinney]

Issue

When a space or org where a service instance lives is deleted, an event is not logged for the service instance delete.

Steps to Reproduce

1. Assuming you have a service called some-service with a plan called some-plan: cf create-space delete-me && cf target -s delete-me && cf create-service some-service some-plan si1 && cf create-service some-service some-plan si2
2. Get the last timestamp of the last service instance delete event by running export SAVED_TIMESTAMP=$(cf curl '/v2/events?q=type:audit.service_instance.delete&order-by=timestamp&order-direction=desc' | jq .resources[0].entity.timestamp)
3. Verify that cf curl "/v2/events?q=type:audit.service_instance.delete&q=timestamp>$SAVED_TIMESTAMP" returns nothing.
4. Delete si1 directly: cf curl /v2/service_instances/$(cf service si --guid)?accepts_incomplete=true -X DELETE.
5. Run cf curl "/v2/events?q=type:audit.service_instance.delete&q=timestamp>$SAVED_TIMESTAMP" and see that a new event was created.
6. Delete the space that si2 is in: cf delete-space delete-me -f.
7. Run cf curl "/v2/events?q=type:audit.service_instance.delete&q=timestamp>$SAVED_TIMESTAMP" and observe that no new event has been created for the deletion of si2.

Expected result

There is a event of type audit.service_instance.delete corresponding to the deletion of service instance that was in a deleted space.

Current result

There is no event for a service instance deleting via a cascade delete.

Possible Fix

Log an event here: https://github.com/cloudfoundry/cloud_controller_ng/blob/master/app/actions/services/service_instance_delete.rb#L54

Note

There are several places in the code that we do log an event for service instance delete, which may not be obvious without some digging. (For example, if the service broker responds to the deprovision request with data instead of the empty JSON body ({}) prescribed in the OSBAPI spec, you can get into a situation where the deletion event is logged, but only if the last_operation state eventually changes to succeeded. From our audit of the possible codepaths, the typical scenario where a cascade delete succeeds and the service broker responds as it should according to the spec, the event isn't logged.

After digging around the code, we came up with the following flow-chart describing when service instance deletes are recorded or not. Each decision point path represents a different codepath that's executed.

Thanks, Jen and @ablease

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/152542410

The labels on this github issue will be updated when the story is started.

[elenasharma]

Thanks for filing this issue, @jenspinney and @ablease .

@zrob This seems legitimate. Tagging you so you can prioritize at will.