Enrichment concurrency issue with control plane

[NDStrahilevitz]

Since we moved cgroup mkdir and rmdir processing into the control plane, there is an indeterminacy if a cgroup exists by the time its initial enrichment is requested. The reason is, the control plane signal may arrive later then the cgroup_mkdir event. Since processing is only done from the control plane, enrichment may be enqueued for a mkdir event before the cgroups struct knows of the cgroup.

Update: The issue is localized to the container_create event. What happens is that the mkdir event reaches the enrichment pipeline stage and triggers an enrichment request before the control plane processes the signal, thus the enrichment request reports a non existant cgroup. This propagates down to the container_create derived event, which it seems is also derived before enrichment finishes.

[NDStrahilevitz]

Possible solution: add logic using sync.Cond to in the enrich procedure if the cgroup id doesn't exist:

1. Check if cgroup is listed for removal - if yes return
2. If not - probably just doesn't exist yet (waiting for control plane signal)
3. Add a sync.Cond wait (possibly in a new map)
4. Proceed to enrichment when cond is fulfilled