Avoid altering parent groups

[markfairbanks]

Closes #439

Note: This is just a slight continuation of #443 that takes care of more cases where the error occurred.