We tried using the preview feature to test what a second button would look like - we didn't click update, just preview. This made the button live in a preview window but also live on the site. Because of this, removing the button didn't work until we first pressed update, then removed the button and pressed update again.
Here's a link to the Screencast (audio is slightly off). Also - we noticed that this issue isn't just the additional buttons, but most custom content fields
We tried using the preview feature to test what a second button would look like - we didn't click update, just preview. This made the button live in a preview window but also live on the site. Because of this, removing the button didn't work until we first pressed update, then removed the button and pressed update again.