pencil2d / pencil

Pencil2D is an easy, intuitive tool to make 2D hand-drawn animations. Pencil2D is open source and cross-platform.
http://pencil2d.org
GNU General Public License v2.0
1.45k stars 272 forks source link

Assertion failure when redoing a selection transformation #1839

Open scribblemaniac opened 5 months ago

scribblemaniac commented 5 months ago

Issue Summary

If a selection is transformed and the undo/redo mechanism is used in a specific order, then this assert will be triggered, and crash on debug builds.

Steps to reproduce

  1. Create a selection with the select tool.
  2. Switch to the move tool.
  3. Transform the selection.
  4. Undo the selection transformation.
  5. Switch to the pen tool.
  6. Redo the selection transformation.
  7. Attempt to draw through the selection.

Actual Results

The program assert fails on debug builds when following the steps outlined in the section above. On release builds, the selection does not get applied before drawing and the drawing tool will behave as if the selection has not been transformed. This will result in some visual anomalies as the brush data conflicts with the selection data. After each stroke is finished, the program behaves as if the stroke had been drawn before the selection was transformed, and the selection remains active with a transformation.

Expected Results

It is expected that the assert does not fail obviously. When redoing a selection, it should probably apply the transform if the move tool is not active, or perhaps just if there is no active selection transform.

System Information

+ Operating System: 22.04 + RAM Size: 16GB + Graphics Tablet: N/A
### Developer Notes This has many similarities to #1838, likely the selection is in a similar state but through different means. There may be one fix for both issues, or separate fixes depending on where the fix is applied to.