Open tyler-hayes opened 1 year ago
Dear Tyler,
Thank you for the interest in our work!
Your Stage 2 understanding is correct. For Stage 1, however, it's wrong. There, we do not use RPN boxes, but only GT boxes and their class predictions. We then run Hunagian Matching on top of GT boxes with generated class assignments.
While your suggested approach is also sound and would be more resemblant of real-world evaluation, in my opinion, we tried to follow existing works [1], including mimicking NCD setup, e.g. [2], as close as possible.
Thanks for the clarification!
One additional question: for use of the Hungarian Algorithm (HA) with a different number of clusters than classes, does the RNCDL method use dummy variables to create a square prediction matrix for HA evaluation?
Yes, we create additional vertices, when necessary, ultimately just keeping the matched pairs that contain GT classes.
On Tue, Apr 4, 2023 at 7:48 AM Tyler Hayes @.***> wrote:
Thanks for the clarification!
One additional question: for use of the Hungarian Algorithm (HA) with a different number of clusters than classes, does the RNCDL method use dummy variables to create a square prediction matrix for HA evaluation?
— Reply to this email directly, view it on GitHub https://github.com/vlfom/RNCDL/issues/8#issuecomment-1496105976, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGQXRBXO2Z6NZBPG5NGP23W7QYCFANCNFSM6AAAAAAUVKJVXU . You are receiving this because you commented.Message ID: @.***>
Hello again, I had another clarification question. Regarding the matching step of GT boxes to novel class IDs, is this step run on the GT test
boxes or the GT train
boxes?
Thank you in advance!
Hello, thank you for your really interesting work!
We wanted to clarify the evaluation procedure from the paper. Could you indicate if the following interpretation is correct or incorrect? If it is incorrect, any clarifications would be greatly appreciated! :)
For an IoU threshold: