search

alvinwan / neural-backed-decision-trees

Making decision trees competitive with neural networks on CIFAR10, CIFAR100, TinyImagenet200, Imagenet
https://nbdt.aaalv.in
MIT License
607 stars 130 forks source link

[Question][Bug?] Why are you using FC layer outputs instead of Neural Backbone outputs during inference? #5

Closed sukeesh closed 4 years ago

sukeesh commented 4 years ago

According to the paper at 3.1 Inference with Embedded Decision Rules

First, our NBDT approach featurizes each sample using the neural network backbone; the backbone consists of all neural network layers before the final fully-connected layer.

So, It means the sample is run on the NN architecture excluding the final fully-connected layer(?)

If so, Why are you including the final fully-connected layer in here before passing x to self.rules.forward_with_decisions? https://github.com/alvinwan/neural-backed-decision-trees/blob/7ef5fe5034281aeb0d10786495d5556e99b98af4/nbdt/model.py#L323-L324

Can you please explain?

sukeesh commented 4 years ago

@alvinwan @daniel-ho @spetryk Can you please explain this confusing part?

alvinwan commented 4 years ago

Hello @sukeesh, good question. This is because average of inner products is equivalent to the inner product of averages.

For a longer explanation, see supplementary material section 2: https://arxiv.org/pdf/2004.00221.pdf.