paper about this code

[Arktis2022]

Hi! Is there a corresponding paper published for this code?

[MarijaStojchevska]

Hi!

I am glad you are interested in Injective Flows!

Unfortunately, there isn't official paperwork on anomaly detection with injective flows model implementation. However, you can read the official documentation for the TRUMPETS: Injective Flows for Inference and Inverse Problemshttps://proceedings.mlr.press/v161/kothari21a/kothari21a.pdf by Konik Kothari et.al. which was my main inspiration, and my model is mostly based on their ideas.

Similarly, as they proposed, I am using an injective map or alternating injective and bijective mappings (for space reduction) followed by a bijective map or multiple bijective mappings (standard normalizing flow) to map input images into latent space (Gaussian distribution). To make things more convenient, I utilized the idea of separate model training as proposed in the above-mentioned paper, where the injective map and the second bijective map are trained separately.

Once the injective flow model (injective map - bijective map) was implemented, I trained the model on non-anomalous images in order to learn their distribution. In this way, I have direct access to the density of the learned distribution both in the output space of the injective map and the output space of the bijective map. By implementing the model in this way, we have two output spaces and two learned distributions, isn't that cool? As you may know, one of the advantages of normalizing flows over other generative models is that they give direct access to likelihoods with respect to the learned distribution. So, my goal was to test the model on both anomalous and non-anomalous test images and for each of them to compute the two likelihoods given the learned distributions in the two output spaces. If the likelihood is far away from the learned distribution we can classify the sample as anomalous (again this is done in both spaces).

In this way, I can see if the first space has better discriminatory performance that the other one. Intuitively, something that looks like an anomaly in one space would probably look like an anomaly in the other space too, which was proven through our work. In the end, we can conclude that the injective flow cannot contribute to better discriminatory performance in OOD problems, however, it is so powerful since it reduces the input data dimensionality and thus fastens the entire training/testing process.

So if you don't suffer from memory/time execution problems and work with data of reasonable dimensions then you can go with classical normalizing flows. On the contrary, if you use high-dimensional images, which is usually the case with these models, you rather use the Trumpet injective flow model.

My implementation was a good experiment of how injective flows perform for OOD, but if you only want to understand the math and the main idea behind injective flows without bothering with their discrimination performance for classification tasks read the Konik Kothari et.al. paper 🙂.

I hope you will find my feedback useful 😉.

Good luck and all the best! Marija

---

From: Arktis2022 @.> Sent: Wednesday, November 16, 2022 5:23 PM To: MarijaStojchevska/Injective-Flow-for-Anomaly-Detection @.> Cc: Subscribed @.***> Subject: [MarijaStojchevska/Injective-Flow-for-Anomaly-Detection] paper about this code (Issue #1)

Hi! Is there a corresponding paper published for this code?

— Reply to this email directly, view it on GitHubhttps://github.com/MarijaStojchevska/Injective-Flow-for-Anomaly-Detection/issues/1, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEMYJTRI7CTXDONAMWWLRJLWIUDAPANCNFSM6AAAAAASCNJ7JQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[Arktis2022]

Hi! I am glad you are interested in Injective Flows! Unfortunately, there isn't official paperwork on anomaly detection with injective flows model implementation. However, you can read the official documentation for the TRUMPETS: Injective Flows for Inference and Inverse Problemshttps://proceedings.mlr.press/v161/kothari21a/kothari21a.pdf by Konik Kothari et.al. which was my main inspiration, and my model is mostly based on their ideas. Similarly, as they proposed, I am using an injective map or alternating injective and bijective mappings (for space reduction) followed by a bijective map or multiple bijective mappings (standard normalizing flow) to map input images into latent space (Gaussian distribution). To make things more convenient, I utilized the idea of separate model training as proposed in the above-mentioned paper, where the injective map and the second bijective map are trained separately. Once the injective flow model (injective map - bijective map) was implemented, I trained the model on non-anomalous images in order to learn their distribution. In this way, I have direct access to the density of the learned distribution both in the output space of the injective map and the output space of the bijective map. By implementing the model in this way, we have two output spaces and two learned distributions, isn't that cool? As you may know, one of the advantages of normalizing flows over other generative models is that they give direct access to likelihoods with respect to the learned distribution. So, my goal was to test the model on both anomalous and non-anomalous test images and for each of them to compute the two likelihoods given the learned distributions in the two output spaces. If the likelihood is far away from the learned distribution we can classify the sample as anomalous (again this is done in both spaces). In this way, I can see if the first space has better discriminatory performance that the other one. Intuitively, something that looks like an anomaly in one space would probably look like an anomaly in the other space too, which was proven through our work. In the end, we can conclude that the injective flow cannot contribute to better discriminatory performance in OOD problems, however, it is so powerful since it reduces the input data dimensionality and thus fastens the entire training/testing process. So if you don't suffer from memory/time execution problems and work with data of reasonable dimensions then you can go with classical normalizing flows. On the contrary, if you use high-dimensional images, which is usually the case with these models, you rather use the Trumpet injective flow model. My implementation was a good experiment of how injective flows perform for OOD, but if you only want to understand the math and the main idea behind injective flows without bothering with their discrimination performance for classification tasks read the Konik Kothari et.al. paper 🙂. I hope you will find my feedback useful 😉. Good luck and all the best! Marija … ____ From: Arktis2022 @.> Sent: Wednesday, November 16, 2022 5:23 PM To: MarijaStojchevska/Injective-Flow-for-Anomaly-Detection @.> Cc: Subscribed @.> Subject: [MarijaStojchevska/Injective-Flow-for-Anomaly-Detection] paper about this code (Issue #1) Hi! Is there a corresponding paper published for this code? — Reply to this email directly, view it on GitHub<#1>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEMYJTRI7CTXDONAMWWLRJLWIUDAPANCNFSM6AAAAAASCNJ7JQ. You are receiving this because you are subscribed to this thread.Message ID: @.>

Thank you very much for your reply! I will read the content carefully