Questions for consultation

[davidhouse2023]

Dear author, it is a great honor to have read your paper. There are two things that I am not sure about: 1. How the CT sequence after obtaining features is input into the subsequent two transformers for processing. 2. How the historical CT information and report information used in the model are obtained

[sezginerr]

Hello @davidhouse2023, for the first question, you can see encoder_decoder.py script (also mentioned in the preprint). For the second question, please see decode in encoder_decoder of ct2replong:

def decode(self, hidden_states, hidden_word, hidden_image, src_mask, att_mask2, tgt, tgt_mask, mask):
        if (mask == None):
            mask = tgt_mask

        hidden_word = self.attn(hidden_word, hidden_states, hidden_states, src_mask)
        hidden_word = self.attn(hidden_word, hidden_image, hidden_image, mask)
        hidden_image = self.attn(hidden_image, hidden_word, hidden_word, att_mask2)
        hidden = torch.cat([hidden_image, hidden_word], dim=1)

        mask_all = torch.cat([mask, att_mask2], dim=2)
        memorym = self.rm.init_memory(hidden_states.size(0)).to(hidden_states)
        memorym = self.rm(self.tgt_embed(tgt), memorym)

        memory = self.attn(self.mlp(memorym), hidden_states, hidden_states, src_mask)
        memory_all= self.attn(self.mlp(memorym), hidden, hidden, mask_all)
        memory = self.mlp1(memory)
        memory_all = self.mlp1(memory_all)
        memory1 = memory

        mask = mask_all

[sezginerr]

I am closing this for now. Let us know if you have further questions.

[hari3100]

Hi @davidhouse2023 , I hope you are doing well, I'm commenting in hopes that you have already ran the CT2Rep model on your own system, could you help me with the inference of this model i have a few questions as well regarding the input size, etc. But the main thing I'm looking for is how can I run the model, I'm new to medical imaging ,and I'm not understanding the code which is already present in the repo, could you maybe share some inference code you wrote, or guide me on how to do it ?....any feedback or guidance will be greatly appreciated...Thank you for your time!