[Question]: Incorrect `condition_mode` parameter value in `get_condition_ppl` function

[charloco]

Describe the bug

In the function get_condition_ppl of the class, when the condition_in_question parameter is "before", the condition_mode parameter should be "before", but it is currently "after". This may cause incorrect behavior of the function when handling the "before" condition.

Here is the relevant code snippet:

def get_condition_ppl(
    self,
    text: str,
    question: str,
    condition_in_question: str = "none",
    granularity: str = "sentence",
):
    ...
    elif condition_in_question == "before":
        return self.get_ppl(
            question + text,
            granularity=granularity,
            condition_mode="after",
            condition_pos_id=self.get_token_length(question) - 1,
        )
    ...

It is suggested to change condition_mode="after" to condition_mode="before" .

Steps to reproduce

No response

Expected Behavior

No response

Logs

No response

Additional Information

No response

[iofu728]

Hi @charloco, the logic here is correct, and I apologize for any confusion caused by the naming convention. Specifically, the "condition_inquestion" modes dictate how the conditional probability is calculated regarding the placement of the question in relation to the context. That is, "before" = $P(x{\text{que}}, x{\text{doc}})$ and "after" = $P(x{\text{doc}}, x_{\text{que}})$.

The "conditionmode" controls the range considered when calculating PPL. For example, "before" = $\text{PPL}([:x{\text{que}}])$ and "after" = $\text{PPL}([x_{\text{doc}}:])$.

Given the different "condition_in_question" modes, we aim to obtain the conditional PPL influenced by the first part on the latter, thus "condition_mode" is consistently set to "after".

[charloco]

Thank you for your explanation. I now understand the logic behind “condition_in_question” and “condition_mode”.