openedx / event-routing-backends

Consume edx tracking events and transform/transmit them to other LRSs.
GNU Affero General Public License v3.0
9 stars 17 forks source link

Add variant / seed to problem_check events #369

Open bmtcril opened 10 months ago

bmtcril commented 10 months ago

See #365 for more details, but the issue is that advanced problems can be randomized as documented here. Those seeds can cause there to be different correct answers for the same problem, making it difficult to understand which values are "correct". We may want to add the variant / seed of the problem to the transforms.

Here is a sample problem to test with:

<problem>

<script type="loncapa/python">

def test_add(expect, ans):
    try:
        a1=int(ans[0])
        a2=int(ans[1])
        return (a1+a2) == int(expect)
    except ValueError:
        return False

target_sum = random.randint(0, 100)
</script>

<p>Enter two integers that sum to $target_sum: </p>
<customresponse cfn="test_add" expect="$target_sum">
  <textline size="40" label="Integer #1"/><br/>
  <textline size="40" label="Integer #2"/>
</customresponse>

</problem>

To use this, in Studio: Add Problem -> Advanced tab -> Blank Advanced Problem. Paste the above. Then you can set the randomization value in the settings to "Always". You should be able to reset the problem and get a different value each time. This does get passed in the tracking event as "variant", but I wasn't able to find an xAPI concept for this information so we'd probably have to create one.

rgraber commented 9 months ago

Hey @bmtcril What do we think is the impact of this work? Is there something that makes it particularly high priority?

e0d commented 9 months ago

@rgraber this summarizes the purpose of the work, https://github.com/openedx/event-routing-backends/issues/365#issuecomment-1801948211

My understanding is that 2U is not using event-routing-backends. Is there a reason this is seen as a risk on your end?

rgraber commented 9 months ago

No, but since we technically own this repo we were wondering how we should prioritize this work. It doesn't seem terribly high priority so may end up on our backlog.

e0d commented 9 months ago

I don't think it makes sense of 2U to own this as you don't use it and everyone focused on building and migrating to Aspects does.

In fact, I thought we had said that @ziafazal and edly would be the owners of this repos some months back.

Either way, given current priories and incentives, I think we should transfer ownership of this repo from 2U to edly and Axim represented by Zia and @bmtcril .

If there are no objections, we can make that effective immediately.

ziafazal commented 9 months ago

I don't think it makes sense of 2U to own this as you don't use it and everyone focused on building and migrating to Aspects does.

In fact, I thought we had said that @ziafazal and edly would be the owners of this repos some months back.

Either way, given current priories and incentives, I think we should transfer ownership of this repo from 2U to edly and Axim represented by Zia and @bmtcril .

If there are no objections, we can make that effective immediately.

I have no objections to it.

bmtcril commented 9 months ago

That's basically how we've been operating, so that makes sense to me. 👍

e0d commented 9 months ago

@rgraber I assume the message above

@rgraber rgraber removed this from Arch-BOM 6 hours ago

means you are happy with the proposal. @bmtcril and @ziafazal will maintain going forward. Let me know if I've gotten that right.