PAIR-code / llm-mediation-experiments

Platform for running experiments involving LLMs as online mediators.
Apache License 2.0
2 stars 3 forks source link

Design/implement separate accounts per experimenter #98

Open cjqian opened 4 weeks ago

cjqian commented 4 weeks ago

Idea: experiments are tied to a single experimenter (no concept of sharing across teams) for simplicity.

Consideration: what if you'd like to share an experiment view with someone else (e.g. sharing playtest results)? For now, should be fine to share a link of a participant's view after the game.

cjqian commented 4 weeks ago

From the document:

"We can easily configure experiment visibility with additional fields in experiment documents + additional experimenter custom claims (auth), but changing custom claims can be tedious so we should plan what logic we want. "

@LeoLaugier @GnRlLeclerc Not sure which one of you added the comment above; could you expand on what custom claims are?

GnRlLeclerc commented 3 weeks ago

From the document:

"We can easily configure experiment visibility with additional fields in experiment documents + additional experimenter custom claims (auth), but changing custom claims can be tedious so we should plan what logic we want. "

@LeoLaugier @GnRlLeclerc Not sure which one of you added the comment above; could you expand on what custom claims are?

Custom claims are a JSON object that we can associate to a specific user in order to specify more information about them. The user cannot change them.

Right now, "granting experimenter rights" is equivalent to setting a logged-in user's custom claims to { "role": "experimenter" }.

This can easily be done in development using the emulators, but is harder to do in production.

Examples: