Tracking instruction-tuned LLM openness. Paper: Liesenfeld, Andreas, Alianda Lopez, and Mark Dingemanse. 2023. “Opening up ChatGPT: Tracking Openness, Transparency, and Accountability in Instruction-Tuned Text Generators.” In Proceedings of the 5th International Conference on Conversational User Interfaces. doi:10.1145/3571884.3604316.
We're interested in instruction-tuned LLMs. Instruction-following behaviour is linked to the RLHF method of InstructGPT but can also be reached with forms of reinforcement learning that do not involve human feedback directly, e.g. with multitask finetuning or with synthetic 'RLHF' data. This creates a bit of a terminological conundrum. We don't want to strictly limit ourselves to RLHF because that would leave only the few models made by entities that have the resources to do large-scale RLHF.
In the overview table we don't have a lot of room for elaborate terms anyway. So, proposal: change RLHF to RL so that we have columns RL_data and RL_weights. Then add an explanatory note along these lines somewhere (perhaps at the bottom of the table).
We're interested in instruction-tuned LLMs. Instruction-following behaviour is linked to the RLHF method of InstructGPT but can also be reached with forms of reinforcement learning that do not involve human feedback directly, e.g. with multitask finetuning or with synthetic 'RLHF' data. This creates a bit of a terminological conundrum. We don't want to strictly limit ourselves to RLHF because that would leave only the few models made by entities that have the resources to do large-scale RLHF.
In the overview table we don't have a lot of room for elaborate terms anyway. So, proposal: change RLHF to RL so that we have columns
RL_dataandRL_weights. Then add an explanatory note along these lines somewhere (perhaps at the bottom of the table).