saulpw / aipl

Array-Inspired Pipeline Language
MIT License
119 stars 7 forks source link

when is this literal in scope? #33

Closed dovinmu closed 1 year ago

dovinmu commented 1 year ago
!literal
gpt-3.5-turbo
gpt-neo-20b
!split>model sep=\n
<snip>
!llm>classification model={model} max_tokens=1
# 'model' is in scope
!format
{model} {classification} ({target_scores_Yes}): {statement}
# model is out of scope! "AIPL Error (line 31 !format): 'model'"

!!python
<snip>
!compute-precision>precision classification target_scores_Yes
!format
{model:15} {task:15} {precision}
!print
# model is in scope again
dovinmu commented 1 year ago

this is the whole script: https://github.com/dovinmu/aipl/blob/5dfb58d65b042e2d9a36794388ad70e2006e7243/chains/benchmarks/bigbench-binary-classification.aipl

dovinmu commented 1 year ago

Possibly related, this is from the README: "By default, each operator is applied across the deepest nested table."

How do you apply an operator to the parent of the deepest nested table?

saulpw commented 1 year ago

In APL/etc the operators are scalar by default, but can be reranked with a "rank" adverb: https://xpqz.github.io/learnapl/rank.html

So maybe we make something like !!rerank 2 which would change the rank of the previous operator to be the parent of the deepest nested table. (0 would be the scalar elements, 1 would be the deepest table, 2 is its parent, etc).

saulpw commented 1 year ago

@dovinmu When you want to apply an operator to a parent table, please open a new issue with the script/snippet, and we'll make it work. I don't think we've needed this case yet.

Also let's try to figure out where and how to explain the scoping rules. We've talked through it a few times in chat and I'm not sure which (if any) of the explanations has been most illuminating. (Also we're still figuring it out ourselves :)

Closing this for now.