search

microsoft / CCF

Confidential Consortium Framework
https://microsoft.github.io/CCF/
Apache License 2.0
761 stars 205 forks source link

Tlai experiment #6134

Closed achamayou closed 4 weeks ago

achamayou commented 4 weeks ago

The usual re-creation as a branch because of TLAi permissions, see #6132

achamayou commented 4 weeks ago

image

I give up.

lemmy commented 4 weeks ago

Apparently an intermittent issue:

image
lemmy commented 4 weeks ago

@pelikhan Is there any non-determinism in relevant parts of GenAIScript that could explain this?

pelikhan commented 4 weeks ago

This happens when we tried to read the LLM configuration information from the environment variables, like OPENAI_API_KEY, ... so there should not be anything non-deterministic.

I will add more logging in genaiscript to list the environment variables it found; but could it be that those are missing in the run? We could have a quick prescript that makes sure the OPENAI_API_KEY is set.

lemmy commented 4 weeks ago

It could have been an intermittent Github action issue, but I think it's less likely.

lemmy commented 4 weeks ago

@pelikhan Happened again in another workflow: https://github.com/microsoft/CCF/actions/runs/8746091121/job/24002294227

pelikhan commented 4 weeks ago

I will push a new cli with added logging.

achamayou commented 4 weeks ago

@lemmy yes, it's not on all the runs, and I can't see any obvious pattern to it. The PR would not have worked anyway because it looks like TLAi only looks at diffs, not whole files.