Closed henktillman closed 5 months ago
Here is a unit test for the credential problem. It has something to do with blobfile.listdir
blobfile._common.Error: Encountered an error when requesting an access token: `invalid_request: AADSTS900144: The request body must contain the following parameter: 'client_id'. Trace ID: 0122581e-b632-44bc-a67d-10a375870d00 Correlation ID: 32846173-b7cc-4195-a763-1646b9860cf5 Timestamp: 2024-03-14 01:03:45Z`. You can attempt to fix this by re-running `az login`.
import blobfile as bf
pieces_path = f"az://openaipublic/neuron-explainer/subject-models/gpt2/small/model_pieces"
piece_files = list(bf.listdir(pieces_path))
print(piece_files)
Using https://openaipublic.blob.core.windows.net
makes no difference.
When you try to access model files in browser, such as gpt2"
via https://openaipublic.blob.core.windows.net/neuron-explainer/subject-models/gpt2/small
, it says that the specified blob file does not exist.
On the other hand, pre-computed activations https://openaipublic.blob.core.windows.net/neuron-explainer/gpt2_small_data/collated-activations/{layer_index}/{neuron_index}.json
can be accessed via browser.
I think model paths are broken.
@henktillman you sir are my hero. I was about to start fixing it myself.
As a sidenote, I don't understand how this project can have 2.9k stars, and nobody took notice that it actually doesn't work for anyone outside of OAI.
Please consider turn on the issue tab. It would make it easier to discuss issues like this one.
(thx, following, https://twitter.com/ianand/status/1767827185170300935)