search

AllenNeuralDynamics / aind-ephys-pipeline

Code Ocean pipeline for ephys processing with Kilosort2.5
MIT License
5 stars 2 forks source link

figurl components broken #20

Open bjhardcastle opened 1 month ago

bjhardcastle commented 1 month ago

Trying to view the drift map here: https://figurl.org/f?v=npm://@fi-sci/figurl-sortingview@12/dist&d=sha1://0bec34a9dd60885850a11a97dee88e28220a85d4&label=ecephys_666986_2023-08-16_15-12-36%20-%20experiment1_Record%20Node%20102%23Neuropix-PXI-100.ProbeB-AP_recording1&zone=aind

(From data asset 5b8cd734-6308-4bc9-8e9c-3eb05ef14e4d)

Error getting file data for sha1://4612348e5f7d8b1db5b596de79a89db0a7a53141: Error loading file data: Error getting file (403) (<?xml version="1.0" encoding="UTF-8"?> <Error><Code>InvalidObjectState</Code><Message>The operation is not valid for the object's access tier</Message><StorageClass>INTELLIGENT_TIERING</StorageClass><AccessTier>ARCHIVE_ACCESS</AccessTier><RequestId>Y29WA5PB9G0949R4</RequestId><HostId>zy6ddnyesE9Ho4Eu1ME4Qu1ifoz81X4ynWelM01JpGHLJdEIwIXAT+jjDU58fshqp+RouXYQP8w=</HostId></Error>): https://s3.us-west-2.amazonaws.com/aind-kachery-zone/sha1/46/12/34/4612348e5f7d8b1db5b596de79a89db0a7a53141?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAWZ4O6ZMIJRNNDJOR%2F20240917%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240917T175545Z&X-Amz-Expires=3600&X-Amz-Signature=d4f4e1b35c15e00fe7ab0eab005681a869b597103a5bf88bb136e421844ec487&X-Amz-SignedHeaders=host
alejoe91 commented 1 month ago

@bjhardcastle can you try again? It's working on my side!

bjhardcastle commented 1 month ago

The time series page loads, but the drift map and motion tabs still display the error. Do they work for you via the same link?

alejoe91 commented 1 month ago

Ah sorry I read it too quickly!

@magland could you take a look? The DRIFT MAP and MOTION tabs were created with the sortingview.views.Image.

magland commented 1 month ago

The message I am noticing is "The operation is not valid for the object's access tier"

I wonder if the bucket is configured to automatically change the access tier for old files? Was this one generated recently or long ago?

The next step is to check whether the following object is actually in the bucket, and if so, is it available for download with a standard http get operation?

https://s3.us-west-2.amazonaws.com/aind-kachery-zone/sha1/46/12/34/4612348e5f7d8b1db5b596de79a89db0a7a53141

What I pasted is not a signed URL so a direct GET is not going to work. You'll need to somehow probe it using AWS SDK or some other tool.

alejoe91 commented 1 month ago

@dyf can you help us out?

dyf commented 4 weeks ago

We are using intelligent tiering for this bucket. Nobody accessed that file for 90 days, so it was archived.

magland commented 4 weeks ago

@dyf Curious if there's a way to unarchive such a file, and whether that can be done through an API call of some kind?

dyf commented 1 week ago

@magland yes, we just have to change the storage class of all of the objects in the bucket from IT to standard.