`681532_2023-10-17` missing ephys devices

[bjhardcastle]

reported by @olsenshawn

QC notebook with traceback: https://osf.io/dt46y?view_only=035876aa04154f04ab5ca1e6be18ee72

I suspect it's a problem with accessing uploaded raw data upload, which changed this week after updating aind-data-transfer

[bjhardcastle]

The problem is that probe E wasn't sorted, but photodocs and insertion records suggest that it was inserted.

May be able to learn more from sorting logs: sorted data asset

[bjhardcastle]

Raw data on E looks like it was not inserted:

import matplotlib.pyplot as plt
import npc_sessions

plt.imshow(
    npc_sessions.Session('ecephys_681532_2023-10-17')
    .acquisition['AP']['probeE'].data[30000*600:30000*600 + 60000, :].T,
    aspect=20,
)
plt.show()

Data look fine from probe E in the next day's session:

plt.imshow(
    npc_sessions.Session('ecephys_681532_2023-10-18')
    .acquisition['AP']['probeE'].data[30000*600:30000*600 + 60000, :].T,
    aspect=20,
)
plt.show()

[bjhardcastle]

@arjunsridhar12345 @egmcbride What's the best course of action here?

• we're currently use the insertion record json to decide which probes to process
• we now have an inserted probe (which will have annotations) but isn't processable
• we could just process the probes that are sorted, but then we won't spot incomplete uploads
• do we need a session config for "probe letters to skip"?

[bjhardcastle]

from output (also captured in processing.json)

Preprocessing recording: experiment1_Record Node 104#Neuropix-PXI-100.ProbeD-AP_recording1 Duration: 6812.98 s Bad channel detection:

• dead channels - 142
• noise channels - 106
• out channels - 0 More than 50.0% bad channels (248). Skipping further processing for this recording. Preprocessing recording: experiment1_Record Node 104#Neuropix-PXI-100.ProbeE-AP_recording1 Duration: 6812.98 s Bad channel detection:
• dead channels - 0
• noise channels - 384
• out channels - 0 More than 50.0% bad channels (384). Skipping further processing for this recording.