dandi / dandi-archive

DANDI API server and Web app
https://dandiarchive.org
13 stars 11 forks source link

Provide access for external services to embargoed content #2066

Open yarikoptic opened 5 hours ago

yarikoptic commented 5 hours ago

Inspired by @aaronkanzer pointer to docs on how integration is happening for zarrs in LINC in

but I guess in many cases for simple .nwb we would be fine if we just point to "minted on demand" (the /download/ end point at the time of external service might already be lacking session/context to authenticate seems since https://neurosift.app/?p=/nwb&url=https://api.dandiarchive.org/api/assets/dd8e4fed-743b-497e-b4da-db0ba71ff5d8/download/&dandisetId=001169&dandisetVersion=draft does not work for me even though I can download that asset in the same browser... attn @magland ) Thus we would just annotate which services in general would have ability to access embargoed assets, and trigger minting URLs upon click instead of providing ready to use URLs.

aaronkanzer commented 4 hours ago

@yarikoptic -- thanks for filing this -- just tagging some working code we have for calling any operation upon button click in the FileBrowser here (this is from the LINC side when we want to issue CloudFront cookies prior to proceeding to neuroglancer)

I'm assuming this this nwb use case you'd also want to ping an endpoint rather than including a potentially minted presigned URL in every asset.metadata upfront -- see code reference here and here

Happy to share additional unsolicited advice 😉 -- (e.g. make sure to leverage aioboto3 here instead of boto3 -- boto is blocking for expensive presigned url construction!)

magland commented 4 hours ago

See https://github.com/dandi/dandi-archive/issues/2065#issuecomment-2450730584

Neurosift supports viewing of embargoed assets (if it's working properly)