search

mastodon-sc / mastodon

Mastodon – a large-scale tracking and track-editing framework for large, multi-view images.
BSD 2-Clause "Simplified" License
69 stars 20 forks source link

Support NGFF using n5-universe #328

Closed ksugar closed 1 month ago

ksugar commented 1 month ago

Overview

This PR implements loaders for the NGFF formats using n5-universe.

Supported formats

Format Metadata Storage Confirmed converter
HDF5 N/A local only Plugins › BigDataViewer › Export Current Image as XML/HDF5 in Fiji
OME-Zarr OME-NGFF v0.4 local or remote bioformats2raw
N5 COSEM or N5 Viewer local or remote File › Save As › HDF5/N5/Zarr/OME-NGFF ... in Fiji

Usage

  1. Select new project from URL in Mastodon launcher.

    image

  2. Press the browse button to open N5 dialog.

    image

  3. Click the Browse button to load local dataset, or input a URL and and press the Detect datasets button. Please use the URLs starting with http: or https: because the s3: protocol is not fully supported at the moment.

    image

  4. Select a (multiscale) dataset and press the OK button.

    image

  5. Save a .xml file that contains the SpimData information.

    image

  6. Press the create button.

    image

  7. Visualize the data in the bdv window.

    image

Work with authentication of the remote storage

If the remote storage requires authentication or fails to load data, a dialog will appear, allowing you to enter the username (or AWS_ACCESS_KEY_ID) and password (or AWS_SECRET_ACCESS_KEY).

image

Alternatively, the user can set the credentials permanently on the system using the AWS credentials provider chain. https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/DefaultAWSCredentialsProviderChain.html

AWS credentials provider chain that looks for credentials in this order:

  • Environment Variables - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (RECOMMENDED since they are recognized by all the AWS SDKs and CLI except for .NET), or AWS_ACCESS_KEY and AWS_SECRET_KEY (only recognized by Java SDK)
  • Java System Properties - aws.accessKeyId and aws.secretKey
  • Web Identity Token credentials from the environment or container
  • Credential profiles file at the default location (~/.aws/credentials) shared by all AWS SDKs and the AWS CLI
  • Credentials delivered through the Amazon EC2 container service if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI" environment variable is set and security manager has permission to access the variable,
  • Instance profile credentials delivered through the Amazon EC2 metadata service
tinevez commented 1 month ago

@ksugar Awesome! I will make basic tests then merge. All in time! Thank you so much Ko

tinevez commented 1 month ago

Doc for the N5 dialog is here: https://github.com/saalfeldlab/n5-ij?tab=readme-ov-file#open-n5