ecomontec / ecoSound-web

Web application for ecoacoustics to manage, navigate, visualise, annotate, and analyse soundscape recordings.
https://ecosound-web.de/
GNU General Public License v3.0
21 stars 3 forks source link

connect recordings from external server over SFTP #218

Open kdarras opened 2 years ago

kdarras commented 2 years ago

To enhance integration of multiple platforms and collaboration it should be possible to use recordings stored on other servers over SFTP.

A sub-button "Connect (SFTP)" should be added to the "Add" button of the recordings addition function (#412 ) so that users are prompted with the required parameters and credentials (just like in Filezilla: user, password, host, port, protocol, etc.)

Thus the website should allow using remote instead of local files. All other data (table data, thumbnails) would be stored in the database and website just like before, though.

This must be secure, and the credentials stored to enable a persistent, read-only connection. The remote server will typically belong to a partner institution/university.

After integrating the remote files, the usual integration steps prompting the user for the parameters that are filled into the recordings table follow.

These remote recording files can accordingly not be deleted. If they are selected for deletion, it should instead be possible to disconnect them, and the warning message should tell the users that the associated meta-data (the data inside the recordings table) will be deleted too.

These recordings can be shared just like locally stored recordings with other users. When users try to access remote recordings but have no access to them, they should be prompted for SFTP access credentials and parameters, and the system should store them securely. An icon should be used to distinguish remote recordings from local ones in all three collection views.

The SFTP connection should be cut when the user logs out. However, passwords need to be entered again every time a user logs back into ecoSound-web and accesses a remote file.

shofstet commented 1 month ago

For us it would be the local filesystem also important (f.e. over SMB or NFS-mount).

shofstet commented 1 month ago

selection

detail

If you select the menu item ‘add local Recording folder’ under ‘recordings’ and specify a local folder, this should be used as the storage location. The images required for ecosound-web should be created and stored internally (sounds/images/x/y/....png). The audio files are stored relative to the specified folder. e.g. ./CAM_A/2024-01-01/Data/SMA03335_20240527_053602.flac

The logic must recursively read in all audio files.

The sounds/sounds/... files are not created. If the original file is not in the correct format (wav or flac/ogg), it will be created temporarily when it is selected or when it is required. If a result folder is specified, this means that the result file is stored under the same file name (folder structure), e.g. ./CAM_A/2024-01-01/Data/SMA03335_20240527_053602.BirdNET.selection.table.txt

Additional input parameters may be required, such as model type e.g. Birdnet-2.4 and possibly a comment field such as details of the model used and/or result suffix. This result file should be parsed and written to the tag database table.

I think that only local folders should be selected. It’s possible to mount different network file protocols f. e. S3, NFS, Samba, FTP.

kdarras commented 1 week ago

Edited first post and moved the content of @shofstet 's post to another issue: #532