B2AI Prep

A simple Python package to prepare acoustic data for the Bridge2AI voice project.

Caution: this package is under active development and interfaces may change rapidly.

Installation

Requires a Python >= 3.10, <3.12 environment

pip install b2aiprep

Usage

See commands available through the CLI:

b2aiprep-cli --help

Convert an audio file to features:

The simplest form takes an audio file, a subject id, and a task name.
```
b2aiprep-cli convert test_audio.wav s1 mpt
```
It will save a pytorch .pt file with a dictionary of features. This can be loaded by torch.load(). The file is named following a simple convention: sub-<subject_id>_task-<task_name>_md5-<checksum>_features.pt

To enable speech to text transcription, specify the --speech2text flag.
Batch process audio files

This requires a CSV file, where there is a column header called filename and then each subsequent line is of the form: path/to/audio.wav

This also supports a CSV file where each line is of the form: path/to/audio.wav,subject_id,task_name

To generate this csv file from the Production directory pulled from wasabi, use command 3.
```
b2aiprep-cli batchconvert filelist.csv --plugin cf n_procs=2 --outdir out --save_figures
```
The above command uses pydra under the hood to parallel process the audio files. All outputs are currently stored in a single directory specified by the --outdir flag.

One can also generate a hugging face dataset in the output directory by specifying the --dataset flag.

To enable speech to text transcription, specify the --speech2text flag.
Generate csv file to feed to batchconvert
```
b2aiprep-cli createbatchcsv input_dir outfile
```
The input directory should point to the location of the Production directory pulled from Wasabi e.g. /Users/b2ai/production.

This directory can have subfolders for each institution, (e.g. production/MIT), and each subdirectory is expected to have all the .wav files from each institution.

Outfile is the path to and name of the csv file to be generated, e.g. audiofiles.csv

The csv file will have a header named filename with all the filenames listed under.
Verify if two audio files are from the same speaker
```
b2aiprep-cli test_audio1.wav test_audio2.wav --model 'speechbrain/spkrec-ecapa-voxceleb'
```
This will use the speechbrain speaker recognition model to verify that the two audio files are from the same speaker.

There is a notebook in the docs directory that can be used to interact with the library programmatically.
Convert the speaker in the source audio file (1st argument) into the speaker of the target audio file (2nd argument) and save the result in the output file (3rd argument)
```
b2aiprep-cli convert-voice data/vc_source.wav data/vc_target.wav data/vc_output.wav
```

Transcribe the audio

b2aiprep-cli transcribe data/vc_source.wav

Or use a different model. Note that the large model may take some time to download.

b2aiprep-cli transcribe data/vc_source.wav --model_id 'openai/whisper-large-v3' --return_timestamps true

BIDS-like data

This package provides conversion from a RedCap/file-based custom structure into a BIDS-like structure for downstream analysis. In addition, utilities are provided for working with the BIDS-like structured data and to support data analysis of voice and questionnaire data.

Convert a RedCap CSV and a folder of audio files into the BIDS format:

b2aiprep-cli redcap2bids bridge2ai_voice_data.csv --outdir output --audiodir audio

The audiodir option above can be omitted, in which case no audio data is reorganized.

See the tutorial.ipynb for a few use examples of data in the BIDS-like format.

Summer School Data Preparation

This command organizes the data with the BIDS-like conversion tool, extracts audio features, and saves the whole thing as a .tar file for easy distribution for the Bridge2AI Summer School:

b2aiprep-cli prepsummerdata \
   [path to RedCap CSV] \
   [path to Wasabi export directory] \
   [desired path to BIDS output] \
   [desired output path for .tar file]

Streamlit dashboard

A dashboard is provided to help navigate the data in the BIDS format. Launch the dashboard from the repository folder with:

streamlit run src/b2aiprep/app/Dashboard.py [path to BIDS directory]

After the Streamlit dashboard opens, please wait for the BIDS data to be loaded while on the Dashboard page, as an error regarding a non-initialized bids_dir directory may pop up if the BIDS data has not been fully loaded.

sensein / b2aiprep