Data structures descriptions and their corresponding files

[adrianSRoman]

Let's elaborate here on the different data structures we are using and how we store then, as well as which file formats we are adopting:

• Impulse response metadata doa_xyz_<mic-type>.pickl / rirdata_dict.pkl (if using db_config.py)

  • File format: Pickle serilaized object
  • Brief description: Object used to save RIR metadata, optionally distance data
  • Data structure:
  • List, length n_rooms
    • Dict, keys doa_xyz or dist
    • List, length n_traj
      • List, length n_height
      • Numpy Array, shape (n_measurements, [3 if doa, 1 if dist])

• Room impulse response data rirs_<room_idx>_<roomname>.pkl

  • File format: Pickle serialized object
  • Brief description: A dict with a nested list/array structure containing all RIR measurements in a given room
  • Data structure:
  • Dict, keys mic or foa
    • List, length n_traj
    • List, length n_height
      • Numpy Array, shape (len_signal, n_channels, n_measurements)

• Generation parameters db_config_<name>.obj

  • File format: Pickle serialized object
  • Brief description: Config file for generating databases with tons of key pointers. Mostly inherited from generation_parameters.py and specified in db_config.py. Current plan is to revise this into .yml files per #1
  • Data structure:
  • _rirpath
    • STR - path to TAU-SRIR_DB
  • _mixturepath
    • STR - path to nixture output
  • _measinfo
    • DICT - keys are room names
    • trajectories - LIST of STR
    • trajectory_type - STR - {'circular', 'linear'}
    • distances - np.ndarray of FLOAT - (1, n_trajs) for circular, (2,3, n_trajs if linear)
    • heights - np.ndarray of FLOAT - (1,[3-9])
    • mic_position - np.ndarray of FLOAT - (1,3)
  • _rirdata
    • np.ndarray - Each row is a room(9,1)
      • np.void - len 4 ???
      • np.ndarray - name of room (1,)
      • np.ndarray - year, 2019 (1,1)
      • np.ndarray - (n_trajs, n_heights) - paths
        • np.void - len 1
        • np.ndarray - (len_traj, 3)
      • np.ndarray of INT - (n_heights,m) - path lengths
        • INT - len_traj
  • _nb_folds
    • INT - number of folds
  • _rooms2fold
    • LIST - (2,) - list of folds
      • LIST - (6,) list of room indices
  • _db_path
    • STR - path to source audio (nigens in our case)
  • _db_name
    • STR - name of database (nigens in our case)
  • _fs
    • INT - sample rate
  • _class_dict
    • DICT - class name (key) to index (item, INT)
  • _class_mobility
    • LIST of INT
  • _classes
    • LIST of STR
  • _nb_classes
    • INT - number of classes
  • _samplelist
    • LIST of DICTS - (2,)
      • class
      • np.ndarray of INT (117,) - class idx
      • audiofile
      • np.ndarray of STR (117,) - sample path
      • duration
      • np.ndarray of FLOAT (117,) - sample length (seconds)
      • onoffset
      • np.ndarray of np.ndarray (117,)
        • np.ndarray of FLOAT - (1,2) - onset/offset of sample
      • nSamples
      • INT - number of samples (117)
      • nSamplesPerClass
      • np.ndarray of FLOAT - (15,) - frequenct of each class
      • meanStdDurationPerClass
      • np.ndarray of FLOAT - (15, 2) - mean and std of durations of each class
      • minMaxDurationPerClass
      • np.ndarray of FLOAT - (15, 2) - min and max of durations of each class
  • _apply_class_gains
    • BOOL - apply per-class gains
  • _class_gaines
    • LIST - per-class gain scaling factor

[ChrisIck]

Added the details I know. I think our storage of the RIR metadata might differ since I'm using the db_config file object, but the structure should be similar.

[ChrisIck]

This may be redundant as we're standardizing our rir/doa storage via SOFA (see #20) and reworking our sound event pipeline via #5