How to connect to raw_sound?

[lukassteiner]

Can you please help me how to connect DartMeltySoundFont with raw_sound?

I am using this example from raw_sound: https://pub.dev/packages/raw_sound/example

In that example, there is this line

await _playerPCMI16.feed(dataBlock);

Which apparently "Feeds the player with raw PCM [data] block" according to the documentation of this method. The parameter of feed() is expected to be of the type Uint8List.

When I look at the sample code of DartMeltySoundFont, I see the variable buf16 of the type ArrayInt16.

How do I cast ArrayInt16 to Uint8List?

Thanks for your help. I am new to the area of digital sound creation.

[chipweinberger]

I won't have time to help you, but the array16 is a byte buffer "under the hood".

Take a look at the internal variables

[lukassteiner]

Thank you for your answer @chipweinberger .

I tried but was unable to figure it out.

If you have a few minutes anyway, I created a question on Stackoverflow with a running example app that combines DartMeltySoundFont and raw_sound but has the cast from ArrayInt16 to Uint8List missing. After I solved this, I will add a PR to this repo with the example of how to connect these two flutter plugins. https://stackoverflow.com/questions/74928785/flutter-how-to-cast-arrayint16-to-uint8list

[lukassteiner]

Got the correct answer:

buf16.bytes.buffer.asUint8List()

[lukassteiner]

One last question @chipweinberger : How can I calculate how many seconds something will play when I initialise the synth with a given sampleRate and blockSize and add one block.

Example: sampleRate: 44100 blockSize: 64

I add 30 blocks. How many seconds will that play and how can I calculate this?

[chipweinberger]

64*30=1920 samples

1920 / 44100 = 0.043 secs

[chipweinberger]

yes feel free to open a PR with an "examples" folder added.

"examples/flutter_raw_sound/flutter_raw_sound.dart"

and feel free to update the readme with a link to that file.

[fly-qp]

I have combined flutter_raw_sound and dart_melty_soundfont. BUT its not very performant and when initializing and/or startPlaying it is lacking a bit. If we can fix this i would be more then happy to update the readme as well as adding my player class as an example :)

[chipweinberger]

This was the test code I made. Haven't touched it in a year.

import 'dart:typed_data'; // for Uint8List
import 'dart:math' as math;

import 'package:flutter/services.dart' show rootBundle;

import 'package:flutter/material.dart';
import 'package:raw_sound/raw_sound_player.dart';

import 'package:chip_flutter_experiments/DartMeltySoundFont/lib/preset.dart';
import 'package:chip_flutter_experiments/DartMeltySoundFont/lib/synthesizer.dart';
import 'package:chip_flutter_experiments/DartMeltySoundFont/lib/synthesizer_settings.dart';
import 'package:chip_flutter_experiments/DartMeltySoundFont/lib/audio_renderer_ex.dart';
import 'package:chip_flutter_experiments/DartMeltySoundFont/lib/array_int16.dart';

class MyPcmMeltyApp extends StatefulWidget {
  const MyPcmMeltyApp({Key? key}) : super(key: key);

  @override
  _MyAppState createState() => _MyAppState();
}

class _MyAppState extends State<MyPcmMeltyApp> {

  final _playerPCMI16 = RawSoundPlayer();

  final ArrayInt16 buf16 = ArrayInt16.zeros(numShorts: 44100 * 2);

  Synthesizer? _synth;

  @override
  void initState() {
    super.initState();

    _loadSynth();

    //  release any initialized player instances
    _playerPCMI16
        .initialize(
      bufferSize: 4096 << 4,
      nChannels: 1,
      sampleRate: 16000,
      pcmType: RawSoundPCMType.PCMI16,
    )
        .then((value) {
      setState(() {
        // Trigger rebuild to update UI
      });
    });
  }

  _loadSynth() async {

    String asset = 'assets/Metronom.sf2';

    ByteData bytes = await rootBundle.load(asset);

    _synth = Synthesizer.loadByteData(bytes, SynthesizerSettings());

  }

  @override
  void dispose() {
    _playerPCMI16.release();
    super.dispose();
  }

  Future<void> _playPCMI16() async {

    if (_playerPCMI16.isPlaying) {
      return;
    }

    await _playerPCMI16.play();

    setState(() {
      // Trigger rebuild to update UI
    });

    _synth!.noteOffAll();

    int ch = 0;
    for (Preset p in _synth!.soundFont.presets){
      // Select instrument
      _synth!.processMidiMessage(channel:ch % 16, command:0xC0, data1:p.patchNumber, data2:0);
      _synth!.noteOn(channel: ch % 16, key: 76, velocity: 120);
      ch++;
    }

    _synth!.renderMonoInt16(buf16);

    await _playerPCMI16.feed(buf16.bytes.buffer.asUint8List());
  }

  Future<void> _pausePCMI16() async {
    await _playerPCMI16.pause();
    setState(() {});
  }

  @override
  Widget build(BuildContext context) {

    if (!_playerPCMI16.isInited) {
      return const Text("not initialized");
    }

    return Column(
      children: [
        Container(color: Colors.orange, height: 100),
        Card(
          child: Row(
            children: [
              IconButton(
                icon: Icon(
                    _playerPCMI16.isPlaying ? Icons.stop : Icons.play_arrow,
                    color: Colors.black),
                onPressed: () {
                  _playerPCMI16.isPlaying ? _pausePCMI16() : _playPCMI16();
                },
              ),
              const Text('Test PCMI16 (16-bit Integer)'),
            ],
          ),
        ),
      ],
    );
  }
}

[chipweinberger]

@fly-qp, a PR would be great.

[chipweinberger]

I added /example/lib/main.dart to the repo