no WaveReader (and writer) on the java side?

[scar20]

Couldn't not find it. Looked in CMakeList.txt and mwengine.i; it appear in CMakeList but is not in mwengine.i. It is because it is optional and we have to add it manually in mwengine.i or simply because it is not supposed to be available? I will build with it for now since I need my model to read from the user directory but tell me if I should not.

[igorski]

The reason why it's not directly exposed is because it is used under the hood by the createSampleFromFile|Asset methods listed in the JavaUtilities class as these cater to the average use case (basically loading a file into the SampleManager or into the TablePool). There is no WaveWriter exposed though (as recording invoked by the MWEngine handles the writing of the output file, once more, under the hood).

That is the sole reason though. If you need direct access to the WaveRead+Write API, there is absolutely no harm done in configuring the makelists to suit your needs :)

[scar20]

OK I got it... its used internally in JavaUtilities so no need to include in mwengine. Sorry for the noise. bool JavaUtilities::createSampleFromFile( jstring aKey, jstring aWAVFilePath )

[scar20]

I though I could live without the need to access those internal classes but now I need a short buffer that will come from current sample. WaveWriter provide exacly what I need - bufferToPCM() - and I can't get it to "materialize" on the java side. I've seen that it was already included in CMakeList.txt and I added the includes in mwengine.i. Then gradle :mwengine:assemble and got those errors at the :mwengine:externalNativeBuildDebug stage in mwengineJAVA_wrap.cxx: error: call to implicitly-deleted copy constructor of 'std::ofstream' (aka 'basic_ofstream<char>') pointing to this:

SWIGEXPORT jlong JNICALL Java_nl_igorski_mwengine_core_MWEngineCoreJNI_WaveWriter_1createWAVStream(JNIEnv *jenv, jclass jcls, jstring jarg1, jlong jarg2, jint jarg3, jint jarg4) {
  jlong jresult = 0 ;
  char *arg1 = (char *) 0 ;
  size_t arg2 ;
  int arg3 ;
  int arg4 ;
  std::ofstream result;

  (void)jenv;
  (void)jcls;
  arg1 = 0;
  if (jarg1) {
    arg1 = (char *)jenv->GetStringUTFChars(jarg1, 0);
    if (!arg1) return 0;
  }
  arg2 = (size_t)jarg2; 
  arg3 = (int)jarg3; 
  arg4 = (int)jarg4; 
  result = MWEngine::WaveWriter::createWAVStream((char const *)arg1,arg2,arg3,arg4);
  *(std::ofstream **)&jresult = new std::ofstream((const std::ofstream &)result); 
  if (arg1) jenv->ReleaseStringUTFChars(jarg1, (const char *)arg1);
  return jresult;
}

Then I though to use it internally as a new static function in javautils since it already use WaveReader even if its not in mwengine.i. So I added: static short* bufferToPCM(MWEngine::AudioBuffer * aBuffer);

short* JavaUtilities::bufferToPCM( MWEngine::AudioBuffer *aBuffer)
{
    return WaveWriter::bufferToPCM( aBuffer );
}

then gradle :mwengine:assemble and hit those errors in the mwengine:compileDebugJavaWithJavac stage in the generated WaveWriter.java class:

cannot find symbol
        MWEngineCoreJNI.delete_WaveWriter(swigCPtr);
                       ^
  symbol:   method delete_WaveWriter(long)
  location: class MWEngineCoreJNI

for each of the methods in the class.

So long story short, I can't get wavewriter out. And I need badly a short buffer to finish wrapping up my model of the app. What about putting it in JavaUtils as a static function as I though as an addition to the utilities?

[igorski]

Hi, what you can do is indeed add the method to the java utilities file like you suggested:

static INT16* bufferToPCM( AudioBuffer *aBuffer )
{
    return WaveWriter::bufferToPCM( aBuffer );
}

(INT16 will turn into short during build). But be sure to NOT expose wavewriter.h and wavereader.h inside mwengine.i as the stack based interface for reading the streams is not compatible with the heap based translation of SWIG.

The fact that these classes are listed in CMakeLists.txt ensures they are built as part of the native binary. The contents of mwengine.i only lists the classes that should be made available to Java/Kotlin. By using the JavaUtilities as a "bridge", you can indirectly call the native layer code without this code having had a transformed API just to be used within Java.

[scar20]

But be sure to NOT expose wavewriter.h and wavereader.h inside mwengine.i

that was my error. Now assemble ok. but

(INT16 will turn into short during build)

It didn't; JavaUtilities.bufferToPCM() return a SWIGTYPE_p_INT16 and I dont know how to cast that into a short, seems very opaque. I tried to reassemble (always clean before) using short as the method return (just in case) but still get a SWIGTYPE_p_INT16 on the java side. Any flags or annotations that I missed? For now, I'll go the long route to copy twice the file (in getSampleFromFile() and someGetShortBufferFunction()) unless there is an easy fix. That would be nice and cleaner to have it available.

[scar20]

Sorry intermixed C++ and Java; I meant I expect to get a short[ ] (not a short*) on the java side.

[igorski]

Ah we now run against one of the limits of using SWIG. The short* is an unbounded Array (pointer-to) and therefor there is no known size when converting from/to Java over JNI. This would not be an issue when the size of the Array is fixed (as we could define a custom typemap), but in your case the PCM data can be of any arbitrary size.

Lets back track a little as I need to understand the use case here before I can think of a suitable solution as I can't quite see how to solve this quickly 🙈

Is it correct to assume that the audio file is a .WAV resource packaged in your application / device storage ? You want to access the raw file contents to display a waveform ?

Or is the audio generated at runtime ?

[scar20]

Is it correct to assume that the audio file is a .WAV resource packaged in your application / device storage ?
You want to access the raw file contents to display a waveform ?

Yes that is correct. if the size have to be known in advance, how about making it like this:

static jshortarray bufferToPCM( AudioBuffer *aBuffer )
{
    int size = aBuffer->bufferSize;
    short* buf = WaveWriter::bufferToPCM( aBuffer );

    return WaveWriter::bufferToPCM( aBuffer );
}

[scar20]

oops, mistyped and send unfinish... I was looking in jni.h for way to create a jshortarray, set it up and send that back, but probably naive thinking with same end result.

Anyhow here is what I do in the model app - the real app install a folder hierarchy in user storage and access is provided through room database:

public static void setUpSample(Application application) {
        String rootDir = "samples";
        String sourcePath = application.getFilesDir().getPath() + File.separator + rootDir;
        File dir = new File(sourcePath);
        String[] list = dir.list();
        int count = 0;
        for (String name : list) {
            Log.d(LOG_TAG, "file in userDir: " + name);
            String key = "00" + count++;
            JavaUtilities.createSampleFromFile(key, sourcePath + File.separator + name);
            File f = new File(sourcePath + File.separator + name);
            short[] buf;
            try {
                buf = getAudioSample(f);  // now copy again in buffer
                shortBuffers.add(buf);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        for (short[] b : shortBuffers)
            Log.d(LOG_TAG, "shortBuffers: " + b.length);
    }

    private static short[] getAudioSample(File f) throws IOException {
        Log.d(LOG_TAG, "in getAudioSample()");
        byte[] data;
        int size = (int) f.length();// - 44; // length minus header length
        data = new byte[size];
        FileInputStream fis = new FileInputStream(f);
        try {
            int read = fis.read(data, 0, size);  // I read the whole thing including header...
//                if (read < size) {
//                    int remain = size - read;
//                    while (remain > 0) {
//                        read = fis.read(tmpBuff, 0, remain);
//                        System.arraycopy(tmpBuff, 0, bytes, size - remain, read);
//                        remain -= read;
//                    }
//                }
        } catch (IOException e) {
            throw e;
        } finally {
            fis.close();
        }

        ShortBuffer sb = ByteBuffer.wrap(data).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer();
        short[] samples = new short[sb.limit()];
        sb.get(samples);
        return samples;
    }

[scar20]

sorry, seem I closed the issue while mistyped. did you get my last comment with public static void setUpSample(Application application) snippet?

[igorski]

I have given this thought and I believe this can't realistically be done. SWIG does not allow dynamic array lengths over JNI. Solutions could be to create wrapper functions that return total length and a per-sample getter, but that's very wasteful on resources. Another one could be to generate the waveform on the native layer, but that would imply adding a library/writing logic to generate an image file, which is not the purpose of the library.

As such, I think it's best to keep this logic (to be clear: drawing a waveform from an audio buffer) on the Java/implementation side.

On your Java implementation, about the line:

int size = (int) f.length();// - 44; // length minus header length

Please note that a .WAV header can have a length that deviates from the specced 44 bytes. Unless you have full control over the .WAV files in your app and can ensure they have the proper header length, its best to scan for the data identifier (4 chars with sum value 0x61746164).

[scar20]

Yea... that was streamlining.... :) Since its just for display and the routine draw from min and max, I figured it doesn't matter that much and I draw everything including the header... Thanks for the tips.

I'm actually looking/trying to build libsndfile to use it as the file importer - that what had kept me silent for a while... So I will use createSampleFromBuffer() instead (and draw from the data itself). The benefit is that I don't get to deal with format (a pain) and it offer more possibility to handle different files types. I had trouble with the files they give me. The bank is originally 44.1Khz from a mac - ProTool I think, I asked them to provide me a 48000 converted one. When I give those sample to createSampleFromFile(), I got 0 lenght file according to the sample manager (and no sound obviously). I have to import and export from Audacity to fix this. And I dont want to do the 100+ samples that way....

Looking at the output of sndfile-info, I can see these is "dirt" in the header so probably the cause the bug which I don't ask you to fix, createSampleFromFile() doesn't need to be bullet proof, just need to provide a clean header. I followed your link for the wav format and nothing there that would explain the meaning of the extraneous "data". That would need further investigation into the format itself but I think its not worth. Since libsndfile is unnafected, it will be better to just dynamic link with it to open files and provide buffers - if I can compile it the right way. In the same vein (but for another thread), I'm looking also at libsamplerate (Secret Rabbit Code) https://github.com/libsndfile/libsamplerate and r8brain-free-src https://github.com/avaneev/r8brain-free-src for sample rate conversion; so to keep all the bank in 44.1 and feed the engine native sample rate. Those two are very high quality conversion, and can be statically liked given their licenses. Rabbit claim to be able to offer real time stretching/pitch through a callback although probably not at same quality level. I have to check that out since this is a functionality need for another "tool" of Fonofone. There is also SoundTouch but their license seems to make it more difficult to integrate. Ok enough of this, I'll try to maintain my thread more MWEngine centric.

Below the sndfile-info output, first the file straight from the mac, and the import/export from Audacity - note the "*** junk : 28 (unknown marker)" which doesn't look reassuring.

File : .\bonjour_hello48monoprotool.wav
Length : 933008
RIFF : 933000
WAVE
*** junk : 28 (unknown marker)
fmt  : 16
  Format        : 0x1 => WAVE_FORMAT_PCM
  Channels      : 1
  Sample Rate   : 48000
  Block Align   : 2
  Bit Width     : 16
  Bytes/sec     : 96000
data : 932884
smpl : 36
  Manufacturer : 0
  Product      : 0
  Period       : 22675 nsec
  Midi Note    : 60
  Pitch Fract. : 0
  SMPTE Format : 0
  SMPTE Offset : 00:00:00 00
  Loop Count   : 0
  Sampler Data : 0
End

----------------------------------------
Sample Rate : 48000
Frames      : 466442
Channels    : 1
Format      : 0x00010002
Sections    : 1
Seekable    : TRUE
Duration    : 00:00:09.718
Signal Max  : 31045 (-0.47 dB)

//////////////////////////////////////////////////////////////////////////////////////////////////////

File : .\bonjour-hello48_16bit.wav
Length : 937944
RIFF : 937936
WAVE
fmt  : 16
  Format        : 0x1 => WAVE_FORMAT_PCM
  Channels      : 1
  Sample Rate   : 48000
  Block Align   : 2
  Bit Width     : 16
  Bytes/sec     : 96000
data : 937900
End

----------------------------------------
Sample Rate : 48000
Frames      : 468950
Channels    : 1
Format      : 0x00010002
Sections    : 1
Seekable    : TRUE
Duration    : 00:00:09.770
Signal Max  : 31052 (-0.47 dB)