Decouple Overwitch from the JACK client

[szszoke]

I'm considering forking your project and turning it into a recording/playback application for Android.

As far as I can tell, everything should work on Android, except for Jack (which is not really an issue).

I'm also thinking that before an Android port becomes possible, I'd have to implement a simple recording mode, where no Jack/PipeWire code is used.

As a start I would likely follow the structure of the current Jack implementation, but instead of sending the samples to Jack, I'd write them to disk as wav files.

When that is done, I could start testing on an Android device and eventually build a UI.

Whichever way the fork would go, I see a lot of potential for cooperating.

Maybe overwitch could be split into liboverwitch, overwitch-cli and overwitch-gui.

An eventual Android port could then use liboverwitch too, but that would require liboverwitch to be decoupled from Jack.

Another option is a complete fresh start, where I would study Overwitch and reimplement bits that are relevant for an Android port.

I'm just bouncing ideas at this point and I thought it would inquiry about future plans for the project.

[dagargo]

I really like the idea.

Whichever way the fork would go, I see a lot of potential for cooperating.

Sure.

Maybe overwitch could be split into liboverwitch, overwitch-cli and overwitch-gui. An eventual Android port could then use liboverwitch too, but that would require liboverwitch to be decoupled from Jack.

This would take some work and would make more difficult to build Overwitch. Also, I'd want to take a proper look at the source code.

However, before we start, do you know that Overwitch is based on dtdump? This project records on disk but it only supports Digitakt and Digitone I think. I mention this because of the queue code it uses to stream to disk.

At the moment, I can only say that the library API would be quite similar to overbridge.h.

Let me take a deeper look and I'll come back to you in a few days.

WRT the interface, it would be something very simple to avoid using the terminal.

[szszoke]

Then maybe I should start from dtdump instead, especially if splitting up overwitch into a library and client apps would make your life harder.

I could also just start on my own and then there wouldn't be extra workload on you.

Eventually we could see if it makes sense to consolidate some of the shared code into a library.

[dagargo]

I'm gonna stop working on the GUI for now and focus on this.

It's not gonna be very difficult since the code is all written. However, there are a few things to do first like changing the internal dependencies and improve the API.

In the long term it will be best if everything has the same code base. Moreover, Overwitch is more mature and has been tested enough.

Let me take a look and I'll be back in a few days. In fact, I'll try later today but I can't promise anything.

[szszoke]

When you said you will stop working on the GUI, and start working on this, what exactly did you mean?

Android port? Refactor to eventually isolate overwitch from Jack and have it as a library?

There are also "logistical" questions to sort out.

I would most like use C++ for the client that would run on Android.

This is partly because I'm more comfortable writing C++ code than C, and partly because there have been some interesting additions to the language which I want to try out. Depending your future plans, this might be irrelevant for Overwitch.

[dagargo]

My plan is to put aside the GUI for now and refactor the code to isolate the overbridge part from Jack and create liboverwitch. I think this is the best option for both of us.

FYI, I'm not gonna develop anything for Android. The GUI is only for the desktop.

I've already started with the refactoring and I think it's gonna be easier than I initially thought yet it still requires some work. In the following days I'll create also a new remote branch for overwitch. A quite an adventurous estimation would be a week. Once it is stable, we can see how we can improve the API for you.

Let me know your thoughts on this.

[szszoke]

This sounds really promising! I'd expect that once I start building the Android client I would see if there are any pain points with the API.

If you end up using the library for your GUI app and the CLI then I think you will also notice if something needs to be changed.

One thing that I can already tell you is that apps don't have direct access to USB devices the same way you do on Linux.

The Android app will have to ask for permission to access the USB device and once that is granted, you can get a file descriptor that is passed to libusb.

If you can take this into consideration then it's good but considering that I don't know exactly what is needed, maybe it is better if you ignore this for now and we add it later, once there is a working Android prototype.

We will both be working on this in our free time, which means an irregular schedule. I'm expecting that I will have to keep a fork of liboverwirch anyway and open PRs once my changes can be considered final.

[dagargo]

Just a small follow-up on the discussion.

I've almost completed the decoupling. Still, there is not a proper API but I'm on it.

[szszoke]

That sounds good!

[dagargo]

The API is almost ready.

There are 2 public structs and 2 different sets of functions.

• One allows to use only Overwitch (now called ow_engine) to get the data, which is useful for batch processing.
• The other lets us use ow_engine thru a resampler (called ow_resampler) to read and write data in sync with an audio server.

In the next few days, I'll publish the code in another branch of this project to discuss the changes, possible improvements and the like.

[szszoke]

Thanks for the update!

I'm excited to see where this will lead!

I'm assuming that eventually even your client applications will use the library that you are writing now - correct?

[dagargo]

Thanks for the update! I'm excited to see where this will lead!

You're welcome. I'm excited too!

I'm assuming that eventually even your client applications will use the library that you are writing now - correct?

Sure, that's the idea.

[dagargo]

Code has been directly merged into the master branch.

There are two internal API's.

1. The one that directly interfaces the Overbridge devices, called engine.
2. The one that provides access to it through a resampler, called resampler. This way, it can be used to synchronize two sample rates and making the Overbridge device sample rate the secondary one. Hence the renaming to j2o to p2o and o2j to o2p (primary to Overwitch and the like).

Additionally a program called overwitch-dump can be used to directly record to disk bypassing JACK entirely. This serves as an example for the first case API while overwitch does it for the second case.

Let me all you know your thoughts on this.

I'm leaving this open for know.

[szszoke]

Thank you! I will try to build it for Android and see how that goes.

[dagargo]

Good luck with that!

[dagargo]

I forgot to mention that we could split the project into a library and binaries. For now, I'm leaving it like this but we could change this in the future.

[szszoke]

I spent some time working on the Android port. It kind of works as a proof of concept but there are pain points.

Overwitch does device enumeration by itself, which is not directly possible on Android. One must ask for permission using the Android Framework and only then can libusb do anything with the USB device.

After the user granted permission for the app to use the USB device, I can obtain the USB file descriptor and pass it to the native code, in the end I am left with a libusb_device_handle pointer.

Example:

extern "C" JNIEXPORT void JNICALL
Java_com_szszoke_overwitch_Dump_openDevice(
        JNIEnv* env,
        jobject /* this */,
        jint descriptor) {
    libusb_context *ctx = nullptr;
    libusb_device_handle *devh;
    libusb_set_option(ctx, LIBUSB_OPTION_NO_DEVICE_DISCOVERY, nullptr);
    libusb_init(&ctx);
    libusb_wrap_sys_device(nullptr, (intptr_t)descriptor, &devh);

    // TODO: use devh
}

I ended up adding a new initializer function called ow_engine_init_from_libusb_device_descriptor. This skips all the device enumeration and just takes a file descriptor. The initialization seems to finish without any reported error.

Here is the beginning of the new function, the rest is unchanged:

ow_err_t
ow_engine_init_from_libusb_device_descriptor (struct ow_engine **engine_,
                int libusb_device_descriptor, int blocks_per_transfer)
{
    int err;
    ow_err_t ret = OW_OK;
    struct ow_engine_usb_blk *blk;
    struct libusb_device_descriptor desc;
    struct ow_engine *engine = malloc (sizeof (struct ow_engine));

    libusb_device_handle *device_handle;
    libusb_set_option(&engine->usb.context, LIBUSB_OPTION_NO_DEVICE_DISCOVERY, NULL);

    if (libusb_init (&engine->usb.context) != LIBUSB_SUCCESS)
    {
        ret = OW_USB_ERROR_LIBUSB_INIT_FAILED;
        goto end;
    }

    libusb_wrap_sys_device(NULL, (intptr_t)libusb_device_descriptor, &device_handle);

    engine->usb.device_handle = device_handle;
    struct libusb_device *device = libusb_get_device(device_handle);
    libusb_get_device_descriptor(device, &desc);
    ow_get_device_desc_from_vid_pid
            (desc.idVendor, desc.idProduct, &engine->device_desc);

    if (!engine->usb.device_handle)
    {
        ret = OW_USB_ERROR_CANT_FIND_DEV;
        goto end;
    }

    ...
}

One issue is that sometimes I get SIGBUS when I call libusb_set_option.

The recorded audio file also has some pops in it.

[szszoke]

A compromise to not make the API too verbose for non-Android targets could be to split the device enumeration and the engine initialization logic and make the initialization logic accept a libusb_device_descriptor, which would come either from the enumeration step or from a helper that ties into the Android framework.

The Android specific code doesn't even have to live in this library. As long as the device enumeration logic is not hard-coded into the Overwitch engine, Android specific logic can live as part of the Android client.

[szszoke]

One more thing, in order for this to work well, it would be nice if the memory allocation to the ow_engine struct would happen outside of ow_engine_init. The reason for this is that in case of the Android client, libusb_set_option needs to be called on the engine->usb.context before libusb_init is called.

Not having the memory allocation encapsulated within the ow_init function would make this easy to do.

[dagargo]

The recorded audio file also has some pops in it.

I've added a track buffer length parameter and decreased the wake-up time of the writer thread.

Let me know if this helps.

[szszoke]

Thanks! I will try it out this weekend.

[dagargo]

I think I may have solved your initialization issue in 7864f51cbfdc0144b08b45daf813b5253218396b.

However, in libusb commit 5c89594f64ed5a14470d9965e558fd9aee1fd42c, they recommend to use LIBUSB_OPTION_WEAK_AUTHORITY instead of LIBUSB_OPTION_NO_DEVICE_DISCOVERY for now as these are both aliases for the same option as the latter is not on the Debian stable version yet.

Let me know if this works for you.

We still need to address the memory allocation but let's focus on the API for now.

[szszoke]

It seems to work (not SIGSEGV), with some minor changes, there are other issues now however.

Here are some logs:

Creating sample (12 channels)...
Starting audio and o2p MIDI thread...
Writing 0 frames to disk...
Done
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 8064, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 16128, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 24192, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 32256, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 40320, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 48384, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 56448, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 64512, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 72576, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 80640, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 88704, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 96768, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 104832, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 112896, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 120960, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 129024, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 137088, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 145152, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 153216, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 161280, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 169344, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 177408, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 185472, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 193536, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 201600, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 209664, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 217728, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 225792, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 233856, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 241920, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 249984, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 258048, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 266112, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 274176, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 282240, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 290304, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 298368, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 306432, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 314496, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 322560, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 330624, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 338688, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 346752, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 354816, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 362880, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 370944, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 379008, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 387072, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 395136, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 403200, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 411264, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 419328, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 427392, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 435456, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 443520, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 451584, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 459648, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 467712, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 475776, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 483840, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 491904, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 499968, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 508032, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 516096, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 524160, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 532224, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 540288, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 548352, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 556416, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 564480, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 572544, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 580608, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 588672, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 596736, buffer.len = 3072000
j2o: Clearing buffer and stopping...
Writing 8064 bytes (168 frames) to buffer...
write_buffer, new_pos = 604800, buffer.len = 3072000
j2o: Clearing buffer and stopping...
o2j: Error on USB audio transfer: Other error
j2o: Error on USB audio transfer: Other error
j2o: Clearing buffer and stopping...

Have you encountered this pattern of errors before?

[dagargo]

There was a race condition here

Writing 0 frames to disk... Done

Fixed in 27df22f65f6dbd60469f33cdfc7d6115d64ca267. Check it out but I'm not sure that was your issue.

[szszoke]

There was a race condition here

Writing 0 frames to disk... Done

Fixed in 27df22f65f6dbd60469f33cdfc7d6115d64ca267. Check it out but I'm not sure that was your issue.

Thanks! I will give it a try in a few hours.

[szszoke]

One more thing. I want to find out if the issues that I'm encountering are Android or ARM issues.

I will try to compile overwitch-dump on a Raspberry PI and see if it works.

[dagargo]

Compiling works and it throws no errors on an old Raspberry Pi B Rev 2 (256 MiB). But it is too slow to cope with all the USB bandwidth and SD writing so a lot of audio frames are lost.

[szszoke]

What does O2P and P2O mean?

[dagargo]

In the begining, I used o2j to indicate the direction of the data, which is Overbridge to JACK in this case. Of course, JACK to Overbridge was j2o.

But when I started to work on this issue that wasn't making sense any more so I change from JACK to primary. Probably confusing but I didnt come out with a better idea.

I'm open to change it. Any suggestion?

[szszoke]

How about INPUT and OUTPUT?

[szszoke]

Compiling works and it throws no errors on an old Raspberry Pi B Rev 2 (256 MiB). But it is too slow to cope with all the USB bandwidth and SD writing so a lot of audio frames are lost.

I tried with a Raspberry PI 4 with 4GB of RAM and it also compiled and run fine. The audio was not perfect but no issues like on Android.

Which leads me to believe that the issues are caused by Android only.

[szszoke]

There was a race condition here

Writing 0 frames to disk... Done

Fixed in 27df22f. Check it out but I'm not sure that was your issue.

Sadly it was not. I am still getting the same issue.

[szszoke]

I compared the output of lsusb -v with the output of a USB info app and they seem to match except at the very end.

I get this from lsusb:

Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        4
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      1 Control Device
      bInterfaceProtocol      0 
      iInterface              0 
      AudioControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdADC               1.00
        wTotalLength       0x0009
        bInCollection           1
        baInterfaceNr(0)        5

And this is from the USB info app:

[dagargo]

How about INPUT and OUTPUT?

IMO, more confusing than o2p. Anyway...

I tried with a Raspberry PI 4 with 4GB of RAM and it also compiled and run fine. The audio was not perfect but no issues like on Android.

Nice. Remember to set everything up to use RT priority. Perhaps, it'll help.

Which leads me to believe that the issues are caused by Android only.

Let's assume for now it's Android related. Perhaps the issue is in the new function that I made to use the descriptor. I haven't tested it but I'll do it tomorrow.

[szszoke]

IMO, more confusing than o2p. Anyway...

The naming scheme that is used today would be fine with some comments for the enum fields.

Let's assume for now it's Android related. Perhaps the issue is in the new function that I made to use the descriptor. I haven't tested it but I'll do it tomorrow.

Just to make sure that I am not going crazy, I reverted back to the the version that I tested earlier and the issue still came up.

[szszoke]

I did really quick and dirty check with ow_engine_init_from_libusb_device_descriptor instead of ow_engine_init_from_bus_address and it seems to work fine.

#include <fcntl.h>

...

static int
run_dump (int device_num, const char *device_name)
{
  ...
  int file_desc = open("/dev/bus/usb/001/004", O_RDWR);

  err =
    ow_engine_init_from_libusb_device_descriptor (&engine, file_desc,
                     DEFAULT_BLOCKS);
  ...
}

[dagargo]

Any progress on the android issues?

Also, I remember that it's possible to reduce the resampling quality and that might help with the dropouts. Let me know if this improves anything if you're not using this already.

[szszoke]

No update. I won't have time to work on this until the weekend.

[szszoke]

Small update regarding my debugging method: I will try to run overwitch-dump directly on a rooted Android installation. I will likely use my Raspberry PI 4 to do this.

This should make debugging easier and hopefully I will get the same errors.

At this point I cannot actually say for sure that the errors are caused by my phone or by some libusb incompatibility so hopefully a different testing device would answer the question.

[szszoke]

Second update: I managed an old Samsung Galaxy J5 (2016) which is running Android 7.1.1 and I will try to use that before I try to install Android on a Raspberry PI 4.

[szszoke]

I am not getting the USB transfer errors on the old phone but I do get unusable audio, most likely due to the phone's low performance.

That means I will have to do more testing with a third Android phone, one that is closer in specs to mine but is a different brand.

[szszoke]

I wonder if we could crowdsource some debugging. If I could create a somewhat more robust and more user-friendly version of the app, we could ask people over Elektronauts to test it and upload logs.

[dagargo]

I am not getting the USB transfer errors on the old phone but I do get unusable audio, most likely due to the phone's low performance.

Great news regarding the errors.

As resampling quality is not an issue here, have you tried with a bigger buffer? Even if there are dropouts, there should be less with larger buffers.

I wonder if we could crowdsource some debugging. If I could create a somewhat more robust and more user-friendly version of the app, we could ask people over Elektronauts to test it and upload logs.

Sounds good to me. However, I'd like to release version 0.3 before as the internal API is somewhat finished and the dumping utility is ready.

Perhaps we could close this issue, release version 0.3 and create an issue for extracting the library to its own project. Then we can improve it based on your needs. Also, this way I can start with version 1.0, which will have another binary for the

If you have a very simple POC I think I could try it on a high spec but old tablet I have around.

Let me know what you think.

[szszoke]

I borrowed my girlfriend's Samsung Galaxy S something new. It runs Android 12 and has great specs. Still no errors but some dropouts.

I will try to play with the buffer size but other than that, there is one more change that you should make for Android and then I think you can close this issue:

In ow_engine_init_from_libusb_device_descriptor when you call libusb_set_option, pass NULL as context instead of engine->usb.context.

From what I could gather, there was some issue or regression in libusb which caused LIBUSB_OPTION_WEAK_AUTHORITY to not work if you try to set it for a specific context.

[szszoke]

I have to work out the mess that is native dependencies on Android before I can release a PoC. I want to release something that other people can build themselves if they want to.

[szszoke]

One more thing: looks like it is not possible to get real-time priority for a native thread on Android. How important is that for overwitch-dump?

[dagargo]

I'll take a look at the other things tomorrow but RT priority is very important. Without that, my PC has dropouts. But a phone has very different requirements than a music PC.

Also, it very important not to schedule Overwitch on cores without a proper FPU.

Finally, perhaps core affinity is something that could help with cache failures but haven't tried it yet. Let me do some research on this before closing this issue.

[szszoke]

I will also do some research and see if there is a way to set a higher priority for the overwitch thread.

[dagargo]

In ow_engine_init_from_libusb_device_descriptor when you call libusb_set_option, pass NULL as context instead of engine->usb.context.

Done in 704500177b8dc87aa87f8e74daf0cbc5169022a6.

OTOH, I have decided to not use any CPU affinity functionality. The reason for this is that I consider that this fine tuning should go altogether with IRQ tuning and possibly with CPU scheduling isolation, which is very computer specific and it can still done with command utilities such as taskset.

[dagargo]

FYI, I've pushed some commits to improve the GUI and a few little changes have been applied to the API.

Hopefully, it won't be a problem for you. But if it is, don't hesitate to ask.

[dagargo]

This change will definitely affect you.

Latencies are now calculated in the engine so it means that both functions for getting the p2o and o2p used spaces are needed.

See 1a8c13d5b54d38b84fd4709c94d739424a581b61 for the exact changes you'll need to apply.

This probably will change in the future as the API is not 100% ready yet.