Closed mitchmindtree closed 5 years ago
cc @freesig here's a very rough outline of how I'd approach this. Feel free to post comments or call if you get started on this before we hang and anything is unclear of if you get stuck :+1:
asio_backend
branchUse rust-bindgen to generate a module of rust bindings to all items/function is ASIO from the ASIO header file(s). The user guide is super handy. There are two options for approaches here - 1. dynamically generate bindings from header at build time using a build.rs script or 2. generate bindings ahead of time using the bindgen CLI. The rust-bindgen folks recommend approach 1. so I reckon we try that. This will mean adding:
build.rs
script to CPAL's cargo.toml directoryasio/sys.rs
module to which the generated bindings will be written (bindgen docs will explain this).asio/mod.rs
module in which we will write a friendly API for what we need around the raw bindings in the sys
submodule.Keep in mind that users should still be able to use cpal on windows without installing ASIO, so we'll have to make sure to fail "gracefully"/silently if ASIO is not available and provide a runtime check for checking whether or not it is available.
We can now begin implementing the API in CPAL. Tomaka's preference for this seems to be to have an operating-system specific extensions module for each module which will contain all items uniquely availble on that OS. As there are currently none of these modules yet in cpal, we will add them. First, we'll add a pub mod os;
to the cpal crate root and a src/os/
directory and a src/os/mod.rs
for that module. This is where the platform-specific modules will be placed. Next add a pub mod windows;
to the src/os/mod.rs
and add the associated src/os/windows.rs
file. This is where we will provide the user-friendly windows-specific API so that they can access it via cpal::os::windows::some_function()
, etc.
Currently, CPAL is setup so that windows defaults directly to using the wasapi.rs
backend (this is basically the WDM backend that Dante won't work properly with). I think what we want to do is create a new backend in the windows
module which delegates to either the wasapi.rs
backend or the asio.rs
backend, depending on whether or not some initialise_asio
function or something was called. E.g. a CPAL user who wants to choose asio on windows if it's available might do something like this:
extern crate cpal;
fn main() {
// Only generate code for this block if we're targeting windows.
#[cfg(target_os = "windows")]
{
if cpal::os::windows::is_asio_supported() {
cpal::os::windows::initialise_asio().unwrap();
}
}
// All following cpal function calls and items will now use the `asio` module internally now rather than `wasapi`.
let event_loop = cpal::EventLoop::new();
}
So internally, I imagine the windows backend event loop looking something like this:
pub enum EventLoop {
Asio(asio::EventLoop),
Wasapi(wasapi::EventLoop),
}
The EventLoop::new()
function would construct the correct variant based on whether or not initialise_asio
was called or not.
CPAL provides a src/null.rs
module as a template for an unimplemented backend, so probably best to copy this into the windows.rs
and asio/mod.rs
modules to make it easier to get started. From here, it should just be a matter of implementing each of these functions (The ASIO one using the asio/sys.rs
module we generated and the windows one by wrapping the asio/mod.rs
and wasapi.rs
implementations).
Dante Virtual Soundcard is complaining about the license being activated too many times. I can get a trial for developing. Might need @JoshuaBatty to login to the Audinate website and ask them for a to reset it because we need to move it to the NUC. Or just send me the Audinate login / pass and I can sort it.
Also a little update on this. I've finally (with lots of help from #rust irc) got cpal to compile and link with the ASIO SDK. I'm about to test the if I can see any asio drivers. One issue I've come across with bindgen / c++ is a linker error when the Base class virtual destructor gets called. I just stopped calling the destructor but this is not a good solution.
Ok I can see the Dante asio driver from the asio-sys test :) It crashes though because I don't think I'm handling the FFI correctly. I'm also having problems with linking the libraries correctly. But I'll do some reading. At least we know it's possible.
Ok I've got it showing 64 channels. This is in the asio-sys example test. So I might tidy it up and try and implement it for the enumerate example.
Use CPAL examples beep and record to test this works.
Please feel free to add / edit these if my understanding is off:
This last one I understand the least. But I'm guessing all "devices" have to match the interface that cpal is expecting.
Plan to get this done in a little 2 - 4 day sprint starting tomorrow.
Nice one @freesig, sounds great, I'll be working on audio_server non-stop until its done starting tomorrow - feel free to hit me up if you'd like help/advice at any point! Maybe we can do a voice at some point and make a game plan for now until completion :+1:
Hey @mitchmindtree today I tried to fill out a Format struct from ASIO. I attempted to get the sample rate from ASIO but I'm stuck on a link error This is the __banch__ Todays work is mostly in asio-sys/examples/enumerate.rs and asio-sys/lib.rs:get_sample_rate() (A lot of the style is overly explicit and immutable. I will optimize it after. Just wanted to get the design clear first) I think that this might be confusing the bindings:
#if IEEE754_64FLOAT
typedef double ASIOSampleRate;
#else
typedef struct ASIOSampleRate {
char ieee[8];
} ASIOSampleRate;
#endif
It's complaining that it can't link:
ASIOGetSampleRate(info:` *mut ASIOSampleRate) -> ASIOError;
but it links to:
ASIOInit(info: *mut ASIODriverInfo) -> ASIOError;
just fine.
They live in the same file and both are in the bindings.
My guess is that bindgen doesn't handle the preprocessor:
#if IEEE754_64FLOAT
We should definitely have a voice chat tomorrow afternoon (my time, your morning I think) because I'm not that sure on how to handle this os::windows mod etc. Right now I'm just writing examples inside asio-sys that mimic the cpal examples.
Ok got passed the link error. Not sure what to set Format.SampleFormat to because it only has I16, U16 and F32 but I think for this computer its F64. Anyway @mitchmindtree we should have a voice chat when you're free just to go over how the os::windows structure works. I think I've got all the ASIO backend to satisfy enumerate, so we could start with trying to get cpal run --example enumerate to work with asio
Ok so now:
Question
Right now the backend is static backend: Backend = Backend::Asio
But how can we set this depending on whether or not the user wants to use asio?
Given static mut is unsafe.
Nice work!!
But how can we set this depending on whether or not the user wants to use asio?
The safest way to do ths would be to use the lazy_static crate - it allows you to lazily initialize statics. Btw, the reason why mutatinng a static mut
is unsafe is that it is accessible and mutable from any thread. If we wrap it in a lazy_static Arc<Mutex<>>
it should be fine. Alternatively we could use an Arc<AtomicBool>
to check, but I think your Backend
enum is probably nicer and more future proof :+1:
Edit: Actually the Arc
is probably not needed.
Yeh I was thinking some sort of lazy init would be what we needed and that lib looks like what we need
Ok really basic version of enumerate is working for asio now. I skipped the search all supported formats for now because its a bit of time and want to get something working with just the default format. Speaking of default, ASIO has no concept of this so I just return the first in the list. Also the data_type in Format isn't right but I will get that going asap.
So today I had a go at beep. I'm able to create the buffers. This is how asio handles it:
Need a way of storing a handle to the stream. I think this will live in EventLoop but I'm not completely sure how this will work.
Ok so I've hit a bit of a wall with the "Running" part.
void (*bufferSwitch) (long doubleBufferIndex, ASIOBool directProcess);
Is called by asio to let you know that it's time to swap and fill the other buffer.How can I communicate from this callback to anything in in cpal.
I'm thinking of going the global variable option but I just wanted some advice before going down that road.
One of the functions in the struct: void (*bufferSwitch) (long doubleBufferIndex, ASIOBool directProcess); Is called by asio to let you know that it's time to swap and fill the other buffer.
By this, do you mean that this callback is the function in which the user is expected to fill the next buffer? If not, what exactly is the user expected to do when this function is called? Just want to clarify this before adding further advice
Yeh exactly. So the doubleBufferIndex is either 0 or 1 and that is set to the index of the buffer you should fill now.
OK sweet, yeah I think the best bet for this would be to use a closure and do something similar to what is going on in the coreaudio backend. It may be necessary to Box
this closure in order to pass it to the ASIO backend without requiring adding type parameters to it. I think the reason why the closure is wrapped in a struct on the other hand is so we can easily cast the "data" void pointer provided by the C callback back to that struct type and then call the inner closure after that is done. Definitely a bit of a tricky task, but should be doable - wanna have a voice chat about this? I'll jump on messenger.
I think I'm getting close to getting example beep to run on asio but I'm stuck on one last problem:
pub fn run<F>(&self, mut callback: F) -> !
where F: FnMut(StreamId, StreamData) + Send
Problem: Box needs to know the lifetime of the closure to make sure the captured data lives long enough.
In the coreaudio version they got around in by doing this:
let callback: &mut (FnMut(StreamId, StreamData) + Send) = &mut callback;
self.active_callbacks
.callbacks
.lock()
.unwrap()
.push(unsafe { mem::transmute(callback) });
Which stores into this:
callbacks: Mutex<Vec<&'static mut (FnMut(StreamId, StreamData) + Send)>>
Seems very unsafe to transmute the callback. Is there another way to get around this lifetime check? _All the other implementations don't store the call back but this would not be ideal for ASIO because of the way that it needs to interact with the callbacks in a global way.
So, the Send
and Sync
traits are two "built-in" traits. These are special in the sense that they're automatically implemented for all types that can implement them. These along with ownership are the reason rust can have such thorough guarantees about thread safety.
Send
is a trait that is auto-implemented for types that can be safely moved between threads. If the language itself can't determine whether or not it is safe (e.g. if it contains a reference bound to a life-time on the current thread's stack, or if it is a raw mutable pointer, etc), the user must manually and unsafely implement the trait themselves, but only if the type and its exposed API meet the trait's expected behaviour of being safe to send across threads.
Sync
is a trait that is auto implemented for types that can be safely shared between threads. Arc
for example requires that the type it is pointing to is Sync
, otherwise a compile-time error might cocur. An easy way to determine whether a type T
is Sync
is if &T
is Send
.
The std docs should also have a lot of info on these.
So the main reason the box needs to know about the lifetime of the closure is because it is being sent from the current thread to the audio thread. This means the lifetime of the given callback must be static as there is no other way to guarantee that the lifetime of the callback is at least as long as its use on the audio thread. In this case, the Box
type we want should be something like Box<FnMut(StreamId, StreamData) + Send + 'static>
- the key being the 'static
lifetime. Unfortunately the CPAL API itself doesn't require this lifetime yet, and I don't think it's been discussed in the issues yet. So by the looks of it, the way the coreaudio backend works around this is by just "assuming" that the lifetime of the given callback is 'static
and forcing the compiler to accept this using unsafe - definitely super unsafe and probably needs some review.
Will talk to ya some more in person soon :+1:
The beep example is now working smoothly. It sounds correct.
For efficient processing and great flexibility ASIO implements channels (input and output) as circular buffers with blocks of data. Actually a double buffer mechanism is used, which can be easily adapted to a great number of different buffer implementations. One buffer contains always data for one single channel only. This approach allows the host application to implement additional processing algorithms very effectively (opposed to an interleaved buffer model). The size of the data blocks is ascertained from the audio driver, to allow the best support for the audio hardware.
My interpretation is that you get a buffer and then that is for say channel 1. Then it swaps and the next buffer is for channel 2. Then it swaps and the next buffer is for channel 3.
There is a ASIOGetSamplePos()
call that gets you some sample position. I think this might be how you know which channel you are sending to but I'm not sure.
Theres probably a reason that portaudio-asio is 3000 lines but I found this great file outlining how they did it HERE I have the next 3 days blocked out to work on this. It's going to be a long project before its very stable and generic so I think we should really focus this on just what is needed for the audio server.
actually on second thought it looks like the buffer pointers are per channel. Which is odd because I'm only using one but I'm getting stereo. This might just be luck because they are next to each other in memory. Gotta love unsafe land
As a fallback, I’ll have a Mac mini on standby.
Out of curiosity, I attempted to compile on a Linux machine (up-to date archlinux) and got a segfault from pulseaudio.
This is obviously in no way something you or Mitch need to look into, but something interesting to look out for in the future.
On Fri, 20 Apr 2018 at 00:42, Tom notifications@github.com wrote:
actually on second thought it's possible that each buffer contains enough memory for all the channels
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mitchmindtree/beyond_perception_audio_server/issues/52#issuecomment-382762483, or mute the thread https://github.com/notifications/unsubscribe-auth/AIPI7RFH9MDgBCwltDdA9koBPko48jVzks5tqKI6gaJpZM4SGVwT .
@BlueJayLouche I think the main issue on linux is that by default PulseAudio takes exclusive access to the audio hardware and CPAL (the cross platform audio backend we're using) currently only targets ALSA directly. As a result if pulseaudio is running, you eventually run into some contention or a data race which can cause a segfault or some other weird undefined behaviour. I think there are a couple workarounds for this:
Just thought I'd add this for the record in case someone comes across this in the future!
@freesig Awesome work on getting the CPAL ASIO beep going and finding that portaudio reference doc, looks super useful!
De-interleaving
This is isn't too uncommon to have to interleave or deinterleave buffers at different parts of the signal chain, and will rarely show up as a bottleneck (especially if we only have to do it once ASIO -> CPAL or vice versa). My understanding is that non-interleaved channels can be more efficient for some purposes and interleaved can be more efficient for others. It would definitely be useful if CPAL actually just served the original buffers whether they are interleaved or not, maybe in some enum.
Endianness
We can possibly use the byteorder
crate for this, but it's hard to know without seeing the implementation just yet.
For some reason Dante produces a device for every two channels on Windows' WDM API. This is an issue as the audio server expects to be able to target a single device - targeting multiple devices would require a significant hack and be a special condition only required on windows.
The easiest way forward is probably to add ASIO support to CPAL.
Tomaka has mentioned that he's happy to see ASIO support added and that it should follow a platform-specific extensions API similar to the way winit and glutin do. See his comment here.
cc @freesig maybe we can work out some day after Monday next week (when I get back) to have a sesh and work on this together :+1: