Add support for updating region samples

[iurienistor]

It would be nice if the API will have a RT safe method for updating a region sample (from a data buffer not a file) for a particular velocity range. This will permit to create a real time sample editor and player by using the API.

[swesterfeld]

Generally I think it would be nice to have some support for editing (rather than no longer be able to change stuff after load). However I think we need two different additions here:

1. there should be a way to modify the regions after load; for instance, you may want to change the velocity range, loop points or the sample - I think just being able to change the sample alone doesn't cover all cases. It is a bit tricky because changing regions cannot be done in audio thread; so we would have to modify a region and somehow atomically swap it with the old one - and voices have a region pointer, so we need to wait until all voices are done with some region before deleting it

Now this is just a first idea here:

update_region (region_id, "loop_start=2000 loop_end=3000 sample=trumpet.wav")

This would replace the region with that id by a region that has these three attributes changed.

2. then you want a way to use samples that are provided by the program rather than loaded from a file - this doesn't sound too difficult, maybe like

   void add_virtual_sample (const string& filename, vector<short>& samples, int channels, int rate);
   void delete_virtual_sample (const string& filename);

[iurienistor]

"somehow atomically swap it with the old one",

I think a simplified version for this from audio thread can be used try lock every time the voices are going to be started, if lock is successfull than swap and release. The user will hear the updated version when the next time will press the key (when voices are started).

If I am looking in this SFZ example I see that for the region key=56 there are attached 4 samples, mapped to 0-30, 31-62, 63-94 and 95-127 velocity ranges respectively. What I wanted to do is to be able to synthesize or process at runtime say the second sample and update it, in this case I need a method like this:

void update_region_sample(size_t region_id, const std::vector<float> &samples, int lovel, int hivel, [... maybe other param]);

Loop points, volume, and other region parameters ca be updated by another method if there is a need to be updated. Can be used atomic types for them. I don't know for what is used channels in regard to sample but if there is a need for a reason also to specify the channel that can be given to update_region_sample().

Also, there is need for another method to create empty regions because I'd like to populate them at runtime by synthesizing them or processing some samples.

size_t create_region(int lokey, int hikey, int keycenter);

it returns region_id.

SFZ example used for percussion (GM-StylePerc.sfz from VSCO-Orchestra):

<region>
sample=Cowbell1_Hit_v1_rr1_Sum.wav
lokey=56
hikey=56
pitch_keycenter=56
lovel=0
hivel=30
volume=25

<region>
sample=Cowbell1_Hit_v2_rr1_Sum.wav
lokey=56
hikey=56
pitch_keycenter=56
lovel=31
hivel=62
volume=18

<region>
sample=Cowbell1_Hit_v3_rr1_Sum.wav
lokey=56
hikey=56
pitch_keycenter=56
lovel=63
hivel=94
volume=11

<region>
sample=Cowbell1_Hit_v4_rr1_Sum.wav
lokey=56
hikey=56
pitch_keycenter=56
lovel=95
hivel=127
volume=4

[swesterfeld]

If you look at

https://github.com/swesterfeld/liquidsfz/blob/74eacc59547fa7a838e4befb8b2b800e1a40547e/lib/loader.hh#L131

you'll see that a region doesn't have a sample, but a shared pointer to a sample cache entry. This allows multiple regions to share the same sample without consuming lots of memory. Or sharing sample data between multiple Synth instances. Cache entries are ref counted, so if the last reference goes away, the sample dies. This is why I don't want to see any std::vector<...> in the region update API here, as we don't store a std::vector<...> in the region itself. Note that we have std::vector<short> (not float) now, because the memory consumption of SFZ files can be really high.

So what I think it should look like is:

void add_virtual_sample (const string& filename, vector<short>& samples, int channels, int rate);
void delete_virtual_sample (const string& filename);

By using this API, you can create a sample cache entry, for instance "foo.wav", which you later can load.

As for the region API, I want to avoid any specific opcodes. Some people need a velocity range, others need a key range, even other users need random or other stuff. So I'd rather have a generic string with opcodes.

size_t create_region (const string& contents);
void update_region (size_t region_id, const string& contents);

So for your case, you would use something like.

add_virtual_sample ("foo.wav",...);
id = create_region ("lokey=56 hikey=56 pitch_keycenter=56 sample=foo.wav");
delete_virtual_sample ("foo.wav"); // region holds refcount so this doesn't take effect immediately.

add_virtual_sample ("bar.wav",...);
update_region (id, "sample=bar.wav");
delete_virtual_sample ("bar.wav");

We could make this object oriented like

VirtualSample sample (synth, "foo.wav", ....); // free if VirtualSample goes out of scope
Region region (synth, "lokey=56 hikey=56 pitch_keycenter=56 sample=foo.wav"); // discard if region goes out of scope
...
region.update ("sample=bar.wav");
...

As for try lock, sure this is fine to use. So if a update_region is called (cannot be done in RT thread) a request to update a region is queued. Then in the audio thread, try lock is successful during process(), we can insert the new region into the region vector. However, since some voice may still be playing, at this point the old region (maybe) needs to be kept alive. We could stop the voice, but that will take some time.

So in general, the last ref count for a region could be dropped in the audio thread. So besides a queue of region update requests, we also need a queue of region free requests, so that if a voice releases the last reference count, the region is not freed at this point, but later, once the main thread looks at the region free queue and executes it. Then for instance the region cache entry shared ptr is released and then if the sample is no longer used it gets freed.

[iurienistor]

If the cache sample is shared between regions and plugin instances than for what I wanted to use the library will not work, I thought that the SampleCache is used per instance (which for a real sampler, yes, better to be shared). But these methods you mentioned above would be useful anyway to create the editor.

[swesterfeld]

The global sample cache is shared, but I believe for virtual samples it should not be. So if you have a virtual sample called "foo.wav" in one Synth instance, this shouldn't have any effect on other Synth instances. The way to do it is to add a sample cache to each Synth instance which caches only virtual samples. I've thought everything through now I believe, so I could implement

// init
  vector<short> samples (...);
  VirtualSample sample (synth, "foo.wav", samples, sample_rate, channels);

  synth.load ("some.sfz"); 
  // every sample=foo.wav opcode will be resolved to the virtual sample

// update
  vector<short> new_samples (...);

  sample.set_samples (new_samples);
  // set_samples would mark the cache entry as expired
  // this would cause the RT thread to update the region sample pointers

So this seems to be approximately the original feature request you had. I'm excluding the Region related update code for now as it is a lot of work to get it right. So for now the only way to update a region would be to re-run load(). This could be made load from string so you don't need to write a .sfz file for this. But it is not RT safe, which means that sound will stop if you edit a region.

My priority for implementing this new VirtualSample API depends on whether or not you would actually need it. If you're confident you can use liquidsfz if this is added, then I'd implement it. If you don't need it, then it would not a very high priority item for me.

[iurienistor]

My priority for implementing this new VirtualSample API depends on whether or not you would actually need it. If you're confident you can use liquidsfz if this is added, then I'd implement it. If you don't need it, then it would not a very high priority item for me.

Thank you for considering my request but I think for know you should not make this as a priority (unless if you think these methods fits into the liquidsfz general plan). Geonkick synth I am developing generates what it synthesizes into buffers, than another module plays the samples from the buffers. This module is a simplified sampler. I thought I could replace this module with liquidsfz, thus, to enable the full potential of a sampler the library offers - polyphony, velocity expression etc. For now I don't know exactly if I'll be able to use liquidsfz or not. For example, Geonkick instance are totally separated, even after restart of hosts/save/load states. Also, the buffers when the controls are changed might be updated with a frequency about 30ms, and I don't know how this fits with liquidsfz, and other issues I need to think about. In the version 3.0 of Geonkick I am going to update the player module, and than I'll try to experiment to see if I can use liquidsfz.