Add a simple GUI

[allo-]

Add a GUI that reads the config file and writes changed values to it. When the program is running at the same time it then reloads the config, just as it does when you change it using a text editor.

Something simple would work. When matplotlib still was part of the (previous) project, I thought about matplotlib Sliders. Now it may be better to look for a real toolkit like PyQt or something similar.

Features that would be useful:

• Sliders for values like blur
• File selector for choosing the background image
• Possibly some function to add items from a list to another list, e.g. to add filters from the list of available filters to a property like background_filters (see #1).

[rriemann]

https://www.qt.io/qt-for-python would be an easy choice for Linux users who have the qt libs easily available.

[schwab]

Here's one that works for me. I use in a jupyter notebook... quick and dirty, but since it updates the yaml file, I can monitor the video stream in another program and tweak in real time.

from __future__ import print_function
from ipywidgets import interact, interactive, fixed, interact_manual
import ipywidgets as widgets
import yaml
with open(r'config.yaml') as file:
    # The FullLoader parameter handles the conversion from YAML
    # scalar values to Python the dictionary format
    config = yaml.load(file, Loader=yaml.FullLoader)

print(config)

@interact(x=widgets.FloatSlider(min=0.0,max=1.0,value=config["segmentation_threshold"], step=0.1, description="segmentation_theshold", continuous_update=False))
def st(x):
    config['segmentation_threshold'] = x
    with open(r'config.yaml', 'w') as file:
        documents = yaml.dump(config, file)
    return x

@interact(x=widgets.IntSlider(min=0, max=100, value=config["blur"], description="blur", continuous_update=False))
def blur(x):
    config['blur'] = x
    with open(r'config.yaml', 'w') as file:
        documents = yaml.dump(config, file)
    return x

@interact(x=widgets.IntSlider(min=0, max=100, value=config["erode"], description="erode", continuous_update=False))
def erode(x):
    config['erode'] = x
    with open(r'config.yaml', 'w') as file:
        documents = yaml.dump(config, file)
    return x

@interact(x=widgets.IntSlider(min=0, max=100, value=config["dilate"], description="dilate", continuous_update=False))
def dilate(x):
    config['dilate'] = x
    with open(r'config.yaml', 'w') as file:
        documents = yaml.dump(config, file)
    return x

The result is a slider for each value that saves upon change. Note: continuos_update=False is required if you use this on other controls to prevent some spurious saves that cause the virtual_cam code to crash.

[mrxz]

I've created a basic UI using PyQt5. It's only concerned with the layers and their filters, but does support most of them (affine isn't supported). All changes are applied in real-time. The processing can be paused to save some CPU, and the program remains running in the background when the window is closed (there's a tray icon for it).

At this point it mostly serves as a PoC, as the code isn't the nicest (was my first time using PyQt5). There's also a ton of small UX things that could be improved; being able to reorder layers and filters is one of them. It can be cleaned up and turned into a PR, but it all depends if this is the direction the GUI should go. Some considerations:

1. The GUI could be separate from the main process and only communicate via the config file. This would make some parts easier and cleaner, but does sacrifice real-time updates. Reason I opted for an integrated approach is that I could let the filters tell which properties they had in a data-driven way. A new filter only has to specify it's properties and it is supported.
2. Some filters have conditional behaviour that is hard to model in the GUI in a data-driven way. A clear example is the move filter, which has a horizontal and vertical property which range from 0-1 when in "relative" mode, but is in pixels in "absolute" mode. To combat this filters should be made with the GUI in mind (:disappointed:), for example splitting move into relative_move and absolute_move. Alternatively the GUI could be tailor-made instead of data-driven, opening the possibility to have all sorts of convenience features (e.g colour picker instead of separate RGB sliders or visually configuring an affine transformation), but will require work for each (new) filter type.
3. It wouldn't hurt to translate the loaded YAML into an internal representation of the layers and filters and operate on that representation. It could iron out small annoyances (like the layer being a dict with a single key representing the layer type). On the other hand, an inverse translation would then be needed in order to save the result back.
4. Perhaps the GUI is focussing too much on the details? Having these graphical controls is nice when tinkering with this project, but the focus could also be on ease of use. The UI could present several "presets" and offer some customization per "preset". The internals of the layers and filters would be abstracted away from the user. This would allow anyone looking for a simple "background blur" or "background replacement" get what they are looking for.

Anyway, to try it out, checkout the gui branch (https://github.com/mrxz/virtual_webcam_background/tree/gui). Make sure PyQt5 is installed and run window.py directly. Feedback is welcome :-)

[allo-]

First: This looks awesome! I really like it.

1. I thought of having a fancy config editor, but maybe a more integrated approach is feasible. When I started things were a lot more static than now. For example blur for the mask is definitely a slider. But for a general UI there is no clear definition of the parameters and ranges for a filter. This would probably need some scheme to be specified or the filter to output some structure with information about its parameters. In another project I use this approach.
2. Splitting the move filter would be no problem, but there are a lot more complicated filters possible. For example I think of having different overlay filters like the one in the halo branch. Structures could become more complicated, but on the other hand parameters like for the affine filter are not user friendly in a text editor either. I think it would be useful to restrict the parameters to flat structures and when some filter would be really more complicated it can load additional parameters from a file or have a string parameter that can contain YAML data or something similar.
3. I am not sure what you want to do.
4. I think it would be reasonable to allow a set of controls like number (without parameters as an text input with range parameters a slider), string, filename.

What do you think about a serialization of the parameters? A filter could return something like

{
    "overlay_image": {"type": "filename"}
    "speed_x": {"type": "number", "range": [0, 100], "input": true, "default": 10},
    "speed_y": {"type": "number", "range": [0, 100], "input": false, "default": 10}
}

The UI would call filter.config() to get this description and then show a file chooser and two sliders from 0 to 100 and for the first one a text input that allows to enter numbers outside of the range.

[allo-]

I see that you already use a python format to specify the parameters.

I think it should be a class method and probably use a dictionary for more clarity what a parameter to the UI control element means.

[allo-]

An idea about

# Ugly hack to get nice termination behaviour :-/
    signal.signal(signal.SIGINT, lambda *a: exit(app, thread))

Did you try the atexit module?

The other option is doing the same as the main program and catching KeyboardInterrupt in __main__.

[allo-]

Another thought:

A function that returns types for the parameters could allow the program itself to do consistency checks, like if a file in a filename parameter exists.

The definition should also include the default value, but to avoid duplicating it (what could mean two different values when it is only changed in one place) the constructor could/should read the default value from the parameter function (or dict).

For the UI I am thinking it is probably the best to couple it loosely. The UI program can import the filters to read their properties, but the webcam program should not need to import the UI, so using the config file as interface is probably the best to minimize the code that needs to be adapted when something in the program structure is changed.

[mrxz]

I've made some changes; the parameter config is now a class method on the filters themselves and is nicely structured, as you've suggested. It's way nicer to read and write and allows for easier expansion, especially for optional parameters (as it's no longer index based).

I've also added input fields for numerical values. I haven't added support for a slider with an inputbox next to it (probably a good idea to just have it for all sliders, why not). But those kinds of UI/UX improvements are probably best for later, once the base of the GUI is in place.

I think it would be useful to restrict the parameters to flat structures and when some filter would be really more complicated it can load additional parameters from a file or have a string parameter that can contain YAML data or something similar.

I think that would indeed be the best approach. In the end, if complex filters do require complex/structured parameters, dedicated GUI components could always be created for them. But whenever possible, keeping it flat and simple is beneficial. In general I prefer a GUI to prevent the user from creating invalid configurations, which you can't really do if users need to point to external YAML files or type YAML by hand in an input field.

3. I am not sure what you want to do.

Conceptually I consider there to be three representations of the layer and filter configuration:

• Persistence representation: YAML structure in the config.yaml file
• Internal representation: Structure held in memory
• Runtime representation: The actual instantiated filters

As it currently stands, the internal representation isn't well-defined and is basically the direct output of parsing the YAML. In the get_filters method various notations are supported, but the GUI I've created assumes a specific one. Making the GUI "aware" of these notations would tightly couple to GUI to the persistence, meaning that would it ever change, the GUI needs to change as well. IF there would be a clear internal representation, this wouldn't be a problem. The GUI knows what to expect and the conversion of the various notations would belong to the reading/parsing of the config file.

Did you try the atexit module?

Thanks for the suggestion, but I'm afraid it will have the same problem. The hack isn't so much the signal handler, but more the timer that runs every 200ms and does nothing. Without it, a SIGINT to the application wouldn't result in termination unless a QT component gets focus (so either the window or the tray icon). So a quick Ctrl+C in the terminal wouldn't suffice and I had to also focus the window. The timer just ensures there's an opportunity every 200ms to handle it. This has something to do with how PyQt5 works.

A function that returns types for the parameters could allow the program itself to do consistency checks, like if a file in a filename parameter exists.

It would indeed be possible to perform such checks without instantiating the filter. Depending on the outcome it could either reject the config or replace all invalid filters (or layers) with dummy/no-op alternatives and log a warning.

Do keep in mind, though, that the validation it can perform is surface level at best, especially for files. While it can check for existence, it won't know if it meets the needs of the filter (file might not be an image or video or 0 bytes in size, etc...).

The definition should also include the default value, but to avoid duplicating it (what could mean two different values when it is only changed in one place) the constructor could/should read the default value from the parameter function (or dict).

Totally agree, avoiding duplication in this case would be nice. Since the definitions will be available for all filters, the handling of the default values can be moved outside the constructor.

For the UI I am thinking it is probably the best to couple it loosely. The UI program can import the filters to read their properties, but the webcam program should not need to import the UI, so using the config file as interface is probably the best to minimize the code that needs to be adapted when something in the program structure is changed.

While I do agree the coupling should be minimal and the core of the program should not depend on the GUI. Having the GUI depend on the filters and some internal parts isn't the end of the world. It's all a matter of defining what the "interface" is. On one hand the config is probably a pretty stable one. But if, as I also mentioned above, the internal representation could be stable interface for the GUI to interact. Personally I think interfacing through the config isn't ideal, but does yield the biggest separation/loose coupling.

[allo-]

The class method looks good.

I think type double vs. numeric isn't consistent yet. I am not sure if the parameters should communicate the required type ("integer") or the required type of input ("numeric"). And think python is calling it float everywhere even when it probably is a double value.

As it currently stands, the internal representation isn't well-defined and is basically the direct output of parsing the YAML. In the get_filters method various notations are supported, but the GUI I've created assumes a specific one.

The idea was to keep it easy to write simple configurations. With YAML and the flat parameter format, writing a simple filter configuration is easier than having a nested structure, the other two formats are just like *args and **kwargs.

I guess for the GUI it is reasonable to choose one format and assume that the user only uses that format, when he wants to edit the config directly. Probably using the dict version and writing the config with indention would be the most readable for users (when writting with indention) and robust against added parameters.

It would indeed be possible to perform such checks without instantiating the filter. There could be a class method to validate parameters that just returns a string error message.

But I think you're right and it is not worth to make this too complicated. Usually the controls will restrict things themself (e.g. range or filetype by extension) and when something is really invalid the program can print and error.

. Having the GUI depend on the filters and some internal parts isn't the end of the world. It's all a matter of defining what the "interface" is.

I thought about the GUI importing the filters (and nothing else), but on the other hand this means importing quite a few modules (like numpy) that the GUI itself does not need.

On one hand the config is probably a pretty stable one. But if, as I also mentioned above, the internal representation could be stable interface for the GUI to interact. Personally I think interfacing through the config isn't ideal, but does yield the biggest separation/loose coupling.

There are a few places where other large changes may happen. I am currently only slowly introducing the heatmap masks, as I am not sure about what filters should be able to do with the data, e.g. if the part masks should be passed through the layers just like the image or not.

This depends on the one hand on design decisions (like what structure the filter should return or if filters should be allowed to modify their input parameters) and on the other hand things may change for better performance. I think there is still stuff to be optimized for copying data less often or maybe running filters in a thread while grabbing the next image.

I hope the layer config will not see big changes, but I am not sure about the internal structures, so writing a config would avoid that the UI needs to be adapted as well.

[mrxz]

I think type double vs. numeric isn't consistent yet.

Indeed, my initial idea was to have numeric handle both floating point and integer values, but it turned out easier to have them separate, but never updated the naming. Will rename them to integer and float as types.

I am not sure if the parameters should communicate the required type ("integer") or the required type of input ("numeric"). And think python is calling it float everywhere even when it probably is a double value.

The cleanest solution is definitely to have the parameters describe the type and not the input/control. It's the filter itself that now exposes a description of its parameters, so it shouldn't be concerned by where or why it's being used. It could be validation, instantiation with default values, a GUI or even something else entirely. If it would contain the desired input type, it would create a coupling to the GUI.

There might be cases where the type alone isn't enough to get the desired input, in those cases we should look into having descriptive ways of hinting at the desired input type. Or, we can introduce pseudo types, similar to how webcam is effectively just a file (which one could argue is just a string for all intended purposes). I think having a color type might end up being convenient (compared to individual R G B sliders).

I guess for the GUI it is reasonable to choose one format and assume that the user only uses that format, when he wants to edit the config directly.

While there's nothing wrong with assuming one, it does pose quite a big limitation: the GUI won't be able to handle all valid configurations. If the parsing would be extracted and reused between the main application and the GUI, that would be resolved. In fact that would even allow supporting alternative config formats (XML, JSON, protobug, pickle, etc...) and the GUI would support it. For this to work the output of the parsing process must be consistent and well defined. The way I see it now, is that in the get_filters function a part of the parsing process is taking place.

I thought about the GUI importing the filters (and nothing else), but on the other hand this means importing quite a few modules (like numpy) that the GUI itself does not need.

If the GUI is standalone, it would indeed not be nice the have to import numpy and the others. If it remains integrated, there is no problem. A solution to the former would be to split the "filter descriptions" and the "filter implementations", but I don't think the overhead of having to update two places whenever filters are added/updated/removed is worth it.

Yet another approach is to have some form of IPC, the main application can expose the filter parameter at runtime and the GUI could post back the config without having to write to the file. Obvious downside being that the GUI won't work without the application running in some form or way.

So, all things considered, I would stick with the approach of having the GUI run with the application itself. While using

This depends on the one hand on design decisions (like what structure the filter should return or if filters should be allowed to modify their input parameters) and on the other hand things may change for better performance. I think there is still stuff to be optimized for copying data less often or maybe running filters in a thread while grabbing the next image.

Indeed, I've also been looking at some performance optimizations. There is quite a bit that can be done, still. I will probably add some suggestions to #20. One key suggestions is indeed to fix the frame representation during processing (e.g. always full-size 4 channels) and allow filters to operation in-place.

One more tricky aspect is the part_masks. In my testing I got quite a significant performance gain by eliminating most of the processing that happens on the part_masks. But if cut out, filters wouldn't be able to make use of it. So I was thinking that if we could determine if any filter in use requires this, we could conditionally perform those computations or not. If generalized we could even detect if the body detection is needed at all for a given configuration or not, in cases where people would use it with just input layer(s) and some filters. (Although I could also see why those users aren't really the main target audience).

I hope the layer config will not see big changes, but I am not sure about the internal structures, so writing a config would avoid that the UI needs to be adapted as well.

While I understand that the config is likely one of most stable components, I would still prefer having the GUI depend on the step after that, for the following reasons:

• GUI will be able to support all valid configurations
• Will require no change in case the configuration format/structure changes or is expanded
• Allows changes made in the GUI to be active without explicit file IO

On top of that, through usage, I've come to find being able to pause and resume the processing quite convenient. So ideally I'd like to see an additional "interface" that the GUI could talk to for those kinds of operations (currently only pausing/resuming and activating a new config).

[allo-]

The cleanest solution is definitely to have the parameters describe the type and not the input/control.

Right

Or, we can introduce pseudo types, similar to how webcam is effectively just a file

I think this is one of the special cases that should just be handled by a generic type.

I think having a color type might end up being convenient (compared to individual R G B sliders).

That right. And the parameters can probably be changed to a rgb tuple. A filter needing them separate can just use sliders (e.g. the ColorFilter filter does not really use color, but multiplies existing ones. The Colorize filter does the same, but by using a gray image as base the effect is probably close enough to "using the color" that a color chooser would be a good input.

While there's nothing wrong with assuming one, it does pose quite a big limitation: the GUI won't be able to handle all valid configurations.

I do not see a big problem here. We'll have the GUI users, the users that edit their config directly and the users that edit their config directly being careful not to use a format that the GUI cannot parse. I think everyone can choose what to do.

The GUI should possibly write the kwargs format, especially because it will probably write all optional values to the config and for some filters it may be hard to remember what the seldom used fifth parameter does. For manual editing this may be not the easiest format (more to type, more chance for typos or remembering the wrong name), but editing a GUI generated config will be easy when it contains all parameters with their names.

For this to work the output of the parsing process must be consistent and well defined. The way I see it now, is that in the get_filters function a part of the parsing process is taking place.

get_filters is a bit grown from parsing arguments for (almost) stateless functions. In the past it created decorators containing the parameters, now it should just parse the three formats and use them as *args and **kwargs for the constructor of the classes. The flat format is a special case and would fail for example when the first parameter of a filter is a list. This is not handled, because it's easier to say "when your first parameter is a list, use one of the other formats" than handling every edge case, when currently no filter has such a problem.

If the GUI is standalone, it would indeed not be nice the have to import numpy and the others. If it remains integrated, there is no problem. A solution to the former would be to split the "filter descriptions" and the "filter implementations", but I don't think the overhead of having to update two places whenever filters are added/updated/removed is worth it.

Right. And the modules are installed anyway. There probably should be some mechanism catch when a filter cannot be imported and just ignore it (catching ImportError), but then the GUI can do the same.

Yet another approach is to have some form of IPC, the main application can expose the filter parameter at runtime and the GUI could post back the config without having to write to the file. Obvious downside being that the GUI won't work without the application running in some form or way. I think it should be able to create a config when the program is not running.

But I thought about that it may be very useful to instantly apply changes to the different segmentation options to finetune the segmentation with direct feedback in the video. Using only the config and reloading all filters on each change means some latency when it would be possible to apply the value to the next frame when the GUI is more integrated.

Indeed, I've also been looking at some performance optimizations. There is quite a bit that can be done, still.

I suppose so. But I am not sure where it is worth to add complexity and where it will only gain very little. Maybe the most important thing would be to get quantized models to work, especially when not using CUDA.

I will probably add some suggestions to #20. One key suggestions is indeed to fix the frame representation during processing (e.g. always full-size 4 channels) and allow filters to operation in-place. I had weird issues in the past when a filter accidentally changed the input and the next one got an unexpected image, that's why it is always copied. I think copying a numpy array is fast because it probably copies the array in C code instead of constructing it again in python. But that's something to discuss in #20.

One more tricky aspect is the part_masks. In my testing I got quite a significant performance gain by eliminating most of the processing that happens on the part_masks. The pre-processing may be quite slow. But it will be slower when every filter needs to do it (again and again). And it may have issues because of converting between cv2, numpy and tensorflow types.

If generalized we could even detect if the body detection is needed at all for a given configuration or not, in cases where people would use it with just input layer(s) and some filters. (Although I could also see why those users aren't really the main target audience). The 24+17 masks for single body parts are not needed for most filters, the main segmentation should IMHO not be optional. Again I would think that it can be fast when using the right numpy/scipy operations.

GUI will be able to support all valid configurations

I do not see the large problem there. You need to know the order of parameters for mapping between args and kwargs and then set the values in the UI. I can split get_filters to create a parse_arguments method later when your problem is that you would like to be able re-use the code.

Will require no change in case the configuration format/structure changes or is expanded

I think the most problems would come from filters that use more complicated input. The general args/kwargs pattern seems right to me. It is simple and it did not result in too unreadable configs in the configurations I used.

Allows changes made in the GUI to be active without explicit file IO

Right, this is interesting for tuning video parameters.

I would just like to avoid to bloat the main script too much as it currently is nice to have it that simple. When I remove the debug_mask sections (I think this can be moved in a filter) and maybe optimize some more parts it will be simpler. The less code there is in the main loop, the easier it is to optimize the remaining parts for performance.

[mrxz]

I think this is one of the special cases that should just be handled by a generic type.

Just to be sure we're on the same page. Currently the webcam filter requires a "file", but instead I've now opted for a specialized type device. But you'd rather see it use the generic type file and possibly some property to indicate what type of file (a capture device)?

I do not see a big problem here. We'll have the GUI users, the users that edit their config directly and the users that edit their config directly being careful not to use a format that the GUI cannot parse. I think everyone can choose what to do.

It won't be a big problem, at the very least the GUI is capable of reading the configs it will write out. My point is more why should we limit it? Simply extract the reading/writing logic to a dedicated file and "normalize" the loaded data so that the logic in get_filters isn't needed. With that you'll get the best of both worlds.

In the same way that it's nice for the GUI to output config files that are still readable/editable, the other way around is equally nice; being able to pick any valid config and use it in the GUI.

I'll see if I can make this change so that it might become a bit clearer what I mean.

The GUI should possibly write the kwargs format

There is one tricky thing about that, currently I use the name of the property more as a display name. If we also record the actual parameter name, there would be duplication (just like with the default values). Unless there's a convenient way to get these names at runtime, I'm not sure duplicating it is worth it.

But I thought about that it may be very useful to instantly apply changes to the different segmentation options to finetune the segmentation with direct feedback in the video. Using only the config and reloading all filters on each change means some latency when it would be possible to apply the value to the next frame when the GUI is more integrated.

That would indeed be useful, it shouldn't be too hard to introduce some controls and apply the changes immediately. I just focussed on the layers and filters initially.

Maybe the most important thing would be to get quantized models to work, especially when not using CUDA.

Indeed, the optimizations that are being made should not make the program overly complex, especially if they don't yield significant gains.

The pre-processing may be quite slow. But it will be slower when every filter needs to do it (again and again). And it may have issues because of converting between cv2, numpy and tensorflow types.

I'm not suggesting letting every filter do it themselves, that would indeed ruin the performance. I'm more thinking of:

part_masks_needed = any([filter.needs_part_masks() for filter in layer_filters])

So it's still the main program that computes it once, but might also not compute it, if not needed by any of the filters currently in use.

(...) the main segmentation should IMHO not be optional. Again I would think that it can be fast when using the right numpy/scipy operations.

No indeed, that's also sort of what I was getting at in my comment, we could, but should seriously consider whether we should. As without it, it would almost defeat the purpose of the program. But if it's possibly to get the performance of that particular operation up to scratch, the complexity might not be needed. That being said, I doubt making the part_masks computation optional is that much work, while in its current state yields a pretty hefty performance increase.

I do not see the large problem there. You need to know the order of parameters for mapping between args and kwargs and then set the values in the UI. I can split get_filters to create a parse_arguments method later when your problem is that you would like to be able re-use the code.

As I said, it isn't a large problem, but conceptually I think the logic is in the wrong place. It's probably a matter of perspective. The way I see it is that the default values and kwargs support is implemented for convenience in the config. But this concern is effectively spread over the parsing of the config, the get_filters method and the individual filters themselves. What I would like to see is to have it closer to the parsing. I'll see if I can prototype this idea a bit, which might help demonstrate it.

When I remove the debug_mask sections (I think this can be moved in a filter) and maybe optimize some more parts it will be simpler. The less code there is in the main loop, the easier it is to optimize the remaining parts for performance.

Couldn't agree more. The debug_mask section is indeed suitable for being a filter on its own.

[allo-]

Just to be sure we're on the same page. Currently the webcam filter requires a "file", but instead I've now opted for a specialized type device. But you'd rather see it use the generic type file and possibly some property to indicate what type of file (a capture device)?

It currently already supports a static image (for creating the example images on the homepage) and using a video file could be possible as well. I am not sure how useful this is, but in the end a device is just a file.

For a few non-filter options, we could add special handling. The real_video_device could of course suggest /dev/video* even when more is supported. I mean this option will always be there and is an important one, so special handling is probably okay.

It won't be a big problem, at the very least the GUI is capable of reading the configs it will write out. My point is more why should we limit it? Simply extract the reading/writing logic to a dedicated file and "normalize" the loaded data so that the logic in get_filters isn't needed. With that you'll get the best of both worlds.

In the end it is just a serialized dict. The question is what structure inside the dict is supported. I think most options can be flat and the layers need to define a type, and a list of filters with lists or dicts of parameters.

There is one tricky thing about that, currently I use the name of the property more as a display name. If we also record the actual parameter name, there would be duplication (just like with the default values). Unless there's a convenient way to get these names at runtime, I'm not sure duplicating it is worth it.

They could be included in the class method that returns the parameters. Some documentation is a good idea anyway.

That would indeed be useful, it shouldn't be too hard to introduce some controls and apply the changes immediately. I just focussed on the layers and filters initially.

Writing it into the config dictionary will work for these options, except for camera device and resolution. This is only slow, because the filters are always reloaded on change (even when neither the layer nor the filter config was changed). So this should be easy to fix by added detection if a layer was changed (e.g. by comparing it with the layer from the previous config).

I'm not suggesting letting every filter do it themselves, that would indeed ruin the performance.

Maybe you can add this to the other issue.

But this concern is effectively spread over the parsing of the config, the get_filters method and the individual filters themselves. What I would like to see is to have it closer to the parsing. I'll see if I can prototype this idea a bit, which might help demonstrate it. I thought I just split the method, so you get args or kwargs (flat parsed into args) using the parse_layer method.

Couldn't agree more. The debug_mask section is indeed suitable for being a filter on its own.

That's one of the things, which are still temporary and can soon be removed when I consider the mask parameters stable.

That's one of the points to think of when not generating all masks. Such a debug_mask filter would need to switch between what masks are generated on config change.

I am still not convinced why generating the mask should have to be slow, but that's something for bug #20.

[schuellerf]

Maybe that's also interesting https://github.com/mrxz/virtual_webcam_background/pull/1 The code definitively needs some cleanup, but now I can fine tune the general settings "on the fly" :-)