JohannesLorenz commented 10 years ago

I had the following idea: we have many plugins, but if you want to write your own for just one song, there is a long way to go:

Write the plugin in C/C++
Register the plugin
Wait for the next LMMS version ;)
Hope that it will also compile on Windows etc ;)

Anyways, I have once written a C interpreter. It builds a so called "abstract syntax tree" from a string input once, and this tree can then be executed multiple times, like a compiled program. We could predefine controllers and effects that use such equations.

Example: Amplifier:

Predefined: An effect with stereo in and out, 16 knobs, and one string input. What you do: Type "for(int i = 0; i < buf_size; ++i) { out_l[i] = in_l[i] * k[0]; out_r[i] = in_r[i] * k[0]; }" into the string input. Meaning: out_l, out_r, in_l, in_r are predefined variables for the in/out buffers. buf_size is predefined for the buffer size. k[0] is knob 0.

Other examples: Controllers, e.g. that delay the controller signal, apply inertia to it or whatever.

What do you think about such an interpreter?

Note: I know scripts are usually slower. But once a script proves as usable, we could just move the script code into the lmms sources, since they have the same languages.

JohannesLorenz commented 10 years ago

A few technical details about the equation solver:

it has only a few 100 lines of code
it requires g++ >= 4.8 or clang >= 3.3
it needs a lot of memory/cpu power while compiling, but it runs fast once it is compiled

zonkmachine commented 10 years ago

It sounds cool and I don't think a slower app is a problem as that is a natural part of the deal with scripted stuff, but wouldn't support for one of the bigger languages csound or pd be a more natural path?

JohannesLorenz commented 10 years ago

Maybe, but I think they might be more complicated to code into LMMS, and for many C developers, they might be more work to learn. Also, you can not easily port them into LADSPA or similar effects, as they are not really C.

eagles051387 commented 10 years ago

I think in addition to what you are saying here Johannes what would be helpful im sure is a web based version of the doxygen commented code for ease of reference correct?

On Tue, Apr 1, 2014 at 3:23 PM, JohannesLorenz notifications@github.comwrote:

I had the following idea: we have many plugins, but if you want to write your own for just one song, there is a long way to go:

Write the plugin in C/C++

Register the plugin

Wait for the next LMMS version ;)

Hope that it will also compile on Windows etc ;)

Anyways, I have once written a C interpreter. It builds a so called "abstract syntax tree" from a string input once, and this tree can then be executed multiple times, like a compiled program. We could predefine controllers and effects that use such equations.

Example: Amplifier:

Predefined: An effect with stereo in and out, 16 knobs, and one string input. What you do: Type "for(int i = 0; i < buf_size; ++i) { out_l[i] = in_l[i] * k[0]; out_r[i] = in_r[i] * k[0]; }" into the string input. Meaning: out_l, out_r, in_l, in_r are predefined variables for the in/out buffers. buf_size is predefined for the buffer size. k[0] is knob 0.

Other examples: Controllers, e.g. that delay the controller signal, apply inertia to it or whatever.

What do you think about such an interpreter?

Note: I know scripts are usually slower. But once a script proves as usable, we could just move the script code into the lmms sources, since they have the same languages.

Reply to this email directly or view it on GitHubhttps://github.com/LMMS/lmms/issues/551 .

Jonathan Aquilina

JohannesLorenz commented 10 years ago

Am Dienstag, 1. April 2014, 07:31:36 schrieb eagles051387:

I think in addition to what you are saying here Johannes what would be helpful im sure is a web based version of the doxygen commented code for ease of reference correct?

You mean so you could copy parts of existing C code into the script editor? Probably, I think.

Independent from that, web based versions of code/Doxygen are actually never a bad idea, imo.

eagles051387 commented 10 years ago

That would eliminate the need to compile them when you compile lmms.

@toby Could we host doxygen generated documentation on SF?

On Tue, Apr 1, 2014 at 4:38 PM, JohannesLorenz notifications@github.comwrote:

Am Dienstag, 1. April 2014, 07:31:36 schrieb eagles051387:

I think in addition to what you are saying here Johannes what would be helpful im sure is a web based version of the doxygen commented code for ease of reference correct?

You mean so you could copy parts of existing C code into the script editor? Probably, I think.

Independent from that, web based versions of code/Doxygen are actually never a bad idea, imo.

Reply to this email directly or view it on GitHubhttps://github.com/LMMS/lmms/issues/551#issuecomment-39212830 .

Jonathan Aquilina

diizy commented 10 years ago

As it happens, I was just thinking about something similar just the other day.

There's some considerations:

Will the scripting engine (or any scripting engine) be fast enough for realtime audio processing? Remember that we have to be able to process audio in realtime, with a reasonably low latency, and reasonably low CPU consumption. Any scripting engine is going to be extra overhead there. If it's only for very simple effects and equations, if it's used sparingly, it might work. But this would again limit the functionality... if the engine takes too much CPU, no one is going to use it.
In order to be useful from a user's perspective, the chosen language should be easy enough to master for a non-programmer. C is often considered a bit clunky. From this perspective, something like Python would be ideal.

So of course, these two goals are in direct contradiction with each other. C syntax is harder, whereas Python is too slow...

Ideally, we should develop our own scripting language, which is optimized for audio work, and has a syntax suitable for non-programmers (like python). But then, do we really have time/resources to do that?

Anyway, this could all be implemented as an effect plugin with an embedded interpreter, so there's really nothing stopping you from doing this within the existing plugin framework of LMMS.

softrabbit commented 10 years ago

IMO, using an existing language would be wise, one with a user base, tutorials, years of development and so on. Before starting on an in-house solution, it'd be good to do some serious evaluation of the existing alternatives, on points like:

language (computer scientists like lisp-y things for some reason, that might possibly be even worse than C?)
embeddability (obviously)
flexibility, e.g. is it audio-only or can it do other things, too?
performance

There are 7 BSD|(L)GPL alternatives in the list over there, maybe one of them fits: http://en.wikipedia.org/wiki/Comparison_of_audio_synthesis_environments

diizy commented 10 years ago

On 04/02/2014 10:33 AM, Raine M. Ekman wrote:

IMO, using an existing language would be wise, one with a user base, tutorials, years of development and so on. Before starting on an in-house solution, it'd be good to do some serious evaluation of the existing alternatives, on points like:

language (computer scientists like lisp-y things for some reason, that might possibly be even worse than C?)

embeddability (obviously)

flexibility, e.g. is it audio-only or can it do other things, too?

performance

There are 7 BSD|(L)GPL alternatives in the list over there, maybe one of them fits: http://en.wikipedia.org/wiki/Comparison_of_audio_synthesis_environments

I've heard some good things about ChucK, and it seems like an interesting concept... the entire language is concurrent, and makes it very easy to work with timing.

http://en.wikipedia.org/wiki/ChucK

JohannesLorenz commented 10 years ago

Just btw: Why not let the plugin compile C code? There are only two cases:

The user has the needed version of g++ and all the libs (easy to check) -> it can be compiled.
Otherwise, it can not be compiled, the user gets a warning on loading the project file.

This will cause some (very few) people to not be able to load some songs, but if you'd write your plugin the old style and develop it with the song, they could not load your songs as well.

So what's so bad about compiling?

tobydox commented 10 years ago

...99% of Windows and OS X users won't be able to use that feature. Besides that, compiling of course would be the best option for optimal performance.

JohannesLorenz commented 10 years ago

Am Mittwoch, 2. April 2014, 03:40:01 schrieb Tobias Doerffel:

...99% of Windows and OS X users won't be able to use that feature.

Agreed, but the same argument would count if I give a Windows user my song together with a plugin that the song needs. So this is no reason agains compiling imo.

That's why I'd suggest trying to write code that does this.

diizy commented 10 years ago

On 04/02/2014 01:40 PM, Tobias Doerffel wrote:

...99% of Windows and OS X users won't be able to use that feature. Besides that, compiling of course would be the best option for optimal performance.

Well... Windows/OS X users won't be able to use LinuxVST at all, OS X users won't be able to use Windows VST's, and Linux users... well, Linux users can use most Windows VST's, but there are definitely some that don't work via Wine even though they work on the Windows version of LMMS.

This is just my opinion, but personally, I don't see it as much of a problem if LMMS has some advantages on Linux vs. other platforms. Linux is free, and 99% of our core features are still cross-platform. Maybe it'll encourage some people into trying Linux - not that that should be a goal of LMMS in itself, but if it happens as a side effect, I'm fine with it.

softrabbit commented 10 years ago

Here's a volume control plugin in FAUST (minus some metadata and a library import):

smooth(c)   = *(1-c) : +~*(c);
gain        = vslider("[1]", 0, -70, +4, 0.1) : db2linear : smooth(0.999);
process     = *(gain);

I think something in this direction is what's needed if we talk compiled code:

No explicit looping over the buffer
The slider value is just a variable (that might or might not be re-checked for every sample, it doesn't matter for the algorithm)

IOW, hide all the stuff that isn't strictly needed and let the plugin writer focus on the DSP code. Maybe restrict him/her to what's in math.h and no more. Or maybe just use FAUST, if the syntax is good enough.

Maybe this actually is something that could be in a separate program, with suitable hooks to make it work smoothly with LMMS but also usable as a stand-alone code generator?

unfa commented 10 years ago

Could a compiled script's binary code be kept in MMP file so the plugin can run even on a system that doesn't have libs and compiler to make the code?

That however could be a major security risk. On 2 Apr 2014 12:00, "JohannesLorenz" notifications@github.com wrote:

Just btw: Why not let the plugin compile C code? There are only two cases:

The user has the needed version of g++ and all the libs (easy to check) -> it can be compiled.

Otherwise, it can not be compiled, the user gets a warning on loading the project file.

This will cause some (very few) people to not be able to load some songs, but if you'd write your plugin the old style and develop it with the song, they could not load your songs as well.

So what's so bad about compiling?

Reply to this email directly or view it on GitHubhttps://github.com/LMMS/lmms/issues/551#issuecomment-39311522 .

diizy commented 10 years ago

On 04/03/2014 02:38 AM, unfa wrote:

Could a compiled script's binary code be kept in MMP file so the plugin can run even on a system that doesn't have libs and compiler to make the code?

That however could be a major security risk.

Yeah let's not do that.

eagles051387 commented 10 years ago

Shouldnt the presets effects etc have their own extension so we can decipher between projects lmms templates presets etc?

On Thu, Apr 3, 2014 at 1:38 AM, unfa notifications@github.com wrote:

Could a compiled script's binary code be kept in MMP file so the plugin can run even on a system that doesn't have libs and compiler to make the code?

That however could be a major security risk. On 2 Apr 2014 12:00, "JohannesLorenz" notifications@github.com wrote:

Just btw: Why not let the plugin compile C code? There are only two cases:

The user has the needed version of g++ and all the libs (easy to check) -> it can be compiled.

Otherwise, it can not be compiled, the user gets a warning on loading the project file.

This will cause some (very few) people to not be able to load some songs, but if you'd write your plugin the old style and develop it with the song, they could not load your songs as well.

So what's so bad about compiling?

Reply to this email directly or view it on GitHub< https://github.com/LMMS/lmms/issues/551#issuecomment-39311522> .

Reply to this email directly or view it on GitHubhttps://github.com/LMMS/lmms/issues/551#issuecomment-39397444 .

Jonathan Aquilina

JohannesLorenz commented 10 years ago

Am Mittwoch, 2. April 2014, 16:38:37 schrieb unfa:

Could a compiled script's binary code be kept in MMP file so the plugin can run even on a system that doesn't have libs and compiler to make the code?

That however could be a major security risk.

Putting in binary code is an extremely bad idea imo:

We get (large) binary code in our XML files. Thus, you can not put them under revision control (you can, but...).
We start getting closed source. Only one related problem: If a newer version of the plugin is needed, you need to contact the author for recompiling. However, the author might not respond (I've seen multiple projects dying because of this). So you can throw away all project files that used this plugin.
Security is even the smallest problem I'd worry about here.

LMMS is open source and should never encourage people to develop closed source plugins (my opinion). I'd suggest to put script/program source code into the XML files.

JohannesLorenz commented 10 years ago

This is just my opinion, but personally, I don't see it as much of a problem if LMMS has some advantages on Linux vs. other platforms. Linux is free, and 99% of our core features are still cross-platform. Maybe it'll encourage some people into trying Linux - not that that should be a goal of LMMS in itself, but if it happens as a side effect, I'm fine with it.

I agree here.

Also note that popular plugins can later still be compiled into LMMS, so Windows users would just have to wait a bit longer.

Toby, will you accept code that allows C++ source code in XML project files?

eagles051387 commented 10 years ago

If you want a language that is good for concurrency check out google's golang

from what I have seen the syntax is very easy to work with too.

On Wed, Apr 2, 2014 at 10:48 AM, Vesa V notifications@github.com wrote:

On 04/02/2014 10:33 AM, Raine M. Ekman wrote:

IMO, using an existing language would be wise, one with a user base, tutorials, years of development and so on. Before starting on an in-house solution, it'd be good to do some serious evaluation of the existing alternatives, on points like:

language (computer scientists like lisp-y things for some reason, that might possibly be even worse than C?)

embeddability (obviously)

flexibility, e.g. is it audio-only or can it do other things, too?

performance

There are 7 BSD|(L)GPL alternatives in the list over there, maybe one of them fits: http://en.wikipedia.org/wiki/Comparison_of_audio_synthesis_environments

I've heard some good things about ChucK, and it seems like an interesting concept... the entire language is concurrent, and makes it very easy to work with timing.

http://en.wikipedia.org/wiki/ChucK

Reply to this email directly or view it on GitHubhttps://github.com/LMMS/lmms/issues/551#issuecomment-39303433 .

Jonathan Aquilina

pgiblock commented 10 years ago

I for one wouldn't support that. It is way too difficult, in my opinion, to sanitize C++ code so you can trust the resulting binary to not be malicious. I could share project files that infect users computers, leak personal data, or destroy the machine. I would prefer to see a restricted dialect, perhaps not even C-derived. This could be compiled to machine language or source-to-source compiled: instrument-script => C => assembly.

diizy commented 10 years ago

On 04/03/2014 10:06 AM, Paul Giblock wrote:

I for one wouldn't support that. It is way too difficult, in my opinion, to sanitize C++ code so you can trust the resulting binary to not be malicious. I could share project files that infect users computers, leak personal data, or destroy the machine. I would prefer to see a restricted dialect, perhaps not even C-derived. This could be compiled to machine language or source-to-source compiled: instrument-script => C => assembly.

Going to have to agree here...

JohannesLorenz commented 10 years ago

Thanks Paul, I've overseen this danger. However:

What if we'd disallow any #includes and #defines? That way, the programmer could not do anything malicious I'd guess?

On 04/03/2014 10:06 AM, Paul Giblock wrote:

I for one wouldn't support that. It is way too difficult, in my opinion, to sanitize C++ code so you can trust the resulting binary to not be malicious. I could share project files that infect users computers, leak personal data, or destroy the machine. I would prefer to see a restricted dialect, perhaps not even C-derived. This could be compiled to machine language or source-to-source compiled: instrument-script => C => assembly.

Going to have to agree here...

diizy commented 10 years ago

On 04/03/2014 11:29 AM, JohannesLorenz wrote:

Thanks Paul, I've overseen this danger. However:

What if we'd disallow any #includes and #defines? That way, the programmer could not do anything malicious I'd guess?

The list of things we'd have to disallow is a bit longer I'm afraid...

includes just bring code in from headers or installed libs, disallowing

them doesn't matter at all when the user can just enter that code into the script manually.

Same with #defines, defines are just macros that expand to code, and disallowing them does nothing as the user can just enter that code manually.

We'd basically have to disallow huge swaths of code... and there'd still be potential for exploitation. We'd be bound to miss something, someone would figure out a way around the restrictions (have you seen all the ways C/C++ code can be obfuscated? trying to account for all the ways malicious code can be written would be a huge, neverending task).

I don't really want .mmp files to be able to execute arbitrary binary code on my computer. That's the kind of thing that's bitten Microsoft in the ass more times than we can count... if this were implemented, I could never open any .mmp file downloaded from the internet. The sharing platform would become a huge target for people looking for members for their botnets.

eagles051387 commented 10 years ago

I think the only language that we could really work with which is platform neutral would be java :( but then again its not really a good way to take audio, but im impressed how well things perform for this virtual organ platform I know of called jorgan

On Thu, Apr 3, 2014 at 10:40 AM, Vesa V notifications@github.com wrote:

On 04/03/2014 11:29 AM, JohannesLorenz wrote:

Thanks Paul, I've overseen this danger. However:

What if we'd disallow any #includes and #defines? That way, the programmer could not do anything malicious I'd guess?

The list of things we'd have to disallow is a bit longer I'm afraid...

includes just bring code in from headers or installed libs, disallowing

them doesn't matter at all when the user can just enter that code into the script manually.

Same with #defines, defines are just macros that expand to code, and disallowing them does nothing as the user can just enter that code manually.

We'd basically have to disallow huge swaths of code... and there'd still be potential for exploitation. We'd be bound to miss something, someone would figure out a way around the restrictions (have you seen all the ways C/C++ code can be obfuscated? trying to account for all the ways malicious code can be written would be a huge, neverending task).

I don't really want .mmp files to be able to execute arbitrary binary code on my computer. That's the kind of thing that's bitten Microsoft in the ass more times than we can count... if this were implemented, I could never open any .mmp file downloaded from the internet. The sharing platform would become a huge target for people looking for members for their botnets.

Reply to this email directly or view it on GitHubhttps://github.com/LMMS/lmms/issues/551#issuecomment-39426850 .

Jonathan Aquilina

softrabbit commented 10 years ago

Don't look at what to disallow, start from nothing and then allow things until you're happy. The C parser mentioned at the start would actually be pretty useful for sanitizing the code, if the compiler input is then built from the parse tree it's possible to only let safe things pass.

diizy commented 10 years ago

On 04/03/2014 12:31 PM, Raine M. Ekman wrote:

Don't look at what to disallow, start from nothing and then allow things until you're happy. The C parser mentioned at the start would actually be pretty useful for sanitizing the code, if the compiler input is then built from the parse tree it's possible to only let safe things pass.

I think we'd only need to allow a x number of variables and ways to manipulate them with mathematic functions, equations etc. Everything else is superfluous.

Some predefined variables: input, output, sample rate, control values from knobs. Add to that some user-definable variables: a limited amount of variables/arrays which the user can define and use.

Some predefined functions: sin, cos, square root, addition, multiplication, power, etc. Control structures: if clauses, for/while loops.

Upside would be that the syntax could be made very simple this way.

eagles051387 commented 10 years ago

When you say Linux Vst's cant you still compile them and they should run on windows or OSX ?

On Wed, Apr 2, 2014 at 1:03 PM, Vesa V notifications@github.com wrote:

On 04/02/2014 01:40 PM, Tobias Doerffel wrote:

...99% of Windows and OS X users won't be able to use that feature. Besides that, compiling of course would be the best option for optimal performance.

Well... Windows/OS X users won't be able to use LinuxVST at all, OS X users won't be able to use Windows VST's, and Linux users... well, Linux users can use most Windows VST's, but there are definitely some that don't work via Wine even though they work on the Windows version of LMMS.

This is just my opinion, but personally, I don't see it as much of a problem if LMMS has some advantages on Linux vs. other platforms. Linux is free, and 99% of our core features are still cross-platform. Maybe it'll encourage some people into trying Linux - not that that should be a goal of LMMS in itself, but if it happens as a side effect, I'm fine with it.

Reply to this email directly or view it on GitHubhttps://github.com/LMMS/lmms/issues/551#issuecomment-39317297 .

Jonathan Aquilina

diizy commented 10 years ago

On 04/06/2014 02:19 PM, eagles051387 wrote:

When you say Linux Vst's cant you still compile them and they should run on windows or OSX ?

Well, that depends of course on whether those LinuxVST's are also WindowsVST's or MacVST's. Has the author made them to be compilable cross-platform. And of course, whether they're open source. Not all LinuxVST's are open source.

Also, even when we implement LinuxVST support, that doesn't necessarily automatically make MacVST support work.

JohannesLorenz commented 10 years ago

One different idea:

You write C code
the C parser reads this C code it and generates (equivalent) C code from it
a compiler finally compiles the generated C code

This should prevent anything malicious (I guess, since the things we allow are restricted by the C parser) and would be as fast a compiled in library.

Any comments about this?

QrchackOfficial commented 10 years ago

Have you guys considered LUA? It's designed for scripting, very lightweight and the only dependency is standard C library. Also 100% cross-platform.

musikBear commented 10 years ago

performance and security... nough said ..except, it would create a humongus confusion in explaining 'What Is LMMS And What Does It Do'.

imho no

Sti2nd commented 10 years ago

It's pretty cool, and obviously for advanced users :pray: We don't always need to target newbies, you know @musikBear

LMMS / lmms

Script-based plugins(effects, controllers, etc.) #551

includes just bring code in from headers or installed libs, disallowing

includes just bring code in from headers or installed libs, disallowing