[feature request] Adding FAUST and/or Pure Data?

[frink]

Wondering if FAUST might be a good companion to HISE when extended effects are need for instruments... The FAUST interpreter could be embedded into HISE fairly easily for live editing and then used to generate C++ at compile time. This would allow for much more complex effects development without need to delve into C++ DSP. The stdfaust.lib already have some advanced modeling for things like vocal tract, strings, bows, percussion, woodwinds, brass, and bells. Not to mention the colossal amount of open source FAUST that seems to be cropping up all the time. (e.g. Guitarix, SEAM, Pultec EQ, Yamaha YC-20, Echoplex Ep-3, Indian Tambura, String Machine, Mass Interaction Models, Weather)

Likewise, PureData can be cross compiled to C++ as well via HVCC with a much longer history of open code available for perusal and reuse. It can also be embedded and interpreted for rapid development. FAUST and PD have very different paradigms allowing for very different developer experiences. Between these two I believe that you could Allow almost endless effects possibilities without the need for a deep understanding of real-time programming. (NOTE: While CSound should also be a contender I didn't find any way to compile it to C++ - the clossest is Cabbage which just embed it...)

So while this is kind of a shot in the dark, my goal here is not merely to expand features but also to expand user-base. With this approach you're likely to add power-users who have been experimenting with sound-design for years. (This will likel give HISE a better name because they know what they are doing and just need to learn a bit of new syntax...) This has the potential of largely increasing the usage of HISE making development and testing much easier by spreading the load across many more parties.

I think that integrating these projects will probably eventually happen in one of two ways:

1. Either support will be added in HISE itself. (I think this would be a boon to the community - but that's just my opinion...)
2. Or someone will figure out how todo this externally and document it. (@davidhealey seem the most vocal about his discoveries)

Both have pros and cons and I'm honestly not sure which is best for each project...

FAUST seems like a better case for inclusion than PD because it is just text files and HISE is already a pretty powerful IDE. The only things required would be a display wrapper to show the FAUST effects interfaces in HISE and an audio wrappers to get sound in and out of the FAUST code doing thing "The HISE Way". If you really wanted to get exotic you could also include an option to include the FAUST interpreter with generated plugins allowing for patches to have different DSP code and for the user to change aspects of the code without needing to recompile.

Anyway, I thought this was worth getting a discussion going in the dev community to see what everyone else thought. If the Forum is a better place feel free to edit this and/or move discussion to where its supposed to be.

Thanks for building such an awesome product!

Cheers! ~M.Frink

[davidhealey]

hvcc is GNU GPL so not compatible with HISE commercial license. Pure Data is BSD, I'm not sure how the attribution clause would be enforced for derivative proprietary works.

I would like to see Faust integrated with HISE as I would like to use some of the physical modelling synthesis functionality. It has been discussed a couple of times on the HISE forum already: https://forum.hise.audio/topic/2577/faust2hise https://forum.hise.audio/topic/3226/faust-dsp-integration

[ustk]

Agree with both of you. I'd like to delve into Wave Digital Filters as well, I know we can find libraries for it in FAUST (or elsewhere too) but I didn't have the time to search for license compatibility yet for commercial projects.

[frink]

FAUST generated code is considered your own license so compiling FAUST generated DSP code should not be a problem. Incorporating the interpreter. The interpreter libfaust is GPL (I believe with a linking exception) and should be fine within HISE IDE but I'm not sure whether it would work within a plugin itself without license contamination. (need to ) There is already a FAUST JUCE project underway which is GPL 3.0 could be referenced at the very least if not borrowed from directly...

hvcc is GNU GPL so not compatible with HISE commercial license. Pure Data is BSD, I'm not sure how the attribution clause would be enforced for derivative proprietary works.

AFAIK hvcc and FAUST (both GPL 3.0 like HISE) treat generated code separate from the original files. (i.e. your code = your license) But we should probably dig deeper on that to be sure. FAUST stdfaust.lib has its own license which is similar to ISC and PD is BSD 3-clause as you have mentioned already. Either way we'll need a new bit of attribution in HISE alongside the acknowledgement of JUCE. Shouldn't be a big deal.

As an aside, it's probably worth it to include all the licenses (HISE + JUCE + FAUST + stdfaust.lib + PD) in the generated plugin anyway to make compliance easier on the end users. Bottom line, licenses should work so long as HISE doesn't become closed source. Should not contaminate generated plugins with GPL code other than what JUCE already does.

Ugh! It seems like it takes a law degree to develop software anymore...

[dustbro]

FAUST + HISE would be a dream for me +1000

[frink]

I wish I had time to do more than comment. Right now my dev time is overfull.

[sletz]

Faust developer here. You guys can first read:

• the Faut Architecture Files documentation
• and more specifically Adapting the Generated DSP
• and Developing New UI Architectures
• and Embedding the Faust Compiler Using libfaust

[frink]

Thanks @sletz! Would there be any interest in the FAUST community with helping to integrate into HISE? Chris is busy with so much stuff here I'm sure this is low priority.

[sletz]

Same here for now ))-;

[sletz]

GRAME has been selected as a mentor organization on GSoC for the Faust project. And HISE integration if one of the proposed projects, hoping somewhere can be interested there?

[christoph-hart]

@sletz Nice to see HISE being included in the proposed projects. I'm happy to help out with the HISE integration. I think there are three levels of integration that might be implemented sequentielly:

1. Faust scriptnode architecture file. A exporter that spits out C++ from the Faust file that follows the API for scriptnode nodes (I'd suggest that all efforts of including Faust are being done on the scriptnode level as this is the way going forward for all DSP related things). The C++ code will then be compiled into a dll and will be loaded into scriptnode just like the C++ created by the scriptnode C++ compiler. For ultrafast results the existing system of loading the dynamic library with the compiled nodes of a project can be hacked.
2. Dynamic Faust code interpreter using LLVM. A node inside the scriptnode system (just like the other JIT nodes that use the SNEX compiler) that use the same conversion code from 1. but allow dynamic recompilation from a text file containing the Faust code.
3. IDE integration. HISE has a pretty extensive code editor system which already supports many languages (Javascript, C++, GLSL, XML) so a Faust syntax highlighter (based on the JUCE::Tokeniser class as well as the file management to work within HISE projects along with some other eye candy (eg. SVG signal graph display) might make the prototyping even more enjoyable.

I'm a bit reluctant about Faust integration myself at the moment because I spent so much time writing the SNEX JIT compiler which basically offers similar benefits (prototyping DSP with a live coding environment), however it lacks real world applications and a community that knows the language (it's a simple version of C++ but that needs to be learned too). So if there enough people who prefer writing DSP in Faust I think it's worthwhile to pursue it.

[sletz]

Thanks @christoph-hart for the detailed plan. Would you be OK to be GSoC co-mentor on the integration?

[christoph-hart]

It depends on the workload and schedule but yes that's definitely something I would enjoy to do.

[sletz]

The coding is between june and september, and mentoring means regular meeting with the contributor. I've added you name in the project, and we'll see how it goes.

[romsom]

Hi there, this is etXzat from the faust slack! I'm interested in working on this as my project. Right now I'm still very busy with work and life and all that, but I'll try to squeeze in the time to have a look around and familiarize myself a bit with the code base. :)

[frink]

@christoph-hart - I've done mentoring before for GSoC. The impact on your schedule as a mentor is usually minimal. Not more than 0.5-1.5hrs/wk. Sharing this with someone at GRAME it should be even easier on you.

@romsom Very excited to read about your interest!!!!

Looks like there are three parts to this integration and both codebases need commits for everything to work ideally:

1. Integrate libfaust into a new type of script node in HISE and teaching the IDE to edit .dsp files
2. Add a separate HISE destination to the FAUST project (based off the existing JUCE destination) so that FAUST coders can work in FAUST IDEs and still code for HISE. (HISE becomes more front end and FAUST more backend)
3. Wire up an options for HISE compilation to trigger FAUST to convert the .dsp code to C++ before compiling

I can see cases where authors might want to compile FAUST to binary and others where they might want the .dsp code to stay dynamic. So we would likely want an option in the script node settings to allow for either way.

The above would be a nice GSoC project with each step taking about a month for a junior dev to complete, submit and get approvals from mentors on verified completion.

[romsom]

I'm reading documentation for ScriptNodes and libfaust and doing some code experiments at the moment. As far as I can tell already I'd say the JIT approach is even more straight-forward to implement because all the heavy lifting is happening in Faust's DSPFactory. The libfaust docs show the basics of it: https://faustdoc.grame.fr/manual/embedding/ The resulting dsp object can be connected to a "UI" that can manipulate its exported parameters and process audio buffers after it has been told the samplerate. The wrapper ScriptNode would provide a fake "UI" to make the parameters available to HISE's scripting enginge.

I have a small prototype application running already that can process a static buffer of floats with jitted code.

@christoph-hart I'm gravitating towards skipping the DLL part in favor of using libfaust from the beginning. Are you fine with that or is there a benefit of part 1 of your integration plan with the dynamic library that I may be missing?

If I understand the docs correctly (@sletz: feel free to correct me if not), it is even possible to save and load the resulting code and we could bundle the compiled code to use that mechanism instead for a static approach. I'll also prototype that and have a look at the resulting binary file format. @frink That would probably correspond to part 3 of your proposed integration plan, i.e., provide a way to statically compile Faust code?

Also if I understand you correctly, in part 2 you suggest creating an Architecture File for Faust and a faust2hise script? Can HISE export a project from the commandline possibly without spawning a gui? That would be a necessity here, I guess.

I'm writing my proposal at the moment and hope to have a first complete draft later today. I don't know if @sletz has already included you in the GSoC communication channels so you'll receive it as well when I upload it. If not and if you're interested, I can send you a link to the drafts. Final submission will be on Sunday, so there is not a lot of time left unfortunately, but maybe you still have time for some feedback as well. I don't want to rob you of your Easter holidays, though :)

[sletz]

it is even possible to save and load the resulting code and we could bundle the compiled code to use that mechanism instead for a static approach.

Yes using save/restore kind of API, but a pure static approach is still probably useful.

@christoph-hart you can possibly connect to the Faust Slack channel if needed: https://faust.grame.fr/community/help/

[sletz]

@frink Are you willing to be a GSoC mentor for this project ? If yes I can add you here ?

[sletz]

If @frink and @christoph-hart want to be mentors, I need to invite you on the GSoG web platform. Can you send me your emails? Thanks.

[christoph-hart]

Are you fine with that or is there a benefit of part 1 of your integration plan with the dynamic library that I may be missing?

No that could also work (as I suggested a bit imprecisely with the phrase "Dynamic Faust code interpreter using LLVM"). The only thing that is important is that the compiled end product has no depencies on JIT compilation anymore as this would introduce a heavy overhead, so the creation of a Faust architecture file that will create C++ code is obligatory for the final step anyways.

Can HISE export a project from the commandline possibly without spawning a gui?

Yes HISE can be used as command line tool to export projects (mainly for CI integration). However I think the preferred workflow would be:

1. Create a project in HISE
2. Add the Faust node in scriptnode, edit the Faust code and use the JIT compilation for instant prototyping.
3. Before (or during) the export, run a faust architecture file that converts the faust code to a "native" C++ node in scriptnode and add it to the project's C++ source files.
4. When you export the plugin in HISE, it will use the converted C++ files and embed it into the final binary without any dependencies on Faust JIT.

So inside HISE, Faust will always be JIT compiled and in the compiled projects it will be C++ code.

@romsom Yes please send me a link to the drafts. You can use the official info at hise dot audio email.

[romsom]

@christoph-hart Thanks for the clarification! You should have just received an email.

After a discussion with @sletz yesterday, my understanding of our goals has improved quite a bit and that is also reflected in my current proposal. To summarize the main goals are:

1. Implement dynamic/jit compilation (using a hard coded script until the editor is ready)
2. Integrate Faust into HISE's IDE
3. Implement static compilation (using transpilation to C++ and alternatively by using the save/restore API)

I will update the motivation for static compilation in my draft to include your arguments about prototyping. The option to export a plugin with dynamic compilation I would leave in place for plugins where the code should be modifiable if you don't mind, though. :)

[romsom]

@christoph-hart Your email seems to have bounced, I'll try from another address. Also the proposal has just been updated. I'd be great if you could have a look at the detailed schedule, especially part 2, which is about the IDE integration. Do you think the amount of time for that is realistic?

Thanks!

[christoph-hart]

Ah that's so annoying, I need to change the website provider :)

However I can read the proposal in the Google Summer of Code page now that @sletz added me as mentor.

The timeline sounds reasonable. The IDE integration might be straightforward as I've recently redesigned most IDE capabilities to work with both SNEX (C++) as well as Javascript, so adding a third Faust option can benefit from the new modularity. The autocomplete system can also register different providers which can be either extracted from the source code using doxygen or read any other XML file to load the Faust autocomplete data.

But let me know when you start digging into the codebase, this part is a bit underdocumented and "overabstracted" - I didn't expect anybody else to actually use / modify this section...

One thing that is really useful in HISE (both for SNEX and Javascript) is the live variable watch system that allows you to inspect variables in real time. I'm not sure whether the Faust JIT system allows live value inspection, but this would yield a significant advantage over standard solutions like VS Code et al (which is the competing workflow).

[sletz]

Thats is great ! We don't currently have ways to access the internal state of the DSP in JIT model. The way we usually do is to add kind of "probe" as additional control outputs and visualise them.

[frink]

@sletz - Sadly I'm a little out of pocket with everything going right now. I'm in the middle of a physical studio build and three separate business endeavors. (Ugh. Don't ever try to do 2 startups at once!!!! - 3 is insanity!!!!) I'm sorry to say that I don't have time to do more than read comments once every week or two... ☹️

To summarize the main goals are:

• Implement dynamic/jit compilation (using a hard coded script until the editor is ready)
• Integrate Faust into HISE's IDE
• Implement static compilation (using transpilation to C++ and alternatively by using the save/restore API)

@romsom - Love it!

One other small feature that I'd like to put into this is the idea of using your own library instead of stdfaust.lib. The easiest way would be to load FAUST without any lib and import("stdfaust.lib"); or any other to get your FAUST workflow. Another thing to consider is where the include path is... (e.g. I might want to include some FAUST that is std to all my projects stdfrink.lib and other FAUST that is specific to my project...) Thinking through these issues now will encourage good project organization which will lead to better results in the end.

Glad to see this coming together!

[frink]

@romsom Just curious where things are at. Catching up on GH commentary and was surprised not to see anything here in a few months. Love to heave what's new.

[romsom]

@frink Looks pretty good actually. I'm wrapping up loose ends at the moment, so expect proper news in October! If you are daunting and/or impatient you can check out my dev branch here, which should probably work out of the box on linux, but only with the latest libfaust (master-dev) and with the interpreter backend enabled on windows. There is no documentation yet, but that'll arrive on my blog in about two weeks.

[frink]

This is beautiful. Wondering about how the merge to master works in this project. Just realized there is a HUGE discrepancy...

I'm curious - @christoph-hart How does develop get merged to master?

[romsom]

Yup, but that's an issue for another day or week. HISE has practically been living on the develop branch for quite some time now.

[sletz]

Not to mention the colossal amount of open source FAUST that seems to be cropping up all the time.

The more interesting projects are regularly added on the Powered by Faust page.

[christoph-hart]

Wondering about how the merge to master works in this project

Yeah there's absolutely no reason to have a master branch this old, but everybody got used to pulling develop.

Anyways, I've merged the two now so HISE doesn't look like a abandoned project when you look at the master branch...

[dromer]

AFAIK hvcc and FAUST (both GPL 3.0 like HISE) treat generated code separate from the original files. (i.e. your code = your license) But we should probably dig deeper on that to be sure.

Correct. Only the hvcc compiler tools are GPL 3.0 but the static c/c++ libraries are MIT and the transpiled code can be licensed to whatever the user wants.

Btw you'd probably want to use my branch of hvcc that contains a lot of updates and runs on modern systems: https://github.com/Wasted-Audio/hvcc