search

sebpiq / WebPd

Run your Pure Data patches on the web.
GNU Lesser General Public License v3.0
923 stars 87 forks source link

Are there plans to maintain this project? #113

Closed sinewave440hz closed 1 year ago

sinewave440hz commented 6 years ago

This is a very interesting project, nice work. What is the current status? Are there plans to maintain it and/or would you be open to PRs or, worst case, relinquishing control to another maintainer...?

MacroMachines commented 6 years ago

I am curious as well, this is so cool and I was just referencing it in the Heavy javascript examples issue I was filing. One option could be to offer it up to Heavy and they could be the go-to guys for PD conversion/running on platforms? Just a thought.

sebpiq commented 6 years ago

Hi all!

Unfortunately, the past year or so I haven’t been able to do any music, nor maintain any of my open source projects. And sadly, I don’t see the situation changing in the near future.

Also, I don’t believe that the current way WebPd is implemented is sustainable (a bunch of hacks on top of web audio api). I had started a refactor to web assembly (see the branches) with a working proof of concept, but again no time to push this forward.

And to complete the picture, there is the issue that I have never received any help nor contribution since I started working on WebPd. I don’t see how handing it over to someone else would change this. If you have insight on this question please let me know. I’m happy to try to make it easier for people to contribute, or even share the maintenance rights with someone that is very interested.

I know the guys behind heavy (@mhroth) and we’ve talked of working together but in the end it didn’t happen mostly from lack of time from me. I might be wrong, but I don’t think they have interest in taking over, though the refactoring I have started is going in the same direction as what they do with heavy. The idea would have been to eventually share the same codebase for DSP in C. But again that takes a lot of time!

So ... any constructive ideas on how to move this forward welcome 🍕!

sebpiq commented 6 years ago

(Small correction : I did receive a few contributions here and there, for example midi support, and I want to thank again these people 💪)

artelse commented 6 years ago

I've recently found Purr Data, a version of Pd with a Javascript/Node/SVG frontend build on nw.js. Now that there is a working webtech frontend to Pd, perhaps webPds audio backend can be bolted together somehow?

sinewave440hz commented 6 years ago

I haven't had the chance to try it yet, but the Heavy compiler can generate javascript too, so we already have an alternative there. Perhaps it's not a workflow that suits everyone, but it's a good start...

artelse commented 6 years ago

Tried to find Heavy, do you have a link?

sinewave440hz commented 6 years ago

Enzien audio have developed it: https://enzienaudio.com

sebpiq commented 4 years ago

I'm happy to announce that I will pick up the work on WebPd and spend the whole month on March modernizing the project.

The plan is to give up the use of Web Audio nodes and to focus on using AudioWorklet. This audio worklet will run a bunch of optimized DSP functions which will be compiled with WebAssembly.

This all means that WebPd should be much nice to use, will implement more DSP objects, and possibly even faster in some circumstances.

I'll give more updates once I have more info / made some progress.

:love_letter:

sinewave440hz commented 4 years ago

Fantastic! Would love to get involved if you’re interested...

sebpiq commented 4 years ago

@sinewave440hz sure that would be awesome ! First I'll lay out the bases and then you can tell me what you would be interested in helping with :)

mgsx-dev commented 4 years ago

Hi, @sebpiq that's great news :-) I maintain a library for a video game framework (mainly a libpd wrapper) and i would be glad to support HTML/JS platform as well. I made few experiments some times ago but there was too many limitations and differences so i can't include it. I think WebAssembly implementation could give some good results and overcome those limitations. I'm not sure i could help for implementation but i could hardly test it when it'll be stable enough. Don't hesitate to ping me when you have something usable that i could give a try. Thank you for your work so far and future work :heart:

bever1337 commented 4 years ago

@sebpiq Thank you for putting in so much time and energy.

I'm also interested in contributing to the project! I know you're only just beginning this refactor, but are there any areas where extra contributors would be helpful? I have more experience in networking and web engineering than DSP specifically, but seeing those fields overlap is super exciting. Have you identified the language you want to compile to wasm?

sebpiq commented 4 years ago

Thanks @mgsx-dev and @bever1337 ! I haven't started the work yet (it will happen over March). But I'll post something in this thread once I know more about where help would be needed :)

sinewave440hz commented 4 years ago

I’ve been playing a bit with web audio and am raring to go here, when the time is right. Just so you know :)

Jonathhhan commented 4 years ago

Hi, i am working on something that is perhaps related. Better to say I use Ofelia (Open Frameworks and Pure Data) with Emscripten. My part was mainly creating the interaction between Javascript and Emscripten. So until now you can load audio files (up to a few minutes) and images and send and receive MIDI data with webMIDI. Here is a test: https://pdspectrum.handmadeproductions.de/ https://pdmidi.handmadeproductions.de/ Recently I had the idea that, if one just want to create sound, it is possible to just use ofxPd and Emscripten (which already works) and build the interface entirely with Java Script. The concept of webPd seems a bit different(and is assumingly more efficient?), but perhaps I can also contribute something here and there.

jonlab commented 4 years ago

That is great! Any news?

sebpiq commented 4 years ago

Hi everyone! So a little status update. I have started the refactoring, restructured a bit the base functionality of the library. I want to make things as modular as possible, so I have started separating the code, and rewriting large parts of it :

https://github.com/sebpiq/WebPd_pd-renderer https://github.com/sebpiq/WebPd_shared https://github.com/sebpiq/WebPd_pd-parser

More modules will come, there is still a long way to go ...

For now I got interrupted because I am working on finishing my master's thesis, deadline is 4th May and after that I go back working full steam on this :)

I'll give more status update as the work progresses ! Hang on !

phi9 commented 4 years ago

Good luck with your thesis.(((φ)))On Apr 22, 2020 4:15 AM, Sebastien Piquemal notifications@github.com wrote: Hi everyone! So a little status update. I have started the refactoring, restructured a bit the base functionality of the library. I want to make things as modular as possible, so I have started separating the code, and rewriting large parts of it : https://github.com/sebpiq/WebPd_pd-renderer https://github.com/sebpiq/WebPd_shared https://github.com/sebpiq/WebPd_pd-parser More modules will come, there is still a long way to go ... For now I got interrupted because I am working on finishing my master's thesis, deadline is 4th May and after that I go back working full steam on this :) I'll give more status update as the work progresses ! Hang on !

—You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or unsubscribe.

jonlab commented 4 years ago

Hi Sebastien! I hope your master's thesis went well. any update on the refactoring of WebPd?

sebpiq commented 4 years ago

Hi @jonlab !

Thesis is now defended and out :) I've just restarted the work on WebPd after a few days rest. As it is on my free time, I can't give an estimate on how fast the work will go, but I'll spend a significant portion of my time on it in the next weeks. I'll keep everyone updated here.

jhoefs commented 4 years ago

Sebastien - first thank you so much for making WebPD possible, it's an amazing addition to the community.

Congratulations on your successful thesis defense - I have a Phd, and so know how laborious, unpredictable, and did I say unpredictable? that process can be

Q - I have a web app ready to be deployed whenever the new Web PD runtime is released. It uses fiddle~ - is fiddle~ going to be supported in this version?

Thank you so much!

sebpiq commented 4 years ago

Hi @jhoefs !

Unfortunately, the main objective for now is to first try to implement all the objects there previously was in older WebPd version.

fiddle~ is a bit on the more complex side dsp-wise, so it is not really a priority.

That being said, there will also be a simple plugin system, so if you are up for it you will be able to implement your own pitch following / onset detection algo :)

jhoefs commented 4 years ago

Definitely.

And, as I understand it, there is a way to load pd externals into WebPD - correct me if I am wrong, that is possible, isn't it?

In which case, helmholtz~ or sigmund~ or something similar could be loaded into WebPD, and then building the appropriate data analysis on top of that to create the same (and probably customized :]) functionality as fiddle~ is pretty trivial, as it is just parsing numbers at that point

Once again, kudos for creating this. It is now up to the rest of us to create awesome DSP apps that turns the web into a sea of dynamic sound....

sebpiq commented 4 years ago

It is now up to the rest of us to create awesome DSP apps that turns the web into a sea of dynamic sound....

Looking forward to see this happen 😄💟

And, as I understand it, there is a way to load pd externals into WebPD - correct me if I am wrong, that is possible, isn't it?

Hmm ... it's more complicated than that. You can't directly load existing externals into WebPd as the existing externals are implemented in C and compiled. One would need to reimplement (or repackage) existing externals so that they can run in the browser and in WebPd. There would be several solutions for this (for example loading helmholtz~ in WebPd) :

  1. somehow compiling helmholtz~ to WebAssembly using emscripten
  2. reimplementing it in JavaScript
  3. reimplementing in C/C++ + compiling in WebAssembly, etc ...
  4. maybe writing a sort of script that could take any external and compile it to a format that is compatible with WebPd (would be awesome) ?

Point is, that since the browser is a completely separate environment than native environments which usually run pd and its externals, you can't directly reuse the code of pd / and its externals there. WebPd is in fact a complete re-implementation from scratch. It pretty much only shares with pd the file format and the vanilla list of objects.

On Fri, Jul 3, 2020 at 5:40 PM jhoefs notifications@github.com wrote:

Definitely.

And, as I understand it, there is a way to load pd externals into WebPD - correct me if I am wrong, that is possible, isn't it?

In which case, helmholtz~ or something similar could be loaded into WebPD, and then building the appropriate data analysis on top of that to create the same (and probably customized :]) functionality as fiddle~ is pretty trivial, as it is just parsing numbers at that point

Once again, kudos for creating this. It is now up to the rest of us to create awesome DSP apps that turns the web into a sea of dynamic sound....

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sebpiq/WebPd/issues/113#issuecomment-653601165, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACJGOYTHRPGTXIYWCY7U5TRZX3WXANCNFSM4E6W4FYQ .

--

Sébastien Piquemal

----- @sebpiq ----- http://github.com/sebpiq ----- http://funktion.fm

marnen commented 4 years ago

I’d be interested to help out. I’m an experienced software developer and composer, and a moderately experienced Pd user, and I’d like to work on this to increase my knowledge of how to implement DSP. What would be a good place to start helping?

sebpiq commented 4 years ago

I am awfully late, so sorry about that ... but now I have again a bit of free time between 2 paid projects, and will continue the work. It is long overdue that I manage to divide tasks and enable other people to help in this huge project !!! And I'm so happy that so many people are interested in helping to make WebPd finally happen :yum:

@sinewave440hz @mgsx-dev @marnen @bever1337 Are you still interested in working with me on this ? Are you fluent with TypeScript, js, dsp, etc ... ? I want to know if there is interest before I put some time in opening collaboration channels, and preparing the project for team work :smile:

marnen commented 4 years ago

Yes, I'm very experienced with JavaScript, and I've used TypeScript a bit in the past.

sebpiq commented 4 years ago

Ok perfect! I'll set up something for collaboration then !!!

On Fri, 28 Aug 2020, 12:31 Marnen Laibow-Koser, notifications@github.com wrote:

Yes, I'm very experienced with JavaScript, and I've used TypeScript a bit in the past.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sebpiq/WebPd/issues/113#issuecomment-682971035, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACJGO326FGKEDEDAWQ7LI3SC7SYPANCNFSM4E6W4FYQ .

giulianobici commented 3 years ago

I'm happy to hear this project will continue. I was working last week with the examples, and it looks excellent. I am still learning more about "Webpd." Anyway, it's promising for different artistic projects, and I am migrating a couple to "Webpd." Congratulations on the project. If you have any update or beta version available to test, I will be glad to try it. I am looking forward to the news.

ok-very commented 3 years ago

Also simply here to express interest and the hope that this will continue to develop, thanks for the interesting project!

vortextemporum commented 3 years ago

Keep up the great work! I am very excited about this project

nkleemann commented 3 years ago

Just posting to let you know that your work does matter and I'm very excited about your progress. :)

meta-meta commented 3 years ago

I'm thankful this project exists and excited to see where it goes next!

mickeyvanolst commented 3 years ago

So excited to see this :-) I'm actually hoping to use this in combination with https://cables.gl/ has anyone tried stuff like this?

droussel commented 3 years ago

Hi @sebpiq,

I was wondering what was the state of the rewrite was? I looked at the various Webpd_* repos to try to wrap my head around the applications architecture but I'm not sure what the state of things are and where I could begin hacking at things. And also, are you still planning to make progress on this on your end as well?

Thanks!

sebpiq commented 3 years ago

Hi all !

I haven't given up on finishing the rewrite work. In fact it is not so far off I believe. I've had issues finding time to work on it, mostly because this is on my free time, and I haven't had any in the past months.

My goal for WebPd in the coming months, is :

Thanks everyone for pushing me to finish this :wink: ! Don't hesitate if you have some comments !

bever1337 commented 3 years ago

@sebpiq Awesome news! I'm still around to help contribute so I'm excited to see what's coming. Have you started setting up any workspaces for collaboration or is that still in the works?

I look forward to seeing the crowdfunding page too, and I'll be glad to kick in what I can.

ogbabydiesal commented 3 years ago

Hey folks! I'd be happy to contribute if possible. I'm specifically interested in getting the ifft~ and block~ objects working to get spectral processing working on the web. Let me know how I might be of service.

sebpiq commented 2 years ago

TL;DR : Faster, better, stronger, WebPd 1.0 is coming (featuring WebAssembly, Audio Worklet and more) ... but it needs your support !

WebPd is a highly modular web audio programming toolkit inspired by Pure Data.

→ it allows Pure Data patches to run in web pages, therefore enabling non-programmers (artists musicians, etc ... ) to design live and interactive audio for the web.

→ it provides experienced web programmers with a complete audio toolkit that is production-ready, and enables efficient audio synthesis and processing in the browser.

You can try a demo of the upcoming version here (LINK).
You can donate money to help making it real (LINK).
Your money will help moving forward the following roadmap (LINK).

FULL VERSION :

For the past weeks, development of WebPd has picked up a good pace, and the project has reached a state where I feel now confident for sharing a demo, a public update on what's going on and a call for crowdfunding.

1. How WebPd Got Here

The project was started in 2010 by Chris Mc Cormick when Firefox released the first implementation of an API that enabled live audio synthesis in the web browser. In 2012, I took over and ported WebPd to a different API called Web Audio API, which has since become the web standard for live audio.

The Web Audio API implementation of WebPd is still the current version (v0.4) and hasn't seen any significant update for many years. It works, but it is hackish and limited. This is due to the fact that the Web Audio API impose its own synthesis and processing functions (oscillators, filters, etc, ... ), and is therefore nearly impossible to customize.

Luckily, the situation is now very different than it was 10 years ago. New APIs and standards have been proposed and adopted by all major browser vendors, making it finally possible to build serious custom audio apps for the web browser :

With these in mind, I've been meaning to re build WebPd from scratch for several years already but it's a daunting task : it requires an entire re-design of the audio engine, and a full re-write of the code to make the project future proof. Another problem is that there isn't yet a good ecosystem of libraries for writing code with the new web audio technologies mentioned above. Therefore, some generic web audio packages need to be built as part of the project (adding to the complexity and the amount of work).

2. What's planned for this new version, WebPd 1.0

First, let me say that WebPd does not intend to be a fully-fledged application like Pure Data is on the desktop, but rather a library for developing web applications. In that sense, conceptually, it is closer to libpd than to Pure Data. Of course, you could build a fully-featured user interface, a Pure Data on the web, using WebPd, but that is out of the scope of the project.

WebPd's goals are :

-> to allow artists to take their Pure Data patches and run these in web pages, therefore enabling non-programmers (sound designers, musicians, etc ... ) to design live and interactive audio for the web.

-> to provide experienced web programmers with a complete audio toolkit that is production-ready, and enables efficient audio synthesis and processing in the browser.

For this to be possible, WebPd needs a small community of users and developers. In fact, I have received many messages inquiring about the status of the project and many have offered help with development, so I know that this community exists.

The first milestone is therefore to build a minimum viable product, write good docs, resources for beginners to find help, find a good platform for questions and discussion, etc ... making it more easy for others to contribute. In the long run, I would like the development work to be as collaborative as possible, and progressively hand over ownership of the project to the community.

3. Crowdfunding campaign

In order to help reach this first milestone, I am starting a small crowdfunding campaign so I can spend more time on the project in the coming months. You can find here a list of what I plan to achieve for WebPd 1.0 with help from that money.

If this is something you'd like to help with, you can donate on the opencollective page of the project. Any amount is welcome.

You can also ask questions, or come share ideas here.

marnen commented 2 years ago

@sebpiq I understand that you want to raise money to help this project survive, but I’m not sure that dumping your entire funding appeal into this discussion with developers who would like to contribute code to this project was all that helpful.

What I’d like to see in this thread is some idea of what needs to happen from a developer point of view and how we might be able to be helpful in that respect.

sebpiq commented 2 years ago

@marnen sorry if that was misunderstood / clumsy. From a developer's point of view it is impossible to help at the moment. I am redesigning the whole thing and it is close to a full time job. I don't see how I could share some of the work / cut out some precise tasks while I am changing all the parts all the time to try to reach something satisfactory.

This is what I also meant with this message. I want to be able for other people to contribute, and other maintainers to join in, but before this is even possible I need to bring the project to a certain point of stability, and that is not yet there, and will still require a lot of work, even though I am working on it actively.

Does that clarify things ?

marnen commented 2 years ago

It does clarify things, but I think you may underestimate how much help you could get from the community if you’d ask for it.

You say that it’s impossible for others to help, but why? Wouldn’t it be possible for others to discuss these designs with you? That way you can talk through the harder problems with other folks with expertise, and you won’t be the only one who understands the design once it stabilizes. :)

sebpiq commented 2 years ago

That's a good point ! I guess I'm so used to not getting help on this, even though I did try at some point that I am not expecting it anymore :sweat_smile: Also ... it does take a massive amount of time.

Here is an overview of the design I am working on at the moment :

This is how the [noise~] node implementation looks like : 

import { NodeCodeGenerator } from '../types'

export const setup: NodeCodeGenerator = () => ``

export const loop: NodeCodeGenerator = (_, { outs }) => {
    return `
        ${outs('0')} = Math.random() * 2 - 1
    `
}

This is the DSP code generated with a simple [noise~] -> [dac~] : 

let o = 0;
let frame = -1;
const isNumber = (v) => typeof v === "number";
let PROCESSOR_OUTPUT1 = 0;
let PROCESSOR_OUTPUT2 = 0;

let pd_0_1655305928329_OUTS_0 = 0;

let pd_0_0_INS_0 = 0;
let pd_0_0_INS_1 = 0;

return {
  loop: () => {
    frame++;

    pd_0_1655305928329_OUTS_0 = Math.random() * 2 - 1;

    pd_0_0_INS_0 = pd_0_1655305928329_OUTS_0;
    pd_0_0_INS_1 = pd_0_1655305928329_OUTS_0;

    PROCESSOR_OUTPUT1 = pd_0_0_INS_0;
    PROCESSOR_OUTPUT2 = pd_0_0_INS_1;

    return [PROCESSOR_OUTPUT1, PROCESSOR_OUTPUT2];
  },
  ports: {
    getVariable: (variableName) => {
      return eval(variableName);
    },
    setVariable: (variableName, variableValue) => {
      eval(variableName + " = variableValue");
    },
  },
};

The plan is to use the same code generators for generating AssemblyScript which then be compiled to WebAssembly.

marnen commented 2 years ago

That’s useful. I’d suggest that it would be helpful to have the README files for each subproject a little clearer about what the project does. For example, I just looked at pd-parser, and I only knew what it does because of your description here—there was nothing on the project itself that described it. That’s probably a barrier to contribution; folks are not going to contribute code if they don’t know what the project does. :)

dfl commented 2 years ago

Thanks for the update! I wonder if it might be possible to leverage some of the development already done with Faust? https://dl.acm.org/doi/fullHtml/10.1145/3184558.3185970 If one could compile a Pd graph to Faust Imperative Representation, then the rest of the chain would be good to go. I can’t find any documentation on FIR however, other than a mention of its existence. Perhaps the people at GRAME.fr would be interested in collaborating?

On Jun 15, 2022, at 8:15 AM, Sebastien Piquemal @.***> wrote:

That's a good point ! I guess I'm so used to not getting help on this, even though I did try at some point that I am not expecting it anymore 😅 Also ... it does take a massive amount of time.

Here is an overview of the design I am working on at the moment :

@webpd/pd-parser https://github.com/sebpiq/WebPd_pd-parser : Parser for pd files. Parses a .pd file into a PdJson format that mimics the structure of the pd file. Types for this format can be found here https://github.com/sebpiq/WebPd_shared/blob/master/types/PdJson.ts @webpd/dsp-graph https://github.com/sebpiq/WebPd_dsp-graph : Utilities for transforming pd patches into simplified DSP graphs. This is the heart of the library. It takes PdJson data and turns it into a simplified DSP graph. Types for this graph format can be found here https://github.com/sebpiq/WebPd_shared/blob/master/types/PdDspGraph.ts. It modifies the PdJson format significantly in the following ways :

Splitting inlets into inlets that can accept only one type (e.g. [osc~] first inlet is split into 0_control and 0_signal). Flattening patches, removing subpatches, abstractions, etc ... Making some of Pd's implicit behavior explicit. For example, when several signals connected to the same inlet, a node is added for summing them @webpd/compiler-js https://github.com/sebpiq/WebPd_compiler-js : Compiler for compiling a DSP graph into a single JavaScript function. The code is generated as text, and can be interpreted on the fly with eval.

This is how the [noise~] node implementation looks like :

import { NodeCodeGenerator } from '../types'

export const setup: NodeCodeGenerator = () => ``

export const loop: NodeCodeGenerator = (_, { outs }) => { return ${outs('0')} = Math.random() * 2 - 1 } This is the DSP code generated with a simple [noise~] -> [dac~] :

let o = 0; let frame = -1; const isNumber = (v) => typeof v === "number"; let PROCESSOR_OUTPUT1 = 0; let PROCESSOR_OUTPUT2 = 0;

let pd_0_1655305928329_OUTS_0 = 0;

let pd_0_0_INS_0 = 0; let pd_0_0_INS_1 = 0;

return { loop: () => { frame++;

pd_0_1655305928329_OUTS_0 = Math.random() * 2 - 1;

pd_0_0_INS_0 = pd_0_1655305928329_OUTS_0;
pd_0_0_INS_1 = pd_0_1655305928329_OUTS_0;

PROCESSOR_OUTPUT1 = pd_0_0_INS_0;
PROCESSOR_OUTPUT2 = pd_0_0_INS_1;

return [PROCESSOR_OUTPUT1, PROCESSOR_OUTPUT2];

}, ports: { getVariable: (variableName) => { return eval(variableName); }, setVariable: (variableName, variableValue) => { eval(variableName + " = variableValue"); }, }, };

The plan is to use the same code generators for generating AssemblyScript which then be compiled to WebAssembly.

@webpd/engine-live-eval https://github.com/sebpiq/WebPd_engine-live-eval : AudioWorklet for running DSP JavaScript code on-the-fly using eval. This is an AudioWorkletNode which takes code as text (under the format exemplified above), and runs it.

@webpd/pd-renderer https://github.com/sebpiq/WebPd_pd-renderer : Renderer for pd files PdJson to .pd text file.

@webpd/shared https://github.com/sebpiq/WebPd_shared : Shared tools and types for the other packages

— Reply to this email directly, view it on GitHub https://github.com/sebpiq/WebPd/issues/113#issuecomment-1156602950, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAACGSH5TJRSFA7BHHDWG3VPHXSBANCNFSM4E6W4FYQ. You are receiving this because you are subscribed to this thread.

sebpiq commented 2 years ago

@marnen I'll do it very soon ! I have just changed again the repository structure, so I want to get this right first :)

@dfl yes ! That was actually one of the options I looked into. I even had a call with Stephane Letz who is one of the main developers behind Faust. In the end I decided to not go with this :

This being said, another objective of the re-implementation is to build an architecture for the project that makes it very modular, so that it would be possible to implement a Faust audio engine without having to redo the whole project.

designerfuzzi commented 2 years ago

after reading where this might go i thought i throw some arguments in here. Seems you are in a decition making phase to organise workload and structure..

where i come from (mindset) I am developing WebMIDI for Webkit (iOS/macOS/tvOS/etc. - UI/NSViews) and i am still in checking checking checking for its stability. To do that i visit random webmidi pages to see if their javascript API is working well as defined or claimed and how my performance compares to .. lets say Chrome WebMidi. There are a looot buggy javascripts out there who claim to follow WebMidi API but actually just emulate, some don't even use performance timing which makes it useless, some javascripts assume you browse always with Chrome, and on and on.. a real mess! Some even do fake midi that presents the whole API as useless, in other words some do not even establish proper midi connections, instead they just emulates and users wonder why it is not working. So to be not confused i go straight for WebMIDI standard and do no emulation nor any kind of workaround. If some webpage-midi fails, it fails, i check their syntax to be sure the mistake is not on my side (backend of the browser/midi server itself). Which, in turn, makes Chrome de facto a good parallel testing environment to compare to. So lesson learned, i have to go straight for standard, otherwise i mess around with random interspaceballbugs i will never solve, because those are out of my influence.
On top i solve the missing User-consent layer which was leading to Apple's decition to not support Webmidi yet. The very same security layer that is implemented ugly in Chrome for some "reason".

Why is this of interest? Because PD without MIDI and in your case then WebMidi API it is like just the half of its idea. Which means for the sake of development you would end up with basically "only" supporting Chrome browsers (yet). Which in turn explains why i do WebMidi for WKWebKit-Views I do machine code that is served from the Webview(browser itself) and interconnects via webMidi.API to callback browser-inner methods/system/functions that deliver objects back to javascript. Thats how WebMidi works basically.. Recreating Buffer handling and virtual Midi ports etc via javascript is completely useless, no matter how someone tries, it must be served from hardware via software to and from javascript API which is called WebMidi then by well defined syntax.

Looking forward.. As PD is in fact already string based script language that is because of its simple syntax extensible as hell you might end up rewriting PD with javascript which as i understand is your proposal here. But then you end up with a parallel universe that does not what PD does already and PD does still more then what webPD would do as you also explained. Not saying such project could not come very very close, it belief it could. As you point out you will never reach the quality of what we all know as PD.. plus you do everything again from scratch, which is indeed a massive workload which leads to a lot of workarounds you have to solve to mimic PD in js.

Knock knock.. So let's talk about some proper API that allows interpretation/handling of PD files to & from browsers. If i have the choice between emulation and PD itself, i would go for the real thing because then the code base is there already. In that case only the web-interactive part is missing. Meaning PD-files and their syntax could become a mime type and so they can be interpreted as js-object-model that allows interaction in browsers, no matter if someone writes some WebAudio PD emulator.

one step ahead it would be technical even possible to allow PD-objects that are not installed on a local machine but come with a webpage and interact with the real PD via the browser.. Which is the BIG deal if that is possible. Clearly the bottleneck for such future is some established javascript command set that allows interaction between browser and PD on a deeper level than just MIDI or OSC or port-messeraroundery.

conclusion. (and don't get me wrong, your project proposal still sounds super attractive) PD: in fact is already a browser, it browses PD files, just lacks any real browser integration to allow users to have controll who is allowed to load what from where and what to do if that fails. So PD as is, is total not webpage friendly nor does it define any url scheme to integrate webpage-based external content as new objects (as easy as f**k). Ergo it feels a little bit outdated and hyper minimalistic and as such leaves a lot users lost in kinda UI-look-and-feel barriers. While it is actually available on all plattforms and is in fact super dupe'r simple. It just does not have an Web-API

what i understand: Your project proposes to circumvent this bottleneck by creating a second universe with its completely independent Javascript based WebAudio engine.

Anybody able to follow what i am writing about?

my proposal is following.. to avoid recreation of the whole universe of pd objects some proper WebPDAPI could be in your mind which allows later on to integrate your work into the real PD world and wise versa. That could look something like (beware non-yet-existend javascript-command-API)

navigator.requestPDAccess({dsp:true}).then(onSuccess, onFailure);

and the browser window would make a connection to PD and deliver and receive objects, sound what ever we like with the known code base

in short such WebPD-API does not exist because nobody made one. Where is no standard, there is no integration and implementation in browsers to interact with the existing PD world. See the potential?

sebpiq commented 2 years ago

Hi @designerfuzzi ! Sorry for the massively slow answer ...

Your project proposes to circumvent this bottleneck by creating a second universe with its completely independent Javascript based WebAudio engine.

Yes ! That's exactly right.

But then you end up with a parallel universe that does not what PD does already and PD does still more then what webPD would do as you also explained

if I'd only wanted compatibility between web browser and Pd, there are already a bunch of projects. For example a WebAssembly build of the whole Pd : https://github.com/cuinjune/purr-data

WebPd on the other hand is a toolkit, it does NOT aim to become a full-blown programming environment (Pd does that well). WebPd will never have an official GUI. In fact, the current project structure only has a small compatibility layer to parse Pd Patches, and then immediately forgets it ever was Pd, by using a custom dsp graph data structure. What WebPd will, hopefully, is provide a comprehensive and extensible sound programming toolkit for the web, which could be used as a production-ready sound engine for all kinds of interactive audio things (video games for example). In that sense, it is loosely related with Pd. Reimplementing things from scratch is necessary.

You might ask then, why calling it WebPd at all ... Well, basically for what makes Pd great :

In any case, thanks for all the feedback @designerfuzzi, and that makes me realize that I need to clarify all these points in the README, because this is not the first time I get similar comments. And if you have more thoughts on the topic I'd also be glad to hear them :blush:

Jonathhhan commented 2 years ago

@designerfuzzi sorry for hijacking this thread (maybe we can discuss it somewhere else - if you want to and have time). I optimized Open Frameworks for running it with Emscripten (also with the aim to run PD patches via ofxPd in the browser). For that, I implemented audioWorklets and webMIDI. First, I am interested in general thougths about that approach, and second, it would be great if you can check out my webMidi implementation (i want to improve it). Here is a patch (which only works, if a MIDI out device is connnected): https://midifilemarkovb.handmadeproductions.de/ And here is one that works with audioWorklets: https://test2.handmadeproductions.de/

Reinissance commented 1 year ago

Reading this discussion I have to admit I don't get everything clearly and thus cannot see exactly what the advantage of using webpd is: Is it, that it only compiles the pdpatch and not the whole pd application with externals like in purr data or the pdwebparty and therefore has little load time, for example?