search

roc-streaming / roc-pulse

Roc Toolkit modules for PulseAudio.
https://roc-streaming.org
GNU Lesser General Public License v2.1
8 stars 2 forks source link

Packaging: Launchpad: From #446 , can't build with pulseaudio modules, Launchpad have no internet connection #11

Closed souravdas142 closed 1 year ago

souravdas142 commented 3 years ago

@gavv This is extension of roc-streaming/roc-toolkit#446

I was packaged roc-toolkit-pulse (rock-toolkit with pulseaudio module) for debian based distros (ubuntu) , you can see at here : roc-toolkit-pulse and want to publish at Launchpad as PPA. But the problem is, to build with pulseaudio module roc-toolkit need to download necessory 3rd party sources, like pulseaudio, libtool etc. and from the launchpad build environment doesn't have any external internet connection so that they that can be downloaded. you can see, the buildlog, I have explicitly check with ping, wget, curl without success. Later I discovered another issue on launchpad itself which confirm my suspect : Launchpad issue

Now the question is how to solve ? So if we self contained every 3rd party sources that will be needed to build with pulseaudio but this will make a high maintained package and we don't want that right ? Now what ?

So, Can we introduce a option like --with-openfec-includes (which is for openfec) for pulseaudio also ? This make launchpad build sucessfull as we can get the pulseaduio headers from libpulse-dev this package. Or this pulseaudio module creation should be leave for the pulseaudio side ? Or should I drop this package for pulseaudio module ?

gavv commented 3 years ago

@souravdas142 Hi!

Having a PPA is super cool, thanks a lot for working on it!

First of all, I'm planning to move pulseaudio modules into a separate repo: https://github.com/roc-streaming/roc-pulse/

You can find more details in this comment: https://github.com/roc-streaming/roc-pulse/issues/13

It's not ready yet, but probably it makes sense to do packaging work using that repo. BTW, the new repo name is roc-pulse, I think it makes sense to rename the deb package too.

The new repo also can download and build pulseaudio (using cmake facilities), or it can use external pulseaudio directory. Pulseaudio-specific dependencies like libtool and jsonc will become specific to roc-pulse and will be removed from roc-toolkit.

Now regarding your questions:

But the problem is, to build with pulseaudio module roc-toolkit need to download necessory 3rd party sources, like pulseaudio, libtool etc. and from the launchpad build environment doesn't have any external internet connection so that they that can be downloaded.

You can work it around by cloning roc-toolkit with vendor submodule, which contains pre-downloaded dependencies: https://github.com/roc-streaming/distfiles/tree/51cf372746e84d7a21c8fd3f4b1c6b86c306dc6e/tarballs

scons will find dependencies there and wont download anything during build.

Another solution is to ensure that all dependencies, except pulseaudio, are packaged separately, and create missing packages if needed. I believe that most of our deps already have packages in ubuntu/debian and we can just depend on them; likely the only missing package is OpenFEC.

So, Can we introduce a option like --with-openfec-includes (which is for openfec) for pulseaudio also ? This make launchpad build sucessfull as we can get the pulseaduio headers from libpulse-dev this package. Or this pulseaudio module creation should be leave for the pulseaudio side ? Or should I drop this package for pulseaudio module ?

For now, it's not possible to build roc pulseaudio modules without building pulseaudio sources, and just using headers from libpulse-dev is not enough. See roc-streaming/roc-pulse#13 for reasoning and discussion of a possible workaround. BTW if you would like to participate, you're welcome.

Currently there are two options:

The new cmake-based build system in roc-pulse has the same two modes.

In both cases, we need a full build of pulseaudio.

Another option is to resolve roc-streaming/roc-pulse#13, then we'll have an option to avoid building pulseaudio. By we'll still need full pulseaudio sources, using libpulse-dev will be never enough for pulseaudio modules, because pulsecore headers are not exported in any way.

Regarding roc-toolkit package: probably it makes sense to split it into a few packages, like:

libroc can either be built with vendored openfec, or you can package libopenfec separately. Note that we use our own unofficial fork, but the upstream is pretty dead. Roc can be built with upstream openfec, but it will leak and crash :)

To summarize roc-pulse dependencies:

gavv commented 3 years ago

Hope this helps. Please let me know if you need any assistance, fixes in build system, etc.

souravdas142 commented 3 years ago

First of all, After packaging without pulseaudio I taught It'll be also easier to package with pulseaudio, but its not actually, It will make high maintainent packaging This is not I want as I don't use pulseaudio but I was taught that time "why shouldn't I also package for pulseaudio, as its not still dead?"

You can work it around by cloning roc-toolkit with vendor submodule, which contains pre-downloaded dependencies: https://github.com/roc-streaming/distfiles/tree/51cf372746e84d7a21c8fd3f4b1c6b86c306dc6e/tarballs

This vendor submodule doesn't contain all pulseaudio src package and need to be added from the last 5 years and the upcomming releases, as different versions of debian/ubuntu has different pulseraudio version's. If I make pulseaudio as bulddependencies for roc-pulse , It will be installed and roc-pulse can match its version but what after that? the version can't be found in vendor repo right? I'll not make it fork and becasue again I don't use pulseaudio. If nothing can be done at your end I'll drop packaging support for pulseaudio make it leave for others that uses ubuntu/debian or pulseaudio. In roc-streaming/roc-pulse#13 also I see some patches have to make for PA to make it compile properly, This is again make it higly maintainence job.

Actually I use Arch linux and It is easier a lot in there in terms of packaging.

For pulseaudio module, Why not the pulseaudio itself make the roc modlule at their end ? Is it not possible like what pipewire did?

And secondly,

libroc (contains libroc.so) libroc-dev (depends on libroc + contains headers) roc-tools or roc-toolkit (contains roc-recv and roc-send)

This will be done.

Thirdly,

you can package libopenfec separately.

This has been done. see : openfec, I named it openfec not libopenfec as Arch AUR also did the same.

Note that we use our own unofficial fork, but the upstream is pretty dead. Roc can be built with upstream openfec, but it will leak and crash :)

Upstream ? You mean Original Upstream from : http://openfec.org/ or from roc-streaming/openfec ? Is it safe to use roc-streaming/openfec ? I actually package openfec from http://openfec.org/

souravdas142 commented 3 years ago

@gavv As you mentioned earlier,

Regarding roc-toolkit package: probably it makes sense to split it into a few packages, like:

libroc (contains libroc.so) libroc-dev (depends on libroc + contains headers) roc-tools or roc-toolkit (contains roc-recv and roc-send) libroc can either be built with vendored openfec, or you can package libopenfec separately. Note that we use our own unofficial fork, but the upstream is pretty dead. Roc can be built with upstream openfec, but it will leak and crash :)

Now,

Package's source name : roc-toolkit package - libroc (contains : libroc.so, libroc.so.0, libroc.so.0.1 and suggests roc) package - libroc-dev (contains : all header files and depends on libroc) package - roc (not roc-toolkit you mentioned prev) (contains : roc-conv, roc-recv, roc-send + their man pages and depends on libroc)

and openfec package renamed to libopenfec

Are they correct now ? you didn't mention roc-conv to include under roc package, but I've included, will it be any problem ?

souravdas142 commented 3 years ago

@gavv any news on this issue ?

gavv commented 3 years ago

Hi!

First of all, After packaging without pulseaudio I taught It'll be also easier to package with pulseaudio, but its not actually, It will make high maintainent packaging This is not I want as I don't use pulseaudio but I was taught that time "why shouldn't I also package for pulseaudio, as its not still dead?"

Well, I agree, packaging roc-pulse would require a bit more effort, compared to other parts of roc.

This vendor submodule doesn't contain all pulseaudio src package and need to be added from the last 5 years and the upcomming releases, as different versions of debian/ubuntu has different pulseraudio version's.

If this could help, we can add some specific versions you need to vendor.

If I make pulseaudio as bulddependencies for roc-pulse , It will be installed and roc-pulse can match its version but what after that? the version can't be found in vendor repo right?

As I mentioned, you can configure roc build system to use pre-downloaded user-provided directory with pulseaudio sources instead of downloading it or searching for it in vendor. Both scons and cmake have such options (they are --with-pulseaudio in scons and PULSEAUDIO_SOURCE_DIR in cmake).

In roc-streaming/roc-pulse#13 also I see some patches have to make for PA to make it compile properly, This is again make it higly maintainence job.

Yeah, but I don't suggest you to maintain those patches by yourself. If those patches would beneficial for packaging, we can integrate them into roc-pulse. In this case roc-pulse will still require pulseaudio sources, but wont require building them.

If nothing can be done at your end I'll drop packaging support for pulseaudio make it leave for others that uses ubuntu/debian or pulseaudio.

Up to you! As you see, we have multiple options:

So just let me know what exact help do you need and feel free to ask questions.

Actually I use Arch linux and It is easier a lot in there in terms of packaging.

Indeed.

For pulseaudio module, Why not the pulseaudio itself make the roc modlule at their end ? Is it not possible like what pipewire did?

It would be the most correct solution and I even discussed this possibility with pulseaudio maintainers. However it will require some polishing of our modules and so far I didn't have time for that.

This has been done. see : openfec, I named it openfec not libopenfec as Arch AUR also did the same.

Yeah, I guess openfec is more natural for arch, while libopenfec is more natural for deb-based distros.

Upstream ? You mean Original Upstream from : http://openfec.org/ or from roc-streaming/openfec ? Is it safe to use roc-streaming/openfec ? I actually package openfec from http://openfec.org/

Yes, openfec.org is upstream, and roc-streaming/openfec is our fork. Our fork is fully compatible with the upstream, but have fixes for memory leaks, crashes, and portability issues. It's mentioned here.

I tried to submit our changes to upstream, but got a reply that the authors currently don't have time resources to review and merge patches; they're currently working on a closed-source successor of original lib and I'm not sure if they will continue maintaining the original lib in future. Maybe they will publish the successor instead, who knows.

I plan to continue some minimal maintaining of the fork, i.e. fix bugs and portability issues. I'm also thinking of switching reed-solomon from openfec to another lib, so that openfec would be needed only for LDPC.

Now,

Package's source name : roc-toolkit package - libroc (contains : libroc.so, libroc.so.0, libroc.so.0.1 and suggests roc) package - libroc-dev (contains : all header files and depends on libroc) package - roc (not roc-toolkit you mentioned prev) (contains : roc-conv, roc-recv, roc-send + their man pages and depends on libroc)

and openfec package renamed to libopenfec

Are they correct now ?

Looks good!

you didn't mention roc-conv to include under roc package, but I've included, will it be any problem ?

Nope, I just forgot about it.

souravdas142 commented 3 years ago

Thanks for your reply @gavv

For pulseaudio module, Why not the pulseaudio itself make the roc modlule at their end ? Is it not possible like what pipewire did?

For now, I'm dropping roc-toolkit-pulse package. And I think above will be the easy solution. And for this I'm leaving debianzed build project archive that may be reused by others roc-toolkit-pulse_debianized.tar.gz

openfec 1.4.2.4 "your" fork is now packaged instead the upstream.

I plan to continue some minimal maintaining of the fork, i.e. fix bugs and portability issues. I'm also thinking of switching reed-solomon from openfec to another lib, so that openfec would be needed only for LDPC.

:+1:

gavv commented 1 year ago

PulseAudio modules are now moved to roc-pulse and it supports both online and offline builds. It does not need to build pulseaudio source dir anymore, thought the dir itself is still needed. Everything in documented in README: https://github.com/roc-streaming/roc-pulse