search

libretime / libretime-debian-packaging

Debian packaging for LibreTime
Other
8 stars 3 forks source link

Packages Required #1

Open paddatrapper opened 7 years ago

paddatrapper commented 7 years ago

The following javascript projects need packaging (separately from LT):

The following PHP projects need packaging:

The following Python projects need packaging:

The following HTML5/Flash projects need packaging:

hairmare commented 7 years ago

Are you sure that you don't want to / can't bundle javascript libraries into the libretime install?

If all libs need separate packages I'm pretty sure that there are heaps more in the libretime-<version>.tar.gz tarball that need their own packages (ie. some python stuff certainly).

In the PHP case, If you do not plan on using the bundled vendor/ directory you can probably use githubs <version>.tar.gz tarball instead.

I've noticed that some of the patched js libraries already address our concerns, so refactoring to more recent upstreams can help there as well (I've went down a large js refactor rabbit-hole while I was releasing alpha 2 but haven't come to an end yet).

paddatrapper commented 7 years ago

Are you sure that you don't want to / can't bundle javascript libraries into the libretime install?

Debian policy prevents this. External libraries cannot be bundled in the package, rather packaged separately.

If all libs need separate packages I'm pretty sure that there are heaps more in the |libretime-.tar.gz| tarball that need their own packages (ie. some python stuff certainly).

I do not know the code base well enough to spot these, so if there are can you please point them out?

In the PHP case, If you do not plan on using the bundled vendor/ directory you can probably use githubs |.tar.gz| tarball instead.

The Debian package needs to depend on the PHP library's package. So tarballs are useful only in doing the actual packaging, but still needs to be separate.

I've noticed that some of the patched js libraries already address our concerns, so refactoring to more recent upstreams can help there as well (I've went down a large js refactor rabbit-hole while I was releasing alpha 2 but haven't come to an end yet).

+1

hairmare commented 7 years ago

If all libs need separate packages I'm pretty sure that there are heaps more in the |libretime-.tar.gz| tarball that need their own packages (ie. some python stuff certainly).

I do not know the code base well enough to spot these, so if there are can you please point them out?

I think pydispatcher and python-vine might be missing from debian, also mutagen might need changes since pypo currently uses a specific version in setup.py.

paddatrapper commented 7 years ago

Thanks. I shall check those out

Lapotor commented 6 years ago

Can you provide the links to the JS Projects from the start post?

Lapotor commented 6 years ago

I searched the open things from top Bootstrap-Datetime:

Colorpicker:

Dropzone:

sPrintf:

Tipsy:

I hope that may help

paddatrapper commented 6 years ago

Thanks. I shall find more links when I get a chance tomorrow. In the mean time, those can be ticked off the list

paddatrapper commented 6 years ago

@hairmare should I create LT issues for the modified library issues?

hairmare commented 6 years ago

Some of the lib changes probably need taking care of by updating and fixing the code to use packages. We might not be able to upstream changes due to us not having originally done them (depending on the project) so we would need to re-implement those.

LibreTime issues might help but I'm a bit afraid that doing the long overdue big refactor of the frontend code will be the only way to get rid of some of the deeply embedded libs that where embedded and locally modified.

paddatrapper commented 6 years ago

@Lapotor, I've added links to the outstanding packages. I am busy packaging js-timezonedetect here, the only things missing are the IPT and the upload, which I shall do tonight. moment-timezone is also ready for IPT and upload

paddatrapper commented 6 years ago

jquery-i18n is packaged and I'll upload tomorrow

hairmare commented 6 years ago

btw, my latest try at getting rid of some of the bundled js code is in https://github.com/LibreTime/libretime/compare/master...radiorabe:dev/js-refactor (with the datatables change that needs backing out of because it seriously breaks hard to fix issues).

Does Debian package multiple version of all js libs? Do we need to work on updating all the js parts to be compatible with upstream?

Robbt commented 6 years ago

I don't see Datatables on the above list of required JS packages, but I did find it in Debian. https://packages.debian.org/stretch/libjs-jquery-datatables

I see that the version is 1.10.16+dfsg-1 where as we are using 1.9.4

I think that Debian only packages a single version of javascript libraries per release. So for this to work we are going to need to refactor our code to meet the API of the packaged version.

Maybe in addition to packages we should do a version comparison as I know that there are API changes that will require code refactoring for a lot of the libraries even if they are packaged in Debian.

Robbt commented 6 years ago

Ok, so the above list is for packages that either have hacks or don't exist. I added links below for version comparison ie the existing debian package vs. the hardcoded one in airtime

Packages missing from the above list for version comparison: Bootstrap - we're using version 2.1 and debian package is 3.3.7+dfsg-2: all jPlayer - we are using Version: 2.7.0 and debian packages 2.7.1+dfsg-1 pUpload - we're at version 1.54 and debian packages 2.1.9 qtio - we're using a nightly version from 2012 where as debian packages 3.0.3~dfsg1-1 As far as waveform-playlist goes even though it originated w/ Airtime former contributor @naomiaro continues to update it with a independent github repo so even though it complicates things if we are going to create packages it might make sense to actually package/build off of a later version.

paddatrapper commented 6 years ago

Yeah, so reach release has a single specific version of each package. Once the release is in freeze, this version does not change. So we need to be compatible with the upstream versions of libraries at the time of release. At this point I'm guessing we should aim for Buster+1, which will be released in 3 years or so, unless we can get everything done by the end of the year around the time Buster goes into freeze

hairmare commented 6 years ago

Will Buster+1 have Python 3 as a default? If it does we need to start using import future in our python code asap.

paddatrapper commented 6 years ago

That's a tricky question... There is a strong drive from the packaging team to move from Python 2 to Python 3, when that will be done is still up in the air. We should try move to 3 though

Robbt commented 6 years ago

Quick question, we've been focusing on javascript debian packages, but how will the composer and pip installed packages be handled ? Is there a zend framework package as I looked it up and it appears that zend framework only exists in wheezie, jessie and sid https://packages.debian.org/sid/zendframework

If we are planning on packaging LibreTime before refactoring and replacing zend framework then it probably makes sense to try to get it all done by the end of the year, but that might be overly ambitious.

paddatrapper commented 6 years ago

Luckily most pip installed libraries should be in Debian already, but there is a packager that generates debian packages from pip repos, which can then be tidied up and uploaded fairly easily. I haven't looked at the required pip and composer packages yet though.

The zend framework you linked is not in testing or stretch because it's EOL - Bug #831418. So we do need to move off it before we can get LibreTime into Debian... I'll try spend some time this week drawing up a list of all libraries we use, be that embedded, pip or composer and their versions compared to Debian

paddatrapper commented 6 years ago

@hairmare looking at that refactor branch, it occurs to me that linking to system-wide js packages may require some changes to the source code, as js packages are installed in /usr/share/javascript/<packagename>/

How does JS source external libraries?

hairmare commented 6 years ago

Getting rid of Zend Framework 1 will we a ton of work. I tried to start the discussion about this early on in https://github.com/LibreTime/libretime/issues/2 but we've been busy taking over and stabilizing the code base.

Client side JS need to be delivered to the client at the right path. It's usually statically hosted at a know path. If the paths don't match we can add aliases in Apache.

Alias /our/path/jquery /usr/share/javascript/jquery

I don't expect many that can't be packaged. Sometimes the question is if we can't refactor to something available and modern so we don't run into legacy issues like with Zend Framework 1 in the longer run.

paddatrapper commented 6 years ago

Yeah, it will require a lot of work... Zend 2/3 are not packaged for Debian, so I agree with your reluctance to move to either.

Ah, those aliases I can easily patch into the packaging then.

I agree with that, I wonder if there is a way of actively tracking library's latest upstream version and creating issues to migrate to it... We use too many libraries to manually monitor that kind of thing

hairmare commented 6 years ago

There don't seem to be lots of PHP frameworks already packaged for Debian. I'm not a big fan of Zend Framework 2+ anyway so I'd rather not go down that route. It looks like symfony3 is available in Debian so using symfony components is a smart choice in any case.

An alternative might also be to go full Python and use something like Django that has first class packaging.

We have to completely refactor the backend in both cases so we might as well split out the html/js parts while we are at it. It doesn't depend which way we pick, both are lot's of work.

The php deps are in a composer.json file after we refactored them.

I was hoping to refactor the js parts so the use bower or npm for similar needs which would also result in a couple of json files containing a list of all the dependencies.

For python there is the pipreqs tool that can generate requirements.txt lists.

paddatrapper commented 6 years ago

Personally I would prefer python backend, but that's purely because I know python and not PHP. Maybe I'll write up a proposal for it to link into LibreTime/libretime#2. Not many talks at FOSDEM today that I'm interested in, so I have a bit of time

Robbt commented 6 years ago

Ok, so this whole discussion has made it clearer to me that we won't be able to add LibreTime as an official debian package until at least Buster + 1 as I really doubt we are going to be able to refactor the codebase by the end of the year.

Would it be possible to figure out a way to build installable debian packages using our existing codebase w/o breaking everything into individual packages. This might be a good stop gap measure for people who want to run LT but don't want to run the install script.

I just figure it might make sense at some point to do this prior to the refactoring, perhaps when we consider our existing codebase to be beta quality so that we can provide a mostly working solution while we refactor everything.

As far as PHP vs. Python goes, I'm more or less neutral. I haven't really learned any python frameworks and I'm more interested in the front-end side of things. I find vue.js to be a fun way of building a UI and I've been working on learning symfony because I'm trying to migrate my stations Drupal site from D6 which was EOL a few years ago to a newer version and they are using symfony components. So that would be the only thing making me feel like symfony might be a good framework to transition to. I look forward to reading any proposals though.

paddatrapper commented 6 years ago

I certainly want to get a working package up somewhere for testing. I'll create a branch for a deb that includes everything and we can see what happens there. It will be pretty limited in where is can be installed though because of the framework not being included in Stretch or testing

isAAAc commented 6 years ago

hi all , is the debian package still on the way ?

paddatrapper commented 6 years ago

I haven't had a chance to work on it lately, but I definitely would like it to go ahead. Even if it is an unofficial deb

paddatrapper commented 5 years ago

I have had some clarification around embedded libraries - we should use packaged versions as much as possible, and where not, they can be embedded