SynoCommunity / spksrc

Cross compilation framework to create native packages for the Synology's NAS
https://synocommunity.com
Other
3.02k stars 1.23k forks source link

Beta / Test packages for DSM 6 #3138

Closed Safihre closed 6 years ago

Safihre commented 6 years ago

Updated 14-07-2018: This issue is now closed as bulk-package-conversion has finished.

Up-vote the issue for the package you are missing here or open a new issue in case we don't have one for the package you need!

~### New packages converted for generic service account, DSM6 compatibility and new architecture compatibility. All are still in testing!~

~### Check the version you need here. Note that the x64 package covers multiple architectures.~

~*Packages will be removed once they are published via the regular repository. Packages downloaded here will then automatically update to the latest official version.**~

Stanzilla commented 6 years ago

Continuing our discussion from https://github.com/SynoCommunity/spksrc/pull/3121#issuecomment-363897704 I had to manually fix all the download/media folders to make it work. That includes every location added to Sonarr if you have more than one. (I have external drives)

ghost commented 6 years ago

I have managed to get the apollo lake versions of Sonarr and Radarr working well with Mono/Jacket etc behind also appear to work well on my DS918+.

Today I tried Transmission and although it seemed to install fine, where I used login credentials for a user I created just for transmission that I have just given read/write permissions to my downloads folder, on launch I get:

401:unathorised unauthorised user

I saw that the SC-download group had a custom permission for my downloads folder so I tried changing this to read/write but this still gives me the same problem. I also uninstalled and reinstalled a couple of times and triple checked permissions and credentials in the GUI.

Does this mean there is a package problem or am I doing something incorrect?

Thanks and have donated

UPDATE - I tried reloading the IP with :9091 and I got a login window. I first tried my normal disk station login but no dice, then out of pure interest I tried logging in with the credentials I created for transmission. Amazingly this worked. It is strange (at least for me) that I would have to login as the transmission user, as I do not have to login for the sonarr or radarr users I created for those apps...

Safihre commented 6 years ago

You are mixing things up I think. During the installation you can choose the username/password for the interface, this is also mentioned clearly. Seperatly there is the disk permissions, which are managed through sc-download and have default read/write and some more. These are not linked and it was mentioned during first installation in the wizard.

ghost commented 6 years ago

Thanks for clarifying. Clearly I need to play closer attention to the instructions on page!

BenjV commented 6 years ago

@Safihre @ymartin59 I was looking at some of the package you made available here and noticed some issue's First of all the folder conf containes double version of files. Probably a file and hard link. This can cause problems during installation of a package.

Secondly: I noticed in the INFO file the following. thirdparty= "yes" This is not applicable anymore as of DSM 5 ctl_stop="yes" Does not belong in the INFO file. It is not a command known to DSM

Thirdly: I was doing some testing of my own packages and I noticed something changed. The package is run as root if I do not use the "su" command to run it as the desired user. So maybe this must be changed in the start-stop-status script, just as the command used for DSM 5.

Safihre commented 6 years ago

@BenjV

The package is run as root if I do not use the "su" command to run it as the desired user.

On DSM6? I didn't test specifically, but all files created by the packages have the right user:group and the packages don't have access to non-package-user locations. So seems they are not root?

@ymartin59 Will you fix the other things as part of #3123?

BenjV commented 6 years ago

This is on DSM 6 only.

I shall test again, maybe I made a mistake myself. I just tested it on my own package (a python) application and saw that the pid file created by the application had as owner "root". This changes to the correct user when I put su ${SYNOPKG_PKGNAME} in front of the start command in the start-stop-status script.

I installed your sickbeard_custom and indeed it is running as sickbeard_custom and not as root. So I have to find out what is wrong with my own package.

By the way your package does not create a shortcut in DSM desktop nor does it create a startup link in the package centre. I am not sure why but I shall try to find out.

Safihre commented 6 years ago

@BenjV Yes if you could find out why that shortcut fails, that would be great. Maybe the symlinks? We already found a problem with the JSON. It should be "allUsers": true and not "allUsers": "true". But I changed that locally and it didn't give me a link. Here's other stuff listed that's holding things up, including the shortcut thing: https://github.com/SynoCommunity/spksrc/pull/3123#issuecomment-364740501

As side note to confirm correct user for the listed packages:

sabtest@XPTest:/$ ls -ltr /volume1/@appstore/couchpotatoserver/var
-rw-rw-rw- 1 sc-couchpotatoserver couchpotatoserver     6 Dec 26 14:40 couchpotatoserver.pid
sabtest@XPTest:/$ ls -ltr /volume1/@appstore/sabnzbd/var
-rw------- 1 sc-sabnzbd sabnzbd    5 Feb 12 18:00 sabnzbd.pid
BenjV commented 6 years ago

I just found that "allUsers": true mistake by myself and changing that gave me a link in the package centre, but no icon on the dsm desktop.

IndoDanny commented 6 years ago

Please also support for ds3617xs. Almost nothing shows up in package manager

Safihre commented 6 years ago

@IndoDanny we are testing the packages, that's the whole point of this post... You can download the packages right here, you need the broadwell packages.

remirr commented 6 years ago

Can someone explain to me what it means when it asks me to give permission to use the "fork" for Sickbeard. Sorry new around GitHub... used to just installing packages.

IndoDanny commented 6 years ago

Thx for your answer! If I download the packages and install them manually, will they get updates with the package manager in the future, when everything is working ok again?

BenjV commented 6 years ago

@Safihre @ymartin59 I just found out why the package icons are not appearing on the DSM desktop You have to change the app/conf file.

Most application are not local but have a browser interface and so grantPrivilege has to be set to "all" and not to "local". Possible settings for "grantPrivilege" are:

  "all"
  "local"
  "ldap"

I would also suggest to add the "advanceGrantPrivilege" to this file. That will give the possibility to more settings via the DSM control panel, privilege settings. You can now grant access for an applicatie to for example a group of users.

So add the two lines below for all packages.

  "grantPrivilege": "all"
  "advanceGrantPrivilege" : true
IndoDanny commented 6 years ago

Is there someone who can answer my question above? I am still postponing manual install because I am still hesitating due to not knowing this..

ymartin59 commented 6 years ago

@IndoDanny I confirm that manual install does not break standard update process from Package Center.

IndoDanny commented 6 years ago

@ymartin59 Thank you for your answer. Then I will manually install the packages that are mentioned above in the list.

Safihre commented 6 years ago

Added Deluge to the list. In case of problems, please post: /volume1/@appstore/deluge/var/delug.log and /volume1/@appstore/deluge/var/delug_install.log

carlos-evolutive commented 6 years ago

I can't install CouchPotato, it says missing package python, but python is installed: DS718+/DSM 6.1.5-15254

screen shot 2018-02-15 at 18 41 19

Stanzilla commented 6 years ago

Did you use the Python package from this post? https://github.com/Safihre/spksrc/releases/tag/python-dsm61-new-all

carlos-evolutive commented 6 years ago

Thank you for your quick response, no, I had no Idea I had to manually install that python package... Which one do I have to use?

Stanzilla commented 6 years ago

The one for your platform, https://github.com/SynoCommunity/spksrc/wiki/Architecture-per-Synology-model

carlos-evolutive commented 6 years ago

I found it already it is apollolake, thank you very much! edit It still does not work though: screen shot 2018-02-15 at 18 53 00 screen shot 2018-02-15 at 18 53 15

Err0r0815 commented 6 years ago

@Safihre the Python package doesnt work with hydra and multi search. are there any fixes?

Safihre commented 6 years ago

What do you mean by "doesn't work"?

Err0r0815 commented 6 years ago

this message show up: https://user-images.githubusercontent.com/3859554/36272341-88a65292-1281-11e8-8ec8-b4c715c883d1.png

Safihre commented 6 years ago

@oneirik install the couchpotato version you downloaded from this page, not from the Community tab.

carlos-evolutive commented 6 years ago

mmm I suppose you mean:

screen shot 2018-02-15 at 19 01 49

from: https://github.com/Safihre/spksrc/releases?after=python-dsm61-new-all

I will give it a try, I really did not know where to find the packages to install them manually...

carlos-evolutive commented 6 years ago

It installed although with a warning: screen shot 2018-02-15 at 19 04 23

I actually do not know if I should see it in the Main Menu or anywhere, but the information about it is that it is running although it doesn't seem to appear anywhere in the UI.

Edit: I looked through the configuration and I am able to access the couchpotato configuration at the port 5050. So it is a partial success and it is working :-) thank you very much.

BenjV commented 6 years ago

The warning is there to tell you that you have to grant access rights to files needed by the package has to be done via the group sc-download. The same is true is you want to give another application or user access right to the files produced by the package. As of DSM 6 all packages are running under an hidden user that is also no in the group "users" so privileges has to be granted via the sc-download group.

The fact that there is no icon on the desktop is a glitch in all the test packages at the moment, as I explained in an earlier post in this topic.

remirr commented 6 years ago

As of now, I have decided to wait until the final packages are released. I have just spent 2 weeks re-setting up my NAS due to a filesystem error during a read/write operation and cannot do that again. I understand it was in no part the fault of these packages but I cannot take the risk.

Is there any timeline on when these will be final? Specifically, Python and Sabnzbd? I see that Sickbeard and Cloudstation already are. (I am referring to the about Apollolake Chipset - DS918+.)

carlos-evolutive commented 6 years ago

Thank you very much for the explanation! I'm just happy that there is a workaround to the issue.

ymartin59 commented 6 years ago

@Safihre you may remove MC from your list as I have just published it

IndoDanny commented 6 years ago

@remirr If you have the possibility, you could install Virtual DSM and test the packages there first

tnaseem commented 6 years ago

Can't get deluge running on my Synology 918+. Each time I run the package, it comes back with "Failed to run the package service."

I installed the Python package python_apollolake-6.1_2.7.14-17, followed by deluge_apollolake-6.1_1.3.15-9.

Log files attached, although deluge.log doesn't update each time I try to run the package.

deluge.log deluge_install.log

Safihre commented 6 years ago

@tnaseem could you maybe also share /var/log/synpopkg.log? Only need the last parts of it, from yesterday/today.

tnaseem commented 6 years ago

Here you go... synopkg.log

From that log it looks like it can't locate the start-stop-status. The scripts folder is also missing in the /volume1/appstore/deluge folder. Could this be the problem?

As a side note, if I run deluge-web directly from the command line (in volume1/appstore/deluge/env/bin), the web interface does run. However, I get the following warning as it runs...

:0: UserWarning: You do not have a working installation of the service_identity module: 'No module named pyasn1.codec.der.decoder'. Please install it from https://pypi.python.org/pypi/service_identity and make sure all of its dependencies are satisfied. Without the service_identity module, Twisted can perform only rudimentary TLS client hostname verification. Many valid certificate/hostname mappings may be rejected.

Then when I attempt to start the daemon in the web ui Connection Manager, the following error pops up in the Terminal:

[ERROR ] 19:43:03 client:585 Deluge cannot find the 'deluged' executable, it is likely that you forgot to install the deluged package or it's not in your PATH. [ERROR ] 19:43:03 json_api:240 Error calling method web.start_daemon [ERROR ] 19:43:03 json_api:241 [Errno 2] No such file or directory Traceback (most recent call last): File "/volume1/@appstore/deluge/env/lib/python2.7/site-packages/deluge-1.3.15-py2.7.egg/deluge/ui/web/json_api.py", line 232, in _handle_request result = self._exec_local(method, params, request) File "/volume1/@appstore/deluge/env/lib/python2.7/site-packages/deluge-1.3.15-py2.7.egg/deluge/ui/web/json_api.py", line 199, in _exec_local return meth(*params) File "/volume1/@appstore/deluge/env/lib/python2.7/site-packages/deluge-1.3.15-py2.7.egg/deluge/ui/web/json_api.py", line 863, in start_daemon client.start_daemon(port, get_config_dir()) File "/volume1/@appstore/deluge/env/lib/python2.7/site-packages/deluge-1.3.15-py2.7.egg/deluge/ui/client.py", line 588, in start_daemon raise e OSError: [Errno 2] No such file or directory

Hope this extra info helps.

Safihre commented 6 years ago

What if you run the daemon from the command line?

sudo env PYTHON_EGG_CACHE=/volume1/@appstore/env/cache /volume1/@appstore/env/bin/deluged
tnaseem commented 6 years ago

I presume you meant:

sudo env PYTHON_EGG_CACHE=/volume1/@appstore/deluge/env/cache /volume1/@appstore/del uge/env/bin/deluged

I get the same error on the command line:

:0: UserWarning: You do not have a working installation of the service_identity module: 'No module named pyasn1.codec.der.decoder'. Please install it from https://pypi.python.org/pypi/service_identity and make sure all of its dependencies are satisfied. Without the service_identity module, Twisted can perform only rudimentary TLS client hostname verification. Many valid certificate/hostname mappings may be rejected.

And the web interface this time is not running. "Site cannot be reached" ERR_CONNECTION_REFUSED.

Safihre commented 6 years ago

This is the Deluge daemon? Not the web-daemon, but the actual program?

tnaseem commented 6 years ago

Yes. Sorry.

If I then run deluge-web, the daemon still shows as offline.

clipboard-20180216

Hitting Start Daemon there, I get:

[ERROR ] 20:16:52 client:585 Deluge cannot find the 'deluged' executable, it is likely that you forgot to install the deluged package or it's not in your PATH. [ERROR ] 20:16:52 json_api:240 Error calling method web.start_daemon [ERROR ] 20:16:52 json_api:241 [Errno 2] No such file or directory Traceback (most recent call last): File "/volume1/@appstore/deluge/env/lib/python2.7/site-packages/deluge-1.3.15-py2.7.egg/deluge/ui/web/json_api.py", line 232, in _handle_request result = self._exec_local(method, params, request) File "/volume1/@appstore/deluge/env/lib/python2.7/site-packages/deluge-1.3.15-py2.7.egg/deluge/ui/web/json_api.py", line 199, in _exec_local return meth(*params) File "/volume1/@appstore/deluge/env/lib/python2.7/site-packages/deluge-1.3.15-py2.7.egg/deluge/ui/web/json_api.py", line 863, in start_daemon client.start_daemon(port, get_config_dir()) File "/volume1/@appstore/deluge/env/lib/python2.7/site-packages/deluge-1.3.15-py2.7.egg/deluge/ui/client.py", line 588, in start_daemon raise e OSError: [Errno 2] No such file or directory

Safihre commented 6 years ago

There is a problem with the Python package I see now. It's an older version that lacks pyasn1 and start-stop-daemon. Will report back when I have recompiled. Travis is not listening to me right now..

IndoDanny commented 6 years ago

I do not see git anymore in the package center for ds3617xs in my VM DSM. Has it been removed? Also not in the list above.

BenjV commented 6 years ago

If I look at the errorlog from @tnaseem then there is something wrong with the deluge package and not with python. The virtual environment which hold the site packages is addressed with "/volume1//env/lib/python2.7/.....

The double slash after volume1 (//) is wrong and should be just one slash (/).

Safihre commented 6 years ago

Thanks for point that out, the whole path is wrong. It should include the @appstore location etc. Will investigate.

BenjV commented 6 years ago

Instead of "volume1/ " you should use ${SYNOPKG_PKGDEST} in the script.

Safihre commented 6 years ago

@BenjV it's weird, I do that everywhere: https://github.com/SynoCommunity/spksrc/pull/3153/files Or do you see a place where I forgot?

Safihre commented 6 years ago

@tnaseem could you download the newly compiled Python package here and then re-install the Deluge package and see if it runs then? (regularly from the package center, not command line) Indeed there was also a problem with pyasn1.

tnaseem commented 6 years ago

@Safihre Just installed that version and deluge ran fine this time :) Many thanks for sorting this!

I did have issues trying to download anything, but it turned out to be a permissions problem on the Download folder. It looks like deluge applies sc-deluge:deluge ownership to the downloads. So, I just gave that user (sc-deluge) RW access to the download folder, which seems to have fixed it. Funny thing is, it did write to the folder and only complained about a permissions error half way through the download, having already written into the folder. Very strange. I've included the deluge.log if it's any use:

deluge.log deluge-web.log

Also noticed that the env folder has root:root ownership. Is this correct?

clipboard-20180218

The deluge-web.log has errors which state it doesn't have write access to the cache within, so wondering if this is the reason. Will change it and see what happens.

Apart from that, all seems to be working great! Many thanks for persevering with this. :)

UPDATE: I changed the ownership of the env folder to sc-deluge:deluge and it no longer complains about not having write access to the cache folder. Hope this helps.

Stanzilla commented 6 years ago

The whole user/groups situation is big mess atm :/ especially when mixing released packages with the testing ones here.