search

SynoCommunity / spksrc

Cross compilation framework to create native packages for the Synology's NAS
https://synocommunity.com
Other
3.02k stars 1.23k forks source link

DSM 7 support framework design #4215

Closed ymartin59 closed 3 years ago

ymartin59 commented 3 years ago

I propose to collaboratively edit this issue description (with comments for discussions about points) so that to prepare how framework has to evolve.

Objectives:

Design proposals:

To investigate

Minimal requirements for DSM7

publicarray commented 3 years ago

@th0ma7 At the moment anything under 7.0 has the following output (I use adminer as my test package):

os_min_ver="3.1-1594"
firmware="3.1-1594"

So I don't see the difference for why 6.1 over 6.0? Maybe this needs to more strict, I don't know.

th0ma7 commented 3 years ago

That is probably caused by mk/spksrc.spk.mk where this is defined (or should be revamed I guess for 6.1-xxxx):

SPK_ARCH = $(TC_ARCH)
SPK_NAME_ARCH = $(ARCH)
SPK_TCVERS = $(TCVERSION)
ARCH_SUFFIX = -$(ARCH)-$(TCVERSION)
TC = syno$(ARCH_SUFFIX)
else
SPK_ARCH = noarch
SPK_NAME_ARCH = noarch
SPK_TCVERS = all
OS_MIN_VER = 3.1-1594
FIRMWARE = $(OS_MIN_VER)
endif
hgy59 commented 3 years ago

@th0ma7 no, no, no To clarify: noarch packages are backwards compatible with DSM 3.1. Only with DSM7 there is a new minimal Version of 7.0-4000 required and this will last for all future versions (until a new minimal version will be introduced, may be in the far future).

And I have investigated in the Github Action code that we can use noarch and must not define noarch-3.1 (or what version ever) for the pre DSM7 versions, and to distinguish the generated spk files with -all- and -dsm7- without the use of a version number.

~BTW (@ymartin59) the support of noarch-dsm7 packages will need an enhancement in synorepo.~

The actual code in mk/spksrc.spk.mk is now:

ifneq ($(ARCH),)
SPK_ARCH = $(TC_ARCH)
SPK_NAME_ARCH = $(ARCH)
SPK_TCVERS = $(TCVERSION)
ARCH_SUFFIX = -$(ARCH)-$(TCVERSION)
TC = syno$(ARCH_SUFFIX)
else
SPK_ARCH = noarch
SPK_NAME_ARCH = noarch
ifeq ($(shell expr "$(TCVERSION)" \>= 7.0),1)
SPK_TCVERS = dsm7
OS_MIN_VER = 7.0-4000
else
SPK_TCVERS = all
OS_MIN_VER = 3.1-1594
endif
ARCH_SUFFIX = -$(SPK_TCVERS)
FIRMWARE = $(OS_MIN_VER)
endif

The only cleanup that shold be done is the removement of FIRMWARE for DSM >= 6.0 in the INFO file.

hgy59 commented 3 years ago 

In file included from /spksrc/toolchain/syno-x64-7.0/work/x86_64-pc-linux-gnu/x86_64-pc-linux-gnu/include/c++/7.3.0/cmath:45:0,
                 from /spksrc/spk/ffmpeg/work-x64-7.0/libaom-gitbb35ba9148543f22ba7d8642e4fbd29ae301f5dc/third_party/libwebm/mkvparser/mkvparser.cc:18:
/spksrc/spk/ffmpeg/work-x64-7.0/libaom-gitbb35ba9148543f22ba7d8642e4fbd29ae301f5dc/third_party/libwebm/mkvparser/mkvparser.cc: In function ‘bool mkvparser::__builtin_isnan(double)’:
/spksrc/spk/ffmpeg/work-x64-7.0/libaom-gitbb35ba9148543f22ba7d8642e4fbd29ae301f5dc/third_party/libwebm/mkvparser/mkvparser.cc:35:45: error: ‘__builtin_isnan’ is not a member of ‘std’
 inline bool isnan(double val) { return std::isnan(val); }

@publicarray I have fixed this cmath errors in #4393 for cross/libicu.

To work around an error in gcc 7.3.0 it can be fixed by adding some flages like:

ADDITIONAL_CPPFLAGS = -D_GLIBCXX_USE_C99_MATH -D_GLIBCXX_HAVE_OBSOLETE_ISNAN -D_GLIBCXX_HAVE_OBSOLETE_ISINF
ADDITIONAL_CXXFLAGS = $(ADDITIONAL_CPPFLAGS)
th0ma7 commented 3 years ago

@hgy59 gotcha, although I doubt that they would really work on OS_MIN_VER = 3.1-1594 but that's another story.

As for the _GLIBCXX_USE_C99_MATH and al, shouldn't we add them by default in mk/meson|cmake| if TC_GCC = 7.3.0 to make things simpler?

hgy59 commented 3 years ago

The publish of noarch-7.0 build works as expected (After the creation of new Firmware Version 7.0-4000). Just published and activated adminer 4.7.6-3 for 3.1 and 7.0. So this is the first DSM7 package in the SynoCommunity Repository and will be available within 24 to 48 hours in the DSM Package-Center.

hgy59 commented 3 years ago

As for the _GLIBCXX_USE_C99_MATH and al, shouldn't we add them by default in mk/meson|cmake| if TC_GCC = 7.3.0 to make things simpler?

@th0ma7 when this works for ffmpeg too, this is a good proposal.

publicarray commented 3 years ago

@hgy59 did you increase the revision number before publishing? I did not up any revision numbers on the dsm7 branch, since there were still changes to various other files. FYI the Adminer package now requires apache2.4. Sorry for the confusion

hgy59 commented 3 years ago

@hgy59 did you increase the revision number before publishing? I did not up any revision numbers on the dsm7 branch, since there were still changes to various other files. FYI the Adminer package now requires apache2.4. Sorry for the confusion

The current revision (-3) was not published yet. So I uploaded both. I have only apache 2.2 on my DS218+ with DSM7 and it runs without any problem (MySQL tested only).

hgy59 commented 3 years ago

@hgy59 did you increase the revision number before publishing? I did not up any revision numbers on the dsm7 branch, since there were still changes to various other files. FYI the Adminer package now requires apache2.4. Sorry for the confusion

@publicarray I was on the wrong way. Just discoverd an old (pre DSM7) installation of adminer in /var/services/web/adminer that was running and not the manually installed dsm7 version. After manually deletion of /var/services/web/adminer the running instance is gone and I can see that the manually installed packages does not work. Have you already evaluated how to install simple web apps in DSM7?

I just deleted the published DSM 7 adminer package.

publicarray commented 3 years ago

@hgy59 I've added SPK_DEPENDS to avoid confusion: https://seby.io/download/synology-7.0/adminer_noarch-dsm7_4.7.6-3.spk https://github.com/publicarray/spksrc/commit/3dfb3d40d303a7d43ec64984d063910f3a6fd164

Yea it works on my machine:tm: The new folder should have been /var/services/web_packages/adminer maybe the migration is broken.

If you continue to have problems can I suggest reinstalling web station?

Was there a reason packages still use Apache 2.2? I'm actually surprised Synology still ships it. From apache's website:

The Apache HTTP Server Project had long committed to provide maintenance releases of the 2.2.x flavor through June of 2017. The final release 2.2.34 was published in July 2017, and no further evaluation of bug reports or security risks will be considered or published for 2.2.x releases.

FYI same for php 5.6, 7.0 and 7.2: https://www.php.net/supported-versions.php (7.4 is the only realistic version to support right now)

hgy59 commented 3 years ago

You cannot uninstall apache 2.2 without accepting the following warning in DSM7:

Uninstalling this package will cause abnormalities in the following functions. 
Do you mant to continue?
Some of the Web Stations's services will not be able to function properly, including default
back-end, personal website, and Virtual Host.

Hopefully apache 2.2 (and other outdated packages) will be removed in the final DSM7...

publicarray commented 3 years ago

Does the warning show up even when you have both apache versions installed? I don't use those functions but I can confirm that the default server works with just apache2.4 installed.

publicarray commented 3 years ago

Oh in DSM6 the checkbox for personal website is disabled until I install old versions of php and apache. DSM7 allows you to mix and match php/apache versions

hgy59 commented 3 years ago

Does the warning show up even when you have both apache versions installed? I don't use those functions but I can confirm that the default server works with just apache2.4 installed.

Yes, I have both versions (2.2 and 2.4 installed) and you can uninstall apache2.4 without warning.

publicarray commented 3 years ago

I think that warning only shows up if you have a 'personal website or Virtual Host' setup from DSM6. I accepted the warning once but it no longer shows up when I uninstall 2.2.

publicarray commented 3 years ago

What is configured for your default server? Screenshot from 2021-01-19 19-35-12 Screenshot from 2021-01-19 19-35-43

publicarray commented 3 years ago

If I install 2.2 and set the default server to 2.2 I get the warning. If I set it to 2.4 and uninstall 2.4 I also get the warning. On Nginx I don't. I guess it's to prevent unexpected errors. I which the warning was clearer though.

To sum up just update the setting for your default server to 2.4 or nginx (unless you explicitly require 2.2 for some reason).

hgy59 commented 3 years ago

Yes you have to remove/update all the Web Service Portal configurations and Script Language Settings referencing outdated apache and php versions. Then you can uninstall the appropriate packages without warnings.

digitalbox94 commented 3 years ago

Another interesting source of information regarding DSM7 package : https://github.com/SynologyOpenSource/ExamplePackages

publicarray commented 3 years ago

Thanks @digitalbox94 but unfortunately it's no very useful. It only demonstrates the new /usr/local/bin/ linking. We already implemented it on the dsm7 branch.

digitalbox94 commented 3 years ago

Ok I've played a little bit with the Icecast package and DSM7 and here are my findings :

By doing some manual tricks directly on the NAS (configuration files & directory creation) I've got Icecast server running fine on my DS716+II with an ezstream flow (ezstream is also embedded in this package). It means it's a good starting point before having a package 100% DSM 7 compatible.

I will continue my investigation but I you have any comments/advice to share for this DSM 7 topic they are welcome.

publicarray commented 3 years ago

Thanks @DigitalBox98, fixed in 9dd89e3df0fc7fa36469507ac07a0e08c5161f38

Most packages have old assumptions on the file layout and permissions so most will need to be manually migrated.

digitalbox94 commented 3 years ago

Many thanks, the start of the server now works without any manual fix on DSM 7 ! :)

Regarding the "service-setup" file, I have identified an issue to fix regarding the "ice cast.xml" CFG-FILE where the correct line is : CFG_FILE="${SYNOPKG_PKGDEST}/var/icecast.xml" It will help to have correct login/password for the admin part

icecast
DigitalBox98 commented 3 years ago

I would like to test the below "proof-of-concept" for DSM7 (but also for DSM6) :

I'm actually fine tuning the solution on DSM7 because sometime the admin page is not available via DSM7 (access to /var/services/web_packages/imagemagick), and sometime it works (need to investigate further). Putting the HTML/PHP pages on another directory (like previous method on DSM6 ie /var/services/web/imagemagick resolve 100% the previous issue on DSM7.

If I would like to propose a PR for this for DSM7, but what is the best below :

Another point : actually I have created & tested it on DSM7 with a branch created from DSM7 branch Does it make senses to propose it for DSM6 with a branch "imagemagick" created from master branch ? (ie would the framework will generate the correct version for DSM6)

Let me know if you are interested to test this (at least the Syno AUTH webapi can be useful for other packages).

EngMarc commented 3 years ago

As for the _GLIBCXX_USE_C99_MATH and al, shouldn't we add them by default in mk/meson|cmake| if TC_GCC = 7.3.0 to make things simpler?

@th0ma7 when this works for ffmpeg too, this is a good proposal. @th0ma7 is the a new ffmpeg for DSM 7 yet?

erdoukki commented 2 years ago

But if you are 3rdparty developer, you can still use root privilege.

This part was removed from the PDF ?

synology has published a beta release of the Development documentation DSM_Developer_Guide_7_0_Beta.pdf. It is newer than the preview version.

One interesting difference I found is for 3rdparty developers:

1. Force lower privilege for package

All packages should provide conf/privilege with package in run-as explicitly. Any privileged operation should be accomplished via resource worker. But if you are 3rdparty developer, you can still use root privilege.