zenny commented 4 years ago

Package Name

GCCemacs (http://akrl.sdf.org/gccemacs.html)

Linux-related link

https://www.emacswiki.org/emacs/GccEmacs http://akrl.sdf.org/gccemacs.html#org248e6b1 http://akrl.sdf.org/gccemacs.html#orgec98bb2

Practical Rationale

This (slowness) comes up often. The first thing folks fresh off the boat from other editors will notice is that Emacs has a low threshold for performance issues. It doesn’t take much to get it to scroll like molasses... There is an unfortunate but necessary adjustment of expectations new users must undergo, when they adopt Emacs. Doom has inherited this curse. It’s raison d’etre is to improve the situation, but I can only go so far, especially if you choose to enable all the most expensive features. You will unavoidable find cases where Emacs is just slow.

Reference: https://github.com/hlissner/doom-emacs/blob/fd3b89854eb13ddb2c8ecc2da68a9cbb0dcdb836/docs/faq.org#why-is-emacsdoom-slow

One of the things that irritated me the most about emacs was how slow it was when the text in one of the files got very long. If a line (paragraphs are considered one line in emacs) gets too long, emacs starts to slow down till you start a new paragraph. As someone who does writing for a living, taking notes, inserting citations, this starts to get very annoying. Upon conversation with the creator of DOOM emacs, I installed GCC emacs. There are a bunch of different ways to do this, but I followed the amazing gist written by mjlbach. When I ran into trouble he took time out to help me figure it out. Much thanks to him. As a result, my emacs runs as fast as it ever has.

Reference: https://github.com/sunnyhasija/DOOMEmacs/blob/master/README.org#move-to-gcc-emacs

Manual installation in Void

Tried to install in voidlinux without success as follows:

$ git clone https://git.savannah.gnu.org/emacs.git
$ cd emacs
$ git checkout feature/native-comp
$ ./autogen.sh
$ ./configure --with-nativecomp
...
configure: error: Installed libgccjit has failed passing the smoke test.
You can verify it yourself compiling:
<https://gcc.gnu.org/onlinedocs/jit/intro/tutorial01.html>.
Please report the issue to your distribution.
Here instructions on how to compile and install libgccjit from source:
<https://gcc.gnu.org/wiki/JIT>.

ATTN: @leahneukirchen (the emacs package maintainer)

leahneukirchen commented 4 years ago

Hi,

we only package released software, and gccemacs is still in heavy development. The build probably fails, because we don't have libgccjit packaged at all, so you would need to start with that.

WONTFIXing for now, but feel free to ask on this thread for help.

zenny commented 4 years ago

Hi,

we only package released software, and gccemacs is still in heavy development. The build probably fails, because we don't have libgccjit packaged at all, so you would need to start with that.

@leahneukirchen Thanks. As you are onto academics yourself as seen in https://leahneukirchen.org/talks/, GccEmacs could be of your interest?!

Anyway, ould you mind briefing the ideal void way to build libgccjit as well as GccEmacs without conflicting other void file structures for maintaining compatibility with void Emacs packages in the future?

Two resources I found useful are:

https://gcc.gnu.org/wiki/JIT which points to https://gcc.gnu.org/onlinedocs/jit/internals/index.html#working-on-the-jit-library for non-standard linux distros
https://www.emacswiki.org/emacs/GccEmacs#toc5 (for manual build of GccEmacs)

WONTFIXing for now, but feel free to ask on this thread for help.

Appreciate that.

Cheers, and stay safe, /z

Javyre commented 4 years ago

Hey @zenny, here is a template for libgccjit that I got working today on my machine (x86-64 glibc). I hope this could serve as a starting point for a real libgccjit template for void:

https://gist.github.com/Javyre/2578176bf09638733c08586e654fb04d

I used this as reference: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=libgccjit

I tried modifying the existing gcc template to have a subpackage for libgccjit, but I couldn't get it to work as I'd keep getting errors telling me to add -fPIC although I had already added it to the flags. Either way, it is not recommended to use the same ./configure call for the gcc compilers as for libgccjit since -fPIC and --enable-host-shared come at a performance cost: https://gcc.gnu.org/onlinedocs/gcc-8.3.0/jit/internals/index.html#packaging-notes

zenny commented 4 years ago

Hey @zenny, here is a template for libgccjit that I got working today on my machine (x86-64 glibc). I hope this could serve as a starting point for a real libgccjit template for void:

https://gist.github.com/Javyre/2578176bf09638733c08586e654fb04d

I used this as reference: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=libgccjit

I tried modifying the existing gcc template to have a subpackage for libgccjit, but I couldn't get it to work as I'd keep getting errors telling me to add -fPIC although I had already added it to the flags. Either way, it is not recommended to use the same ./configure call for the gcc compilers as for libgccjit since -fPIC and --enable-host-shared come at a performance cost: https://gcc.gnu.org/onlinedocs/gcc-8.3.0/jit/internals/index.html#packaging-notes

@Javyre Thanks for help, I shall test and get back with my results.

Meanwhile, would you mind taking some time to explain the steps you took to build libgccjit? Appreciate that. Fyi, I am new to xbps package build.

Cheers and stay safe, /z

Javyre commented 4 years ago

I basically just looked at the arch packagebuild and the packaging notes on the gccjit website and I transcribed the parts of the gcc void template that were relevant to libgccjit and x86-64.

I'm not sure whether libgccjit should have it's own package template or be yet another subpackage of gcc. I think the best choice would be to have it as a subpackage of gcc and add an extra ./configure and make call in do_configure() and do_build() so that libgccjit could be built with --enable-host-shared and not the rest of the packages.

@zenny Assuming you're familiar enough with shell scripts, I suggest just starting and Ctrl-Fing through the manual whenever you're confused. (worked for me)

Also, I updated the gist with a terrible (but working) template for gccemacs that I am not proud of. Worth noting:

I had to add libgccjit.so.0 libgccjit-9.3.0_6 to common/shlibs to get my gccemacs to install since xbps couldn't find the provider of that .so.
I had to disable the hooks/post-install/11-pkglint-elf-in-usrshare.sh hook since gccemacs (at least for the time being) stores eln files in /usr/share. If I understand correctly, this was fine before since they would be el or elc files which are portable, but with native-comp, now those files are actual binaries. This would definitely need to be fixed and discussed with andrea or another gccemacs dev. (It is unclear from the mailing list what the eln search path is or where it is even defined)

ap4y commented 4 years ago

I have made packages for both libgccjit and emacs-28.0.50 with nativecomp build option. You can grab patches here:

ericonr commented 4 years ago

@ap4y is this currently working on musl? I needed this patch:

Fix poisoned calloc errors when building libgccjit

---
 gcc/jit/jit-playback.c  | 4 ++--
 gcc/jit/jit-recording.c | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/gcc/jit/jit-playback.c b/gcc/jit/jit-playback.c
index b74495c58..070463d71 100644
--- a/gcc/jit/jit-playback.c
+++ b/gcc/jit/jit-playback.c
@@ -18,6 +18,8 @@ You should have received a copy of the GNU General Public License
 along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */

+#include <pthread.h>
+
 #include "config.h"
 #include "system.h"
 #include "coretypes.h"
@@ -40,8 +42,6 @@ along with GCC; see the file COPYING3.  If not see
 #include "gcc.h"
 #include "diagnostic.h"

-#include <pthread.h>
-
 #include "jit-playback.h"
 #include "jit-result.h"
 #include "jit-builtins.h"
diff --git a/gcc/jit/jit-recording.c b/gcc/jit/jit-recording.c
index a332fe875..7184a481a 100644
--- a/gcc/jit/jit-recording.c
+++ b/gcc/jit/jit-recording.c
@@ -18,6 +18,8 @@ You should have received a copy of the GNU General Public License
 along with GCC; see the file COPYING3.  If not see
 <http://www.gnu.org/licenses/>.  */

+#include <pthread.h>
+
 #include "config.h"
 #include "system.h"
 #include "coretypes.h"
@@ -25,8 +27,6 @@ along with GCC; see the file COPYING3.  If not see
 #include "pretty-print.h"
 #include "toplev.h"

-#include <pthread.h>
-
 #include "jit-builtins.h"
 #include "jit-recording.h"
 #include "jit-playback.h"
-- 
2.28.0

zenny commented 3 years ago

Tried to compile libgccjit first with the patches provided by @ap4y in https://github.com/void-linux/void-packages/issues/24240#issuecomment-699700450, but failed with the following error:

...
checking whether to enable maintainer-specific portions of Makefiles... no
configure: creating ./config.status
config.status: creating Makefile
mv: cannot move './confxaxD9R/out' to 'Makefile': No such file or directory
config.status: error: could not create Makefile
=> ERROR: libgccjit-9.3.0_1: do_configure: '${configure_script} ${configure_args}' exited with 1
=> ERROR:   in do_configure() at common/build-style/gnu-configure.sh:7

Same with while running ./xbps-src pkg emacs:

...
=> emacs-28.0.50.e5b052d_1: running do_configure ...
/void-packages/srcpkgs/emacs/template: line 60: ./configure: No such file or directory
=> ERROR: emacs-28.0.50.e5b052d_1: do_configure: './configure --without-x $(vopt_with
dbus) ${configure_args}' exited with 127
=> ERROR:   in do_configure() at srcpkgs/emacs/template:60

I am on gcc and used proot, fyi. Thanks

Javyre commented 3 years ago

Hey, I managed to make @ap4y 's changes work for me: https://github.com/void-linux/void-packages/compare/master...Javyre:emacs-nativecomp

The libgccjit template is unchanged from his, but his emacs template was missing the compiled .eln and .pdmp files in /usr/lib/. (I also updated the git sha to the latest on the nativecomp branch)

To compile emacs I use ./xbps-src -o ~gtk2,~x11,cairo,harfbuzz,nativecomp pkg emacs

zenny commented 3 years ago

Hey, I managed to make @ap4y 's changes work for me: master...Javyre:emacs-nativecomp

The libgccjit template is unchanged from his, but his emacs template was missing the compiled .eln and .pdmp files in /usr/lib/. (I also updated the git sha to the latest on the nativecomp branch)

To compile emacs I use ./xbps-src -o ~gtk2,~x11,cairo,harfbuzz,nativecomp pkg emacs

Thanks @Javyre . I pulled your revised emacs template and appended related files, yet no go :-(

Reports emacs-common missing and thereafter some missing support with libgccjit and finally fails to create Makefile

$ ./xbps-src -o ~gtk2,~x11,cairo,harfbuzz,nativecomp pkg emacs
...
   [runtime] emacs-common-28.0.50.3e3843512bfae0b7a532f633e45d4c140807ec9b_1: not found (subpkg, ignored)
   [runtime] emacs-common-28.0.50.3e3843512bfae0b7a532f633e45d4c140807ec9b_1: not found (subpkg, ignored)
   [runtime] emacs-common-28.0.50.3e3843512bfae0b7a532f633e45d4c140807ec9b_1: not found (subpkg, ignored)
   [runtime] desktop-file-utils-0.26_1: found (https://alpha.de.repo.voidlinux.org/current)
   [runtime] hicolor-icon-theme-0.17_2: found (https://alpha.de.repo.voidlinux.org/current)
...
checking for isl 0.15 or later... no
required isl version is 0.15 or later
The following languages will be built: c,jit
*** This configuration is not supported in the following subdirectories:
     gnattools gotools target-libada target-libhsail-rt target-libstdc++-v3 target-libphobos target-zlib target-libbacktrace target-libgfortran target-libgo target-libffi target-libobjc target-liboffloadmic target-libitm target-libsanitizer target-libvtv
    (Any other directories should still work fine.)
checking for default BUILD_CONFIG...
checking for --enable-vtable-verify... no
checking for bison... no
checking for byacc... no
checking for yacc... no
checking for bison... no
checking for gm4... no
checking for gnum4... no
checking for m4... no
checking for flex... no
checking for lex... no
checking for flex... no
checking for makeinfo... no
/builddir/gcc-9.3.0/missing: line 81: makeinfo: command not found
checking for expect... no
checking for runtest... no
...
checking whether to enable maintainer-specific portions of Makefiles... no
configure: creating ./config.status
config.status: creating Makefile
mv: cannot move './conf30zSA8/out' to 'Makefile': No such file or directory
config.status: error: could not create Makefile
=> ERROR: libgccjit-9.3.0_1: do_configure: '${configure_script} ${configure_args}' exited with 1
=> ERROR:   in do_configure() at common/build-style/gnu-configure.sh:7

Javyre commented 3 years ago

Looks like it's missing texinfo when building libgccjit. I just pushed to my branch again, this should be fixed.

(make sure to ./xbps-src clean libgccjit)

zenny commented 3 years ago

Looks like it's missing texinfo when building libgccjit. I just pushed to my branch again, this should be fixed. (make sure to ./xbps-src clean libgccjit)

Not yet!

Even after appending texinfo in the https://github.com/void-linux/void-packages/blob/b378df45dbf113ac181aa5c9a45143dfb5974f2a/srcpkgs/libgccjit/template#L12 and the cleaned the source, still fails to build and compile:

...
   [runtime] emacs-common-28.0.50.3e3843512bfae0b7a532f633e45d4c140807ec9b_1: not found (subpkg, ignored)
   [runtime] emacs-common-28.0.50.3e3843512bfae0b7a532f633e45d4c140807ec9b_1: not found (subpkg, ignored)
   [runtime] emacs-common-28.0.50.3e3843512bfae0b7a532f633e45d4c140807ec9b_1: not found (subpkg, ignored)
...
bsdtar: Failed to set default locale
...
checking whether to enable maintainer-specific portions of Makefiles... no
configure: creating ./config.status
config.status: creating Makefile
mv: cannot move './conf8CQSPW/out' to 'Makefile': No such file or directory
config.status: error: could not create Makefile
=> ERROR: libgccjit-9.3.0_1: do_configure: '${configure_script} ${configure_args}' exited with 1
=> ERROR:   in do_configure() at common/build-style/gnu-configure.sh:7

Even after installing isl15 and isl15-devel which is at v 0.19 :

[*] isl15-0.19_1             Integer Set Library
[*] isl15-devel-0.19_1       Integer Set Library - development files

It gives out the following error (thogh does not seem inevitable):

checking for isl 0.15 or later... no
required isl version is 0.15 or later
The following languages will be built: c,jit
*** This configuration is not supported in the following subdirectories:
     gnattools gotools target-libada target-libhsail-rt target-libstdc++-v3 target-libphobos target-zlib target-libbacktrace target-libgfortran target-libgo target-libffi target-libobjc target-liboffloadmic target-libitm target-libsanitizer target-libvtv
    (Any other directories should still work fine.)
checking for default BUILD_CONFIG...

zenny commented 3 years ago

Thanks to user natrys in the voidlinux IRC for pointing out not to use proot. Actually using proot was causing the compilation and build issue stated above.

zenny commented 3 years ago

Reopened because the updgrade to the latest version is a rolling issue with many changes to feature/native-comp and libgccjit upstream.

lane-core commented 3 years ago

If this package is being considered for inclusion, particularly after the announcement from the gccemacs team that libgccjit is no longer considered alpha status, I have created a package template based on the newest gcc template which is working well for me in my emacs-nativecomp template. Should I submit a pull request?

ericonr commented 3 years ago

Hm, do you see much usefulness for it as a standalone package? I guess gccemacs is going to take a while to be released in any stable form :/

ericonr commented 3 years ago

@lane-brain I think it should be ok to package it, just not sure how easy to test it will be :P

lane-core commented 3 years ago

@ericonr I considered building it as a subpackage of gcc similar to the treatment of other frontends like gcc-ada, but the due to the need to flip the --enable-host-shared flag you have to do a whole separate build, so it's not exactly streamlined into the build process as it stands. The only files provided by this package are the soname and build headers.

To cover the bases here:

The JIT frontend is not quite API stable yet but getting there. This package is provided by debian sid and has been present in ubuntu for some time. The package has been working well on my end used alongside my home-rolled emacs nativecomp+pgtk template.
This frontend does have built in tests according to the documentation. I built them into the docheck function so I've ran the tests and attached the results, which look good. Test suite took me about 5 mins on my i5-10210U. (note: to run the test suite, the c++ frontend needs to be enabled in the build)._ jit.log
RE: usefulness, libgccjit has a fairly wide user base under doom-emacs, although admittedly the library seems to be exclusvely provided for the emacs use case. Practically speaking, libgccjit is decently vetted and used as a daily driver for what is a critical piece of software for many users. The doom-emacs repo numbers nearly 10k users and uses libgccjit by default if it is present.
My only reservation with regards to packaging it as an official package is that void does not provide an emacs-nativecomp build. Although if I do recall, a regular emacs build should be able to run .elc files compiled from straight.el.

This is one of the more ambitious packages I've rigged together as I've picked up xbps-src over the last months, and I had to do my own research as far as what build flags from the void gcc package would be appropriate to include in this one, referencing other distro build scripts for the library and keeping in mind what the build template isn't compiling (rendering certain build flags superfluous). Given my inexperience, the package could use some peer review.

ericonr commented 3 years ago

From when this last came up on IRC, I believe the preferred direction was to have a separate template for it indeed, so you went on a good route :) We want to avoid even more piling up in the GCC package, since any fix to the components inside requires a full rebuild.

What will be interesting to find out is how easily or not it works with cross builds :p

We can't have a gccemacs package; it's not released yet, and afaik the authors don't consider it stable.

I think you have a bit of a misunderstanding regarding what doom-emacs is. It's a framework that sits atop an already built emacs. Choosing to use/support native compilation in emacs is a build time choice, and doom emacs goes on top of that, with or without the feature enabled, though it can certainly take advantage of the speed up. I use doom-emacs locally, but I definitely don't use gccemacs yet.

Although if I do recall, a regular emacs build should be able to run .elc files compiled from straight.el.

From what I understood, this definitely required running the gccemacs branch, and shouldn't work with our current build of emacs. I will be pleasantly surprised if that's not the case.

Feel free to open a PR ;)

lane-core commented 3 years ago

sounds good :)

Submitted at PR #28785

xianwenchen commented 3 years ago

I have been trying to build libgccjit and gccemacs manually.

I was able to build libgccjit.

However, I could not build gccemacs. I think the problem was that the configure script with emacs assumed that libgccjit was installed in the /usr. In my case, I installed libgccjit to /home.

I have stopped trying to get gccemacs to build manually at the moment.

From the status of #28785, I expect that libgccjit will be available from Void's official repository very soon. I plan to wait til then and try to build gccemacs manually.

I think at the moment, building gccemacs manually seems to be the best option.

lulcat commented 3 years ago

in my distro I build both libjit and what I call libgccjit , (I guess it could have been gccjit). Either way, yes you A. add it to --enable-languages=jit. howevr, you should build it independently from the rest with only --enable-languages=jit.

This is because unlike the rest, you want to also pass --enable-host-shared :)

This is with glibc. I am currently building musl and jit fails with poison calloc thing, so I suppose a patch here and ye. Hope that helps.. (Regardless of being musl or not, the building independently advice stands).

rawhuul commented 3 years ago

Hey,there! When Emacs 28 will land in main repo.??

leahneukirchen commented 3 years ago

Certainly not before a release tarball is provided.

soanvig commented 2 years ago

@leahneukirchen I'm not really familiar with a terminology here. What precisely do you mean is required?

rawhuul commented 2 years ago

@leahneukirchen I'm not really familiar with a terminology here. What precisely do you mean is required?

If you look here, the latest release tarball is version 27.x and if you need Emacs 28, you need to compile it from cloning from savannah repo. So, @leahneukirchen is holding on until upstream provides latest release tarball.

Software need to be used in version announced by authors as ready to use by the general public - usually called releases. Betas, arbitrary VCS revisions, templates using tip of development branch taken at build time and releases created by the package maintainer won't be accepted.

From Quality Requirements

void-linux / void-packages

[Package Request] GccEmacs #24240

Package Name

Linux-related link

Practical Rationale

Manual installation in Void