Investigate size of express_parser.so for Ruby 3.0, 3.1

lutaml / expressir

Ruby parser for the ISO EXPRESS language

3 stars 2 forks source link

Investigate size of express_parser.so for Ruby 3.0, 3.1 #122

Closed CAMOBAP closed 2 years ago

CAMOBAP commented 2 years ago

Intro

During the investigation of docker image size in the scope of https://github.com/metanorma/metanorma-docker/issues/147 I have found that size of express_parser.so for Ruby 3.0, 3.1 (21 MB) is 3 times more than express_parser.so for Ruby 2.7 (7.2 MB)

Questions

Is there a potential to reduce the size of express_parser.so for Ruby 3.0, 3.1?
If this is not possible maybe we can conditionally remove unnecessary binaries. For example if we install expressir gem for Ruby 3.1, can we remove 2.7 and 3.0 variants?

ronaldtse commented 2 years ago

@maxirmx can you help check? Thanks!

maxirmx commented 2 years ago

I will look at it

maxirmx commented 2 years ago

Is there a potential to reduce the size of express_parser.so for Ruby 3.0, 3.1?

There is some difference between ruby 2.x and ruby 3.x that causes linker to behave differently. Furthermore, it happens on redhat-like distributions only (expressir employs rake-compiler-dock that uses manylinux image to create native gems). If I build on Ubuntu, 2.7 version is 21 Mb as well

I can look at it further but I was not able to develop any plan yet.

CAMOBAP commented 2 years ago

Is there a potential to reduce the size of express_parser.so for Ruby 3.0, 3.1?

There is some difference between ruby 2.x and ruby 3.x that causes linker to behave differently. Furthermore, it happens on redhat-like distributions only (expressir employs rake-compiler-dock that uses manylinux image to create native gems). If I build on Ubuntu, 2.7 version is 21 Mb as well

I can look at it further but I was not able to develop any plan yet.

BTW how do you think about the second question? I just checked several gems and neither of them implements such behavior. Maybe there are some obvious reasons why it's not possible?

maxirmx commented 2 years ago

If this is not possible maybe we can conditionally remove unnecessary binaries. For example if we install expressir gem for Ruby 3.1, can we remove 2.7 and 3.0 variants?

Expressir employs rake-compiler-dock to create native extensions
Rake-compiler-dock links libruby statically (I do not understand why, probably to ensure stable environment)
Since libruby is linked statically a separate binary is required for each Ruby API version Please refer to nokogiri that also employs rake-compiler-dock and has 4 binaries packaged

We can probably develop bundler plugin that will remove all extra binaries not only for expressir but for other gems as well

maxirmx commented 2 years ago

I have to say that there is something wrong with expressir linkage. It is not only this size issue but also Alpine issues (#106) Other gems built with rake-compiler-dock are glibc-neutral. This is the positive side of static linking.

ronaldtse commented 2 years ago

Thanks @maxirmx for the continued investigation!

maxirmx commented 2 years ago

It looks like we can have only two versions of the library packed - 2.x and 3.x Ruby 2.7 cannot load 3.x library and 3.0 and 3.1 appear to be the same. It will save 21Mb. @CAMOBAP does it make sense ?

maxirmx commented 2 years ago

I found the difference in compilation 2.x vs 3.x
I believe I will be able to create a version where both 2.x and 3.x files will be ~ 7.5 Mb

ronaldtse commented 2 years ago

Thank you @maxirmx !