RubyGems and SketchUp - Githubissues

This issue discusses the current state of gem integration with SketchUp.

Issues:

Common (macOS & Windows)

RDoc is not installed. Other default gems (gem uninstall won't work) may depend on it. Not sure why this is done. If it's related to RubyGems generating RDoc, that can be disabled by setting:
```
ENV['GEMRC'] = "#{RbConfig::TOPDIR}/.gemrc"
```
Then add the file, contents should be:
```
install: -N
update:  -N
```
Bundled gems cannot be required. They are installed properly on macOS, they are missing on Windows. On macOS, setting the following should fix:
```
ENV['GEM_PATH'] = "#{RbConfig::TOPDIR}/lib/ruby/gems/#{RbConfig::CONFIG['ruby_version']}"
```
To test it with current versions, Gem.paths = ENV in the console is required to reset RubyGems' paths. Note that this will not uninstall bundled gems unless one passes the folder to the gem command. Even then, file permissions may stop it.

Windows

Please install Bundled gems and all gemspecs as on macOS. Normally, RubyGem's looks in
```
File.join(RbConfig::CONFIG["rubylibprefix"], "gems", RbConfig::CONFIG["ruby_version"])
```
which can be shortened by setting the following in platform_specific\rbconfig.rb
```
CONFIG["rubylibprefix"] = "$(prefix)"
```
The above won't match the folder structure of macOS, but it works. If one sets:
```
ENV['GEM_PATH'] = "#{RbConfig::TOPDIR}/gems/#{RbConfig::CONFIG['ruby_version']}"
```
And then does Gem.paths = ENV (as mentioned above) to test current copies.

I added the gems files to my Windows 2023 install, and after the above, one can require a bundled gem, as the above changes also allow on macOS.

One thing that seems odd is that installing gems takes quite a while. Also, my stand-alone Ruby 2.7 took a while. There are some large files that RubyGems downloads with gem info and version info. I normally work with Ruby master locally on Windows & Ubuntu, and it seemed to install gems much quicker.

I'll try to check as to the reason...

General Gem Info (for those unfamiliar with gems)

For many years, the term Ruby 'standard library' (std-lib) has been used, and referred to items that were considered seperate from Ruby 'core'. Or, to use them one had to require them first.

Orginally, they were always part of the Ruby install and its repository's code.

As of Ruby 2.2, it was divided into two categories, default gems and bundled gems.

Today, almost all items in the std-lib have their own code repository and are available as gems. The code for default gems is included in the Ruby code repo, and maintainance code exists to sync them both ways with their own repos. The code for bundled gems does not exist in the Ruby code repo, and Ruby's build code installs them by downloading them from the RubyGems website. All of the Ruby code repositories are listed at https://github.com/ruby.

Below is a list of the bundled gems in Ruby 2.7.7 and Ruby master. Note that many std-lib items have been added, and that will probably continue. Hence, it's probably a good idea to get them working correctly.

                 Bundled Gems
———— Ruby 2.7.7 ————    ——— Ruby Master ———
                         debug        1.8.0
                         matrix       0.4.2
minitest     5.13.0      minitest     5.18.1
                         net-ftp      0.2.0
                         net-imap     0.3.6
                         net-pop      0.1.2
                         net-smtp     0.3.3
net-telnet   0.2.0       Removed
power_assert 1.1.7       power_assert 2.0.3
                         prime        0.1.2
                         racc         1.7.1
rake         13.0.1      rake         13.0.6
                         rbs          3.1.1
                         rexml        3.2.5
                         rss          0.2.9
test-unit    3.3.4       test-unit    3.6.1
                         typeprof     0.21.7
xmlrpc       0.3.0

Another consideration with gems is where they are installed. In standard stand-alone Ruby, the default location is within Ruby's lib folder.

RubyGems allows one to also install gems in a user folder (the --user-install argument). These gems allow use by any Ruby install matching the ABI Ruby version. ABI version is always the Ruby version with the last number changed to zero (the patch version). So, gems installed this way can be used by both a stand-alone Ruby and SketchUp's Ruby, assuming the ABI versions match.

Lastly, gems can be divided into three categories. The first are pure Ruby gems. The second are gems that contain c source code, and need build tools to compile them. These are often called extension gems. SketchUp cannot install extension gems. Some examples are production app/web servers, Nokogiri, and database access gems.

A single version of a gem can have multiple gems built. A common one is a pre-compiled gems. These gems are Ruby version limited, and contain precompiled *.so files, so build tools are not required. Pre-compiled gems are built by the gem maintainers, and are not available for many extension gems. Pre-compiled gems can be installed in SketchUp.

Finally, some tools that may be helpful. There are two files in one of my repos.

https://github.com/MSP-Greg/sketchup-misc/blob/main/su_gem.rb

SUGem wraps gem commands so they can be used from the Ruby Console in SketchUp. Some gem commands may prompt the user for an answer to a question, these do not work with SUGem. One example is uninstalling a gem, and two different versions are installed. There are often ways around the issue, one can uninstall a particular version of a gem with:

SUGem.uninstall <gem name>:<gem version>

https://github.com/MSP-Greg/sketchup-misc/blob/main/ruby_info/su_info.rb

This file writes a lot information about the Ruby install to the console. Last part shows all gems available to SketchUp, and what type they are (default, bundled, installed, user).

Feel free to clone/fork the repo. PR's welcome.

Ruby & RubyGems allow for limited modifications to the build structure. This is often done by groups packaging Ruby, and allows accounting for read-only locations, build tools, etc.

The above changes can be added to a file that Ruby/RubyGems loads. The file locations for macOS & Windows are shown below for SU 2023. The file is in the RubyGems lib folder, at defaults/operating_system.rb:

/Applications/SketchUp 2023/SketchUp.app/Contents/Frameworks/Ruby.framework/Versions/2.7.1/lib/ruby/2.7.0/rubygems/defaults/operating_system.rb
C:/Program Files/SketchUp/SketchUp 2023/Tools/RubyStdLib/rubygems/defaults/operating_system.rb

As mentioned, currently all files are installed on macOS, but cannot be loaded. Adding the above file to my macOS SU install fixes that.

The following shell script will enable loading the included bundled gems with SU 2023. With '23' changes to '22', it also works with SU 2022. It creates a directory and two very short files.

An example of why this should be corrected is base64.rb will be removed from the stdlib and become a bundled gem in Ruby 3.4. base64.rb is currently loaded by one of the included SketchUp extensions. Lastly, many of the calls in base64 are very short methods that call String.unpack1 and Array.pack, so the dependency is easily removed.

#!/bin/sh

cd /Applications/SketchUp\ 2023/SketchUp.app/Contents/Frameworks/Ruby.framework/Versions/2.7.1/lib/ruby/2.7.0/rubygems
mkdir 'defaults'
cd defaults

printf "ENV['GEMRC'] = \"#{RbConfig::TOPDIR}/.gemrc\"\n" >> ./operating_system.rb
printf "ENV['GEM_PATH'] = \"#{RbConfig::TOPDIR}/lib/ruby/gems/#{RbConfig::CONFIG['ruby_version']}\"\n" >> ./operating_system.rb
printf "Gem.paths = ENV\n" >> ./operating_system.rb

cd /Applications/SketchUp\ 2023/SketchUp.app/Contents/Frameworks/Ruby.framework/Versions/2.7.1
printf "install: -N\nupdate:  -N\n" >> .gemrc

It has been quite a long time since I looked at gems and SketchUp's edition of RbConfig. But when I did (in the past) the "rbconfig.rb" file and so the RbConfig object were not quite "kosher" for the end users machine.

Ie, especially the paths were a mixture of valid paths for the user's environment and invalid paths, the latter which were either only valid for the development machine upon which SketchUp's Ruby was compiled or some generic paths for a "normal" system Ruby install.

Windows

3. ... Normally, RubyGem's looks in

    File.join(RbConfig::CONFIG["rubylibprefix"], "gems", RbConfig::CONFIG["ruby_version"])

This is an example of what I meant above ... in SketchUp RbConfig::CONFIG["rubylibprefix"] returns a non-existing path. I never understood why whomever "packaged" Ruby up with the Windows edition made the Ruby lib folder named "RubyStdLib" instead of just keeping the "lib/ruby/2.7.0" folder hierarchy. Things with Ruby might have worked much better.

Ruby & RubyGems allow for limited modifications to the build structure. This is often done by groups packaging Ruby, and allows accounting for read-only locations, build tools, etc.

The above changes can be added to a file that Ruby/RubyGems loads. The file locations for macOS & Windows are shown below for SU 2023. The file is in the RubyGems lib folder, at defaults/operating_system.rb

I do remember telling Thomas about this file years ago and how it could be used to make gems used better within SketchUp. But I don't think it has ever been leveraged. It is basically empty "out of box" as I recall.

3. The above won't match the folder structure of macOS, but it works. If one sets:

    ENV['GEM_PATH'] = "#{RbConfig::TOPDIR}/gems/#{RbConfig::CONFIG['ruby_version']}"

And then does `Gem.paths = ENV` (as mentioned above) to test current copies.
I added the gems files to my Windows 2023 install, and after the above, one can require a bundled gem, as the above changes also allow on macOS.

This will not likely happen (in this way) for SketchUp gems. The gems that extensions need to load on the fly must be in the User %AppData% path because of User Access Control. This is why SketchUp creates the gem path there.

If bundled gems also need to be updated between SketchUp releases then they also would need to be in the %AppData% path. If they do not need to be updated except by the installer for new releases, then they can either be in the %ProgramFiles% or %ProgramData% path.

Ie, when testing better gem situations, be sure that you are not logged in to a Admin account or are running with elevated privileges. Ie test using a normal user non-admin account so you can experience the permissions errors such users would face.

MacOS

2. Bundled gems cannot be required. They are installed properly on macOS, they are missing on Windows. On macOS, setting the following should fix:
    ENV['GEM_PATH'] = "#{RbConfig::TOPDIR}/lib/ruby/gems/#{RbConfig::CONFIG['ruby_version']}"

Just to confirm, are you saying that bundled gems are actually included within the SketchUp bundle Ruby framework ?

Also, changing the gem path again would create permission issues for extensions needing to download and install gems. This is why SketchUp sets it into the ~/Library/Application Support path.

It seems perhaps it might be a "safer" fix just to push, prepend or insert the bundled gems path somewhere into the $LOAD_PATH array.

@DanRathbun

Much of your previous post deals with concepts related to using stand-alone Ruby, and also building, testing, and packaging Ruby. Also related are the various needs of the many *nix distributions, which vary a great deal in terms of security, locations of various file types, etc. I suspect you don't have much experience with those things. So, I'll just respond to a few things.

Re RbConfig, much of its functionality is needed for building & testing Ruby. It is also used by RubyGems/Bundler for config options and when compiling extension gems. So, what seems odd in the SU copy is of no consequence.

If bundled gems also need to be updated between SketchUp releases

If one is aware of the ramifications of doing so, that's fine. One can update a bundled gem, it will be installed in AppData, and RubyGems will load the most recent version by default. You can specify which version to load. There are *nix distros where the Ruby installation is secure as in Windows, and user installed gems are placed elsewhere.

Just to confirm, are you saying that bundled gems are actually included

On macOS, yes, but they can't be required/loaded because RubyGems isn't configured properly, so it can't find them.

It seems perhaps it might be a "safer" fix just to push, prepend or insert the bundled gems path somewhere into the $LOAD_PATH array.

Every addition to $LOAD_PATH means a potential for more drive access, as Ruby will search $LOAD_PATH looking for required files. Also, there is no single $LOAD_PATH for bundled gems. They are gems but included with Ruby. Previously, they may have been part of the std-lib, which does have a single load path. As gems, they are meant to be loaded by RubyGems.

I've added info on setting up RubyGems properly on Windows. See the README at:

https://github.com/MSP-Greg/sketchup-misc/tree/main/windows_rubygems

Logged as: SKEXT-3888

SketchUp / api-issue-tracker

RubyGems and SketchUp #912

Common (macOS & Windows)

Windows

General Gem Info (for those unfamiliar with gems)

Windows

MacOS