DEVELOPMENT.md directions do not work

[Oglopf]

The instructions as listed lead to an error:

git clone https://github.com/OSC/ondemand.git ~/ondemand/src
cd ~/ondemand/src
rake dev:start
rake aborted!
LoadError: cannot load such file -- dotenv
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'
/home/trav/ondemand/src/lib/tasks/rake_helper.rb:4:in `<top (required)>'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in `require'
/home/trav/ondemand/src/Rakefile:13:in `<top (required)>'
/usr/share/rubygems-integration/all/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
(See full trace by running task with --trace)To get things to run correctly I've had to do the following:

git clone https://github.com/OSC/ondemand.git ~/ondemand/src
cd ~/ondemand/src
mkdir .bundle
vim .bundle/config

Then adding an entry to ensure my vendor/bundle path is used:

```

BUNDLE_PATH: vendor/bundle

Now getting the `Gems` that seem to be needed:bundle install

After this running `bundle exec` to ensure those local `gems` are used works:bundle exec rake dev:start

I'm trying to understand if I'm building this wrong, or if there is something I'm missing about how all this works and should be done. Why did the simple `rake dev:start` not work?

┆Issue is synchronized with this [Asana task](https://app.asana.com/0/1202821104799596/1203550075137288) by [Unito](https://www.unito.io)

[gerald-byrket]

@Oglopf

I'll see if i can duplicate tomorrow.

-gerald

[Oglopf]

Even with this running, I get errors trying to do any work in the container as it seems like I'm missing Gems when trying to build apps within it.

For instance, if you now want to go work on the dashboard, you then connect this up as a dev app by doing the usual...cumbersome setup of running bundle exec rake dev:exec then su <user> then cd ondemand/dev then ln -s ../src/apps/dashboard/ dashboard then bin/setup but at this step I get errors around:

An error occurred while installing stringio (3.0.2), and Bundler cannot continue.

In Gemfile:
  sdoc was resolved to 2.4.0, which depends on
    rdoc was resolved to 6.4.0, which depends on
      psych was resolved to 4.0.6, which depends on
        stringio
/usr/share/gems/gems/rake-13.0.6/lib/rake/file_utils.rb:67:in `block in create_shell_runner': Command failed with status (5): [bin/bundle check 1>/dev/null 2>&1 || bin/b...] (RuntimeError)
    from /usr/share/gems/gems/rake-13.0.6/lib/rake/file_utils.rb:57:in `sh'
    from bin/setup:59:in `block in <main>'
    from /usr/share/ruby/fileutils.rb:139:in `chdir'
    from /usr/share/ruby/fileutils.rb:139:in `cd'
    from /usr/share/gems/gems/rake-13.0.6/lib/rake/file_utils_ext.rb:35:in `chdir'
    from bin/setup:50:in `<main>'

And at this point I'm at a total loss as to where what gems were installed locally or here in this container. Worse still, if you have a version mismatch of the ruby version you are in huge trouble, unless you go through the hassle of using rbenv or rvm both of which are broke on certain distributions, and we provide no indication of.

I have never used a container flow like this, so I think I need some lessons on how this should work. For myself, I typically just make code changes and rebuild my container, but with OOD we are doing an image pull that is locked down, then doing dev work in OOD in the container, and I can't seem to get the thing to do much beyond loading config files for me.

[gerald-byrket]

@Oglopf

definitely problem

[treydock]

The OnDemand dev container runs OnDemand installed to the OS, the gems are installed into OnDemand SCL paths if you install them using the tools to install gems from within the container. The only things mounted into the container are these paths:

https://github.com/OSC/ondemand/blob/d2b8ea3d1c29f57a6ce9ddb73b3906c180068357/lib/tasks/development.rb#L73-L82

The vendor/bundle to run Rake commands outside the container is just from Gemfile at root of project which is not the Gemfile that runs OnDemand and vendor/bundle is not going to be used to run OnDemand inside the container. Also generally speaking you don't want to create .bundle and modify the config, the config is set by executing proper bundle commands like bundle install --path vendor/bundle. By using --path bundler will save that path to .bundle/config so things know where to look for gems. But those gems are just for doing things like running Rake tasks or running E2E testing. The gems for running OnDemand are inside the container.

If you modify the Gemfile for something like dashboard or myjobs, then you have to install gems in the container and you'll want to do that using something like rake build && rake install from root of dev environment IN the container, but you don't need to install gems to modify code, the gems will already be installed when the container is built.

You do need some form of Ruby to run Rake outside the container. I use RVM because it's very simple to manage on OS X.

I haven't started with fresh cloned repo in years so here goes:

cd /tmp
git clone git@github.com:OSC/ondemand.git
# I already have Ruby 3.0 installed
$ rvm use ruby-3.0@ondemand-new --create
$ bundle install
$ rake dev:start
# This builds the container that has OnDemand installed from nightly RPMs
# Once the container is built, it will be running in daemon mode so persists in background
$ docker ps | grep ood
f486a635c9d6   ood-dev:latest   "/sbin/init"   About a minute ago   Up About a minute   0.0.0.0:5556->5556/tcp, 0.0.0.0:8080->8080/tcp   ood-dev
$ rake dev:exec
# At this point I'm in the container
bash-4.4# whoami
root
bash-4.4# echo $HOME
/Users/tdockendorf
bash-4.4# pwd
/Users/tdockendorf
bash-4.4# ls
ondemand
bash-4.4# ls ondemand/
bash-4.4# 

The only issue I see is some unfortunate assumption about where OnDemand lives. Rather than mount pwd to $HOME/onemand it appears to mount $HOME/ondemand to $HOME/ondemand but on my laptop I put all code in different place so OnDemand lives at $HOME/git/ondemand which is why above my ondemand directory is empty. I don't know the history behind that decision. I think it should assume the root of the project is where OnDemand lives.

If you want to see how gem lists are different. Run gem list in the container, then with SCL then outside the container:

# Outside container, used for Rake and E2E testing
$ gem list | wc -l
     161

# OS gems (not used by OnDemand)
bash-4.4# gem list | wc -l
78

# OnDemand Gems
bash-4.4# scl enable ondemand -- gem list | wc -l
197

I can't illustrate installing Gems for OnDemand in the container per the above issues with bad assumptions about paths. but I think it would go something like this:

$ rake dev:exec
# Now in container
cd ~/ondemand
scl enable ondemand -- rake build
scl enable ondemand -- rake install

[Oglopf]

I'm closing this out as it seems like it works as expected per Trey's comments

[Oglopf]

The container still does not run correctly using the instructions on ubuntu 22. You must use bundle exec rake dev:start and even then, I can't seem to hit the web interface though login and mounts work fine.

[Oglopf]

Also, why do we only have an image for Rocky 8 to do this?

[johrstrom]

Also, why do we only have an image for Rocky 8 to do this?

Do you want a different OS?