search

LDMX-Software / docker

Docker build context for developing and running ldmx-sw.
https://ldmx-software.github.io/docker/
GNU General Public License v3.0
1 stars 2 forks source link

Update OS to Ubuntu 24.04 #102

Open tvami opened 2 weeks ago

tvami commented 2 weeks ago

Is your request related to a problem? Please describe.

As a precursor to https://github.com/LDMX-Software/docker/issues/101 we should update the OS in the container (this also fixes some of the Boost issues we saw recently)

Describe the solution you'd like. What packages do you want installed?

tvami commented 2 weeks ago

It seems somewhat obvious how to change this, OTOH it's not obvious at all how we'd test ldmx-sw until it's integrated. @tomeichlersmith ?

tomeichlersmith commented 2 weeks ago

See https://github.com/LDMX-Software/docker/pull/97

Dependabot attempted the simple option for us and it unfortunately didn't work. ROOT 6.22.08 does not compile with the newer GCC that comes with the Ubuntu 24.04 update. This leaves us with many options but I think two are the most helpful:

  1. Update ROOT as well as the OS (and Python3, and Boost, and ....). This would be a pretty large technical change but would allow us to e.g. require C++20 which is required by ACTS now and get a newer Python and Boost along with the newer compiler.
  2. Build Boost from source. Boost was built from source when the image was Ubuntu 18.04 based and so I don't think it would be too crazy difficult.

(1) also has other questions that could come up

All this to say... switching OS and compiler versions has a lot of cascading effects. None of these hurdles are impossible, especially for experienced human compilers, but they would require time to investigate.

tvami commented 2 weeks ago

Does Pythia6 compile with the newer compiler?

I thought Einar was using Pythia8. Where do we use Pythia6? We really should move away from that for Physics reasons if we do

Update ROOT as well as the OS

Do you have an example on how this was done in the past?

tomeichlersmith commented 2 weeks ago

Pythia6 is needed for GENIE although last time I heard from @wesketchum GENIE was moved to Pythia8, we just need to move to a newer release of GENIE as well. Edit: @wesketchum 's draft update to GENIE adding Reweight (https://github.com/LDMX-Software/docker/pull/79) includes installing Pythia8 which I'm guessing means that (after some testing) Pythia6 could be dropped.

https://github.com/LDMX-Software/docker/blob/1c729e21b606ac9400e2e29c184a645e14c39338/Dockerfile#L102

The last OS update did not require a change in ROOT https://github.com/LDMX-Software/docker/pull/54 and there wasn't a container before that so this is "new" territory. I don't expect ROOT's serialization to change much when we move to a newer version but it might trigger us to update some of the hacks I've implemented within the Framework (https://github.com/LDMX-Software/ldmx-sw/commit/cfe38da4246609f356c73425ab8f004e71742dbe and https://github.com/LDMX-Software/ldmx-sw/commit/5ddc20eb5284da3a062ccfd64de620b6854fc1a1).

tomeichlersmith commented 1 week ago

Why did you choose ROOT 6.30.09? Are you opposed to moving to the latest release whenever we get around to this?

tvami commented 1 week ago

Are you opposed to moving to the latest release

not necessarily against it if it works, I know Chiara just ran into some compatibility issues, but yeah sure I can check first if it works with something newer

tvami commented 1 week ago

@tomeichlersmith do you know what llvmlite is used for? I get this msg

19.06 Collecting llvmlite==0.39.1 (from -r /etc/python_packages.txt (line 7))
19.15   Downloading llvmlite-0.39.1.tar.gz (132 kB)
19.17      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.8/132.8 kB 11.7 MB/s eta 0:00:00
19.21   Preparing metadata (setup.py): started
19.54   Preparing metadata (setup.py): finished with status 'error'
19.54   error: subprocess-exited-with-error
19.54   
19.54   × python setup.py egg_info did not run successfully.
19.54   │ exit code: 1
19.54   ╰─> [8 lines of output]
19.54       Traceback (most recent call last):
19.54         File "<string>", line 2, in <module>
19.54         File "<pip-setuptools-caller>", line 34, in <module>
19.54         File "/tmp/pip-install-9rou24tx/llvmlite_0d42782734874170af7ea3e526020dfc/setup.py", line 55, in <module>
19.54           _guard_py_ver()
19.54         File "/tmp/pip-install-9rou24tx/llvmlite_0d42782734874170af7ea3e526020dfc/setup.py", line 52, in _guard_py_ver
19.54           raise RuntimeError(msg.format(cur_py, min_py, max_py))
19.54       RuntimeError: Cannot install on Python version 3.12.3; only versions >=3.7,<3.11 are supported.
19.54       [end of output]
19.54   
19.54   note: This error originates from a subprocess, and is likely not a problem with pip.
19.54 error: metadata-generation-failed

I'll remove it for now, let's hope we dont need this anymore

tvami commented 1 week ago

Nvm, it's about the versioning, if I dont enforce a given version it works

tomeichlersmith commented 1 week ago

(llvmlite is a python binding of llvm so that numba can (partially) compile code before evaluating it to achieve some performance improvements)