install process is a bit clunky

[tjlane]

@antonbarty if it's cool with you guys I'll take a crack at this. Any reason not to replace the build all script with a more standard configure/make/make install workflow?

[tjlane]

update: I should have pulled before opening the issue, looks like this is being actively worked on. Maybe I can test out the new system and give some feedback instead!

[antonbarty]

Hi TJ

No problem at all - tidying up the build scripts would be appreciated.

PS: I did update some stuff • Removed making myana version from make by default as it's not always needed, and is the hardest to get to work • Added build-libcheetah and build-psana scripts at top level (they were in examples before and not found) • Talked to Marc M. about proper ordering of sit_setup, etc. in build-all. Which is an overkill script as it always does a del_upgrade and sconz -c for every build…

On Mar 23, 2013, at 10:36 AM, TJ Lane notifications@github.com wrote:

@antonbarty if it's cool with you guys I'll take a crack at this. Any reason not to replace the build all script with a more standard configure/make/make install workflow?

— Reply to this email directly or view it on GitHub.

[antonbarty]

Feedback appreciated.

If there's a better way to do it - great.

eg: I'm sure there's a better way to auto-make all the .o files in cheetah.lib without writing it out manually… just never been high enough priority to fix.

On Mar 23, 2013, at 10:38 AM, TJ Lane notifications@github.com wrote:

update: I should have pulled before opening the issue, looks like this is being actively worked on. Maybe I can test out the new system and give some feedback instead!

— Reply to this email directly or view it on GitHub.

[FilipeMaia]

I just went through the odissey of trying to get cheetah working with the portable psana(ana) version. I think we could do a much better job in simplifying the building. The building of libcheetah works reasonably well. The file interface with the rest is the problem. When building with psana for example right now we're also building psana itself, while I think we could use the one from sw/releases/ana-current and only compile the shared library module. Also there seems to be a mix of configuration files, source files and build scripts sprinkled throughout the source tree.

[rjbean]

Rick and I also took the view that the building of psana every time was not necessary which is why we removed from the source folder and added some comments to try and work out the proper method of linking the cheetah-ana-mod etc. libraries to the pre-built psana. For some reason the psana source code was added back into the cheetah source tree. I found the linking problem to be related to the distinction between releases/ana-current and /releases/ana-specific_release in the path when you run sit_setup. Like Filipe I find the building of the libcheetah library to be no problem. A 'more normal' install process would definitely be appreciated!

[tjlane]

Great. I'm actually going on vacation for a few days here, but when I get back maybe I'll take a look at this from the view of a dummy trying to install for the first time (pretty accurate representation lol) and make some tweaks as appropriate until it makes sense!

[antonbarty]

Hi folks

First of all it's good news that the library itself builds pretty easily. The Makefile could be neater, but it basically works. If we ever write a reader for SLAC hdf5 files, that should also be easy to compile stand alone.

So the problem lies with the compilation of psana and how that is implemented. I was just at CXI, and saw most of the time people were using the build-all script - because that is what worked for them the first time. In most cases build-all is massive overkill - it builds the libCheetah, then updates the psana release to current versions, cleans all directories and recompiles psana and the cheetah module from scratch. Necessary maybe once, but much more work than is needed for implementing simple changes.

By way of contrast - when I make a change, it's a case of build-libcheetah and build-psana, which is simply a scons across the cheetah_ana_mod module. Usually not more than a few files need recompilation and it's pretty quick.

I think what has happened is that something doesn't work, a solution is found - or rather some things are done, then it works, so the conclusion is that 'some things' must be needed and are added to the scripts - and then this propagates. By thinking about what is actually needed when, we can and should be more critical and careful of the scripts.

Before implementing a solution I'd like to understand more about the psana dependencies, to determine the minimum set of compilations required.

First - it is sometimes necessary to update psana to implement changes made at SLAC. When psana is updated, we need to compile against the new headers etc. That's where the sit_setup, relupgrade, sit_setup part of the script comes from (scons scripts must match psana version, and all that needs updating). A scons -c is thrown in for good measure, just in case. No need to do this every time - only when something changes or breaks in psana. That's where the update-psana script comes in.

Second - when libCheetah changes, what needs to be recompiled? Clearly the changed modules in libCheetah itself. If one of the central headers has changed (e.g.: event or global) then everything should be recompiled, and the psana module recompiled as well. No need to do the full relupgrade, scons -c, etc.

Third - how is Cheetah-psana module loaded? Dynamically, I presume. That implies that if the function calls don't change, only the module needs recompilation. I have been bitten before by changing a .h that was incorporated in only some of the .o files due to a broken makefile, and all hell breaking loose when some .o file tries to access the wrong bit of a structure that has changed in content.

Maybe one size does not fit all

• One script for implementing small changes in libCheetah (compiles libCheetah and cheetah-ana-mod)
• One script for updating psana (if the file reader is broken)
• One script for make all - the thing to be done from a clean download

Thoughts?

Cheers

Anton

On Mar 26, 2013, at 11:22 AM, rjbean notifications@github.com wrote:

Rick and I also took the view that the building of psana every time was not necessary which is why we removed from the source folder and added some comments to try and work out the proper method of linking the cheetah-ana-mod etc. libraries to the pre-built psana. For some reason the psana source code was added back into the cheetah source tree. I found the linking problem to be related to the distinction between releases/ana-current and /releases/ana-specific_release in the path when you run sit_setup. Like Filipe I find the building of the libcheetah library to be no problem. A 'more normal' install process would definitely be appreciated!

— Reply to this email directly or view it on GitHub.

[FilipeMaia]

Understanding psana dependencies is the key step. Another very useful thing would be to separate the source tree from the build tree and delete as much junk as possible. I also think update-psana should be called something like recompile psana modules which is what it should do.

[hgliu]

I totally agree that recompiling psana is not necessary. I had trouble with the LD_LIBRARY_PATH very often, and the default script did not compile all required modules in psana for me.

Is it possible to pack all necessary files in cheetah without bothering psana update? Let's presume that the source tree is already up to date for psana. It will be useful to get cheetah installed on other machines besides SLAC computers.

Haiguang

On Wed, Mar 27, 2013 at 12:22 AM, FilipeMaia notifications@github.comwrote:

Understanding psana dependencies is the key step. Another very useful thing would be to separate the source tree from the build tree and delete as much junk as possible. I also think update-psana should be called something like recompile psana modules which is what it should do.

— Reply to this email directly or view it on GitHubhttps://github.com/antonbarty/cheetah/issues/1#issuecomment-15468742 .

[rkirian]

The LD_LIBRARY_PATH is currently needed for psana to find the cheetah module. That's not the right way to do things, and it has caused a lot of confusion. Ideally, we would simply point a new user to a compiled "cheetah" frontend program (e.g. psana) and it would "just work", without the need to configure an environment. I wonder if it is possible to build psana such that the search path to the modules actually exists in the compiled program. LD_RUN_PATH didn't seem to work for me. Does anyone know how to do this?

Rick

On Mar 28, 2013, at 2:12 AM, hgliu wrote:

I totally agree that recompiling psana is not necessary. I had trouble with the LD_LIBRARY_PATH very often, and the default script did not compile all required modules in psana for me.

Is it possible to pack all necessary files in cheetah without bothering psana update? Let's presume that the source tree is already up to date for psana. It will be useful to get cheetah installed on other machines besides SLAC computers.

Haiguang

On Wed, Mar 27, 2013 at 12:22 AM, FilipeMaia notifications@github.comwrote:

Understanding psana dependencies is the key step. Another very useful thing would be to separate the source tree from the build tree and delete as much junk as possible. I also think update-psana should be called something like recompile psana modules which is what it should do.

— Reply to this email directly or view it on GitHubhttps://github.com/antonbarty/cheetah/issues/1#issuecomment-15468742 .

— Reply to this email directly or view it on GitHub.

[FilipeMaia]

I think the solution is to set the RPATH when building the cheetah module such that it finds the libs it needs and also RPATH when building psana itself such that it finds the module.

[rjbean]

The problem I can see there is that if we want to use the pre-installed LCLS version of psana then the RPATH has to depend on where the cheetah modules are built (which will necessarily change).

On the subject of LD_LIBRARY_PATH; when you run sit_setup in the cheetah/source/lcls-psana directory the .../sw/releases/ana-current/... path is changed to .../sw/releases/ana-0.9.1/... in your LD_LIBRARY_PATH. For me this was usually the step which allowed psana to find the cheetah-ana-pkg,chuck-ana-pkg and the xtcProofreaderpkg libraries.

[rkirian]

I don't think we really want to use the pre-installed psana. At the moment, it makes sense to do so, but if we can modify the RPATH as Filipe suggests, then it makes more sense to build our own psana. That way, we avoid the LD_LIBRARY_PATH completely, as well as version issues (like when they update the "ana-current" links).

Would be cool if we could avoid all this "sit_setup" business too.

Rick

On Mar 28, 2013, at 2:12 PM, rjbean wrote:

The problem I can see there is that if we want to use the pre-installed LCLS version of psana then the RPATH has to depend on where the cheetah modules are built (which will necessarily change).

On the subject of LD_LIBRARY_PATH; when you run sit_setup in the cheetah/source/lcls-psana directory the .../sw/releases/ana-current/... path is changed to .../sw/releases/ana-0.9.1/... in your LD_LIBRARY_PATH. For me this was usually the step which allowed psana to find the cheetah-ana-pkg,chuck-ana-pkg and the xtcProofreaderpkg libraries.

— Reply to this email directly or view it on GitHub.

[FilipeMaia]

I think we can avoid the sit_setup part as long as we roll our own build system which I am seriously considering... That way you would only have to specify the location of PSPKG_ROOT at build time (with some reasonable defaults so that it works out of the box at SLAC) and nothing else.

On Thu, Mar 28, 2013 at 2:50 PM, rkirian notifications@github.com wrote:

I don't think we really want to use the pre-installed psana. At the moment, it makes sense to do so, but if we can modify the RPATH as Filipe suggests, then it makes more sense to build our own psana. That way, we avoid the LD_LIBRARY_PATH completely, as well as version issues (like when they update the "ana-current" links).

Would be cool if we could avoid all this "sit_setup" business too.

Rick

On Mar 28, 2013, at 2:12 PM, rjbean wrote:

The problem I can see there is that if we want to use the pre-installed LCLS version of psana then the RPATH has to depend on where the cheetah modules are built (which will necessarily change).

On the subject of LD_LIBRARY_PATH; when you run sit_setup in the cheetah/source/lcls-psana directory the .../sw/releases/ana-current/... path is changed to .../sw/releases/ana-0.9.1/... in your LD_LIBRARY_PATH. For me this was usually the step which allowed psana to find the cheetah-ana-pkg,chuck-ana-pkg and the xtcProofreaderpkg libraries.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/antonbarty/cheetah/issues/1#issuecomment-15589235 .

[antonbarty]

I also think update-psana should be called something like recompile psana modules which is what it should do.

This sounds like a good idea.

There should be two scripts: • One to rebuild Cheetah once something is changed; do the source/Makefile and re-make cheetah-ana-mod. • One to rebuild psana and whatever else may required if there is an update to the LCLS system, so that old code is not compatible with new XTCs. That's the relupgrade and scons -c and the rest.

PS: Talking to Marc M. he says the right way to do it is: cd $CHEETAHDIR/source/lcls-psana . /reg/g/psdm/bin/sit_setup.sh relupgrade ana-current addpkg psana . /reg/g/psdm/bin/sit_setup.sh scons -c scons

Yes, two sit_setup.sh - but apparently this is the consistent way to do it.

A.

[antonbarty]

It will be useful to get cheetah installed on other machines besides SLAC computers.

Portablility of psana is the catch; Using the SLAC HDF5 files as a backup if psana can't be compiled may be a workaround.

[antonbarty]

Many of the compilation incompatibilities have been because of different versions of the build scripts pulled or lying around compared to the versions of software actually in someone's directory. If we don't include psana, we still need to tell everyone how to update the linking to use the right version of psana linked modules to go with the compiled cheetah-ana-mod Using a central psana is a recipe for things mysteriously breaking when someone at SLAC recompiles and we didn't notice that we needed to recompile a module or something.

After hassles with backwards compatibility - we leave a copy of the cheetah executable with every experiment, just in case we need to run the old code because new code tanks for some reason. A relic of being scared by myana, perhaps; this would not be possible with linking to a central version maintained by someone else

On Mar 28, 2013, at 2:50 PM, rkirian notifications@github.com wrote:

I don't think we really want to use the pre-installed psana. At the moment, it makes sense to do so, but if we can modify the RPATH as Filipe suggests, then it makes more sense to build our own psana. That way, we avoid the LD_LIBRARY_PATH completely, as well as version issues (like when they update the "ana-current" links).

Would be cool if we could avoid all this "sit_setup" business too.

Rick

On Mar 28, 2013, at 2:12 PM, rjbean wrote:

The problem I can see there is that if we want to use the pre-installed LCLS version of psana then the RPATH has to depend on where the cheetah modules are built (which will necessarily change).

On the subject of LD_LIBRARY_PATH; when you run sit_setup in the cheetah/source/lcls-psana directory the .../sw/releases/ana-current/... path is changed to .../sw/releases/ana-0.9.1/... in your LD_LIBRARY_PATH. For me this was usually the step which allowed psana to find the cheetah-ana-pkg,chuck-ana-pkg and the xtcProofreaderpkg libraries.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub.