source deleted after package build

[8f46c63a-2596-4880-ba93-a09d8ba6c056]

The current build mechanism unconditionally deletes build/pkgs/$packagename/src. This should be triggered manually (or upon package update).

• it's impossible to install twice or create packages without recompilation
• creating/editing/testing patches is irksome without the source code
• it's impossible to use packages that don't copy/move to $SAGE_LOCAL (future?)

Fixing this requires

• rename src to src-$version (or something equivalent), to avoid confusion
• keep track of the patch state (don't try to patch twice)
• ... ?

Component: build

Keywords: build src clean

Reviewer: Felix Salfelder

Issue created by migration from https://trac.sagemath.org/ticket/14792

[jhpalmieri]

comment:1

In Sage right now, you can set the variable SAGE_KEEP_BUILD_SPKGS to not delete the temporary build directories. For an individual spkg installation, you can also do sage -i -s blah.spkg: the -s flag says to save (keep) the build directory. Note that keeping all of the build directories can take up a lot of disk space, so it is not a good idea to turn it on by default.

(This also doesn't quite do what you want: if you install an spkg once and then again, it will start over from scratch. You can manually change to the build directory, though, and do whatever recompilation or reinstallation you need.)

These features are documented in the installation guide.

[8f46c63a-2596-4880-ba93-a09d8ba6c056]

comment:2

Replying to @jhpalmieri:

Note that keeping all of the build directories can take up a lot of disk space, so it is not a good idea to turn it on by default.

I see your point about space consumption. Maybe it should be switched on with some sort of developer mode, or just if .git is present.

(This also doesn't quite do what you want: if you install an spkg once and then again, it will start over from scratch. You can manually change to the build directory, though, and do whatever recompilation or reinstallation you need.)

Exactly, that's why i have created the ticket. Rebuilding, installing or exporting packages should not require manual intervention (nor recompilation).

[83660e46-0051-498b-a8c1-f7a7bd232b5a]

comment:3

Replying to @jhpalmieri:

In Sage right now, you can set the variable SAGE_KEEP_BUILD_SPKGS to not delete the temporary build directories.

SAGE_KEEP_BUILT_SPKGS that is...

Another odd thing is that you can't just test an spkg (after it's been built / installed) even if you kept its build directory [other than by starting a Sage subshell, changing to the directory and running ./spkg-check].

And rerunning (parts of) spkg-install is usually also an adventure.

[8f46c63a-2596-4880-ba93-a09d8ba6c056]

comment:4

Replying to @nexttime:

Another odd thing is that you can't just test an spkg (after it's been built / installed) even if you kept its build directory [other than by starting a Sage subshell, changing to the directory and running ./spkg-check].

Package operations must be controlled from toplevel. The packages don't know their dependencies (currently), and spkg-install files are not in a shape to do anything else. Running checks can be done with make <packagename>-check (in my implementation).

And rerunning (parts of) spkg-install is usually also an adventure.

The spkg-install programs must be idempotent (of course). Just like make install is for all sane packages. (It's not hard to fix this!).

I think I should add something like AC_ARG_ENABLE([keepbuilt]...). I don't see a reason why it should be disabled in case .git is present. How are rebuilds caused by branch change implemented/handled currently?

[jdemeyer]

comment:5

Given that this is already an option (either using the environment variable or using the -s option), can this be closed as "wontfix"?

[8f46c63a-2596-4880-ba93-a09d8ba6c056]

comment:6

Replying to @jdemeyer:

Given that this is already an option (either using the environment variable or using the -s option), can this be closed as "wontfix"?

it will not solve the problem. before you notice, the source has gone.

go ahead and close the ticket. try to make sure that i am the only one.

[jdemeyer]

Changed author from Felix Salfelder to none

[jdemeyer]

Reviewer: Felix Salfelder

[jdemeyer]

comment:7

Replying to @sagetrac-felixs:

it will not solve the problem. before you notice, the source has gone.

go ahead and close the ticket. try to make sure that i am the only one.

You speak in mysteries to me. But I will do as you suggested and close the ticket.