REP 128: add term for devel/install spaces

[dirk-thomas]

• [x] update REP #79
• [x] review/update catkin docs
  • [x] added link to REP: http://wiki.ros.org/action/diff/catkin/migrating_from_rosbuild?action=diff&rev1=100&rev2=101
  • [x] added link to REP and result space term: http://wiki.ros.org/action/diff/catkin/workspaces?action=diff&rev1=6&rev2=7
  • [x] .rst based documentation @jack-oquin
• [x] review/update catkin_tools docs and CLI help @wjwwood
• [x] review/update wstool docs and CLI help @wjwwood
• [x] review/update vcstools docs and CLI help @wjwwood

The following items seem to not require any documentation changes:

• catkin_make / catkin_make_isolated CLI help

[dirk-thomas]

All folder have specific terms defined in REP 128 but in documentation we often want to refer to folders which have setup files generically. These can either development space or install space. Therefore we want to define a specific term for these. The following options came up in the recent discussion:

• run space
• execute space
• setup space
• fhs space

After some internal discussion we think that:

• "setup" is not appropriate (even that the folder contains setup files) but these folder are more intended to be used rather then to set up something
• "fhs" is too specific for this term and would lead to more questions (what is fhs?).

Therefore we suggest "run space" and the following addition to the REP:

Run Space
---------
This generic term is introduced for folders which contain the setup files.
Both development spaces and install spaces are instances of a run space.

@cottsay @jack-oquin @jbohren @mikepurvis @tfoote @vrabaud @wjwwood

Please propose alternative terms if you feel strongly about this - or also comment if you agree with the proposed term.

[jbohren]

+1 to "run space"

[jbohren]

Can we also define "workspace chaining" in the REP? Currently it looks like this tutorial is the only place it's mentioned on ros.org outside of ros-answers.

[jbohren]

Can we also define "workspace chaining" in the REP? Currently it looks like this tutorial is the only place it's mentioned on ros.org outside of ros-answers.

Also in such a way that we can also be consistent, and we can choose one of the following descriptors I've seen used for what is essentially the same action:

• chaining
• overlaying
• extending
• inheriting

[jbohren]

Also, it's worth noting that since "run-spaces" (pending naming) are what are actually sourced, chaining workspaces is technically chaining "run-spaces".

[jack-oquin]

For reference, the current REP-0128 is here: http://www.ros.org/reps/rep-0128.html

I have no strong objection to the run-space term, but defining it as "folders which contain setup files" does not address the valid objection to calling them setup-spaces:

• "setup" is not appropriate (even that the folder contains setup files) but these folder are more intended to be used rather then to set up something

We need to describe what the term means to users. I am not entirely sure exactly what that is, but it vaguely "provides a mechanism for defining the environment needed to run ROS packages defined in that run-space, and the one it extends (recursively)".

If I understand the terms correctly, catkin can build a workspace using only a source-space and implicit or explicit reference to the "underlying" run-space it "overlays". Doing that produces a new (or newly-updated) run-space in its devel-space. Sourcing the setup there provides the environment necessary to execute ROS nodes using what was built.

That reminds me somewhat of the confusion in terminology caused by fuzzy use of <build_depend> and <run_depend> in package.xml format one. Does the run-space really set up exactly the environment needed for using what we now call an <exec_depend>? If so, then exec-space might be clearer to those familiar with our history, and I would prefer it, instead.

[jbohren]

If so, then exec-space might be clearer to those familiar with our history, and I would prefer it, instead.

Yeah, the term run-space (a space that's needed for executing or building dependent packages) is analogous to the meaning of <run_depend>. This is a good point that @jack-oquin makes, though. You would need to source the setup file if you want to build against this space even if you're not running anything against that.

Meanwhile! The addition of the --extend-ws flag in catkin build makes it so people don't actually have to source these files in order to chain workspaces, so maybe the name isn't as significant.

That being said, here are some other name ideas:

• result-space
• product-space
• target-space

[jack-oquin]

You would need to source the setup file if you want to build against this space even if you're not running anything against that.

Really? I did not realize that.

If so, run-space and exec-space are both terrible names: -1

[jbohren]

You would need to source the setup file if you want to build against this space even if you're not running anything against that.

Really? I did not realize that.

Right, via the current auto-extension behavior you'd need to source a workspace in order to build another workspace against it.

If so, run-space and exec-space are both terrible names: -1

More options (included previous, sorted by my preference):

• deploy-space
• result-space
• product-space
• target-space
• app-space
• live-space
• env-space
• use-space
• operation-space / op-space
• functional-space
• hier-space (from hierarchy)

[jbohren]

Other options:

• setup directory / setup dir
• setup tree
• mock-space as in mockup of a FSH tree
• prime-space
• catkin context
• workspace realization

[NikolausDemmel]

What has come up previously, but not in the recent discussion, is the question whether there should be separate "run" and "build" setup files, i.e. setupfiles that create an environment to rosrun/roslaunch vs setup files that create an environment to build against. In the light of that overloaded and somewhat conflicting functionality of the setupfiles (highlighted nicely in the last part of Thibault's comment https://github.com/catkin/catkin_tools/issues/67#issuecomment-45709489), I strongly object to the names "run space" or "exec space" or the like (independent of whether splitting up the setup files into "run" and "build" setup files is feasable in a way that makes things less confusing, not more, given catkin's history).

I don't have a favorite choice, but I think @jack-oquin has a good point

We need to describe what the term means to users. I am not entirely sure exactly what that is, but it vaguely "provides a mechanism for defining the environment needed to run ROS packages defined in that run-space, and the one it extends (recursively)".

We should first agree on the definition, and then find an apropriate name: IMO the abstract definition is something like a "location that provides a mechanism for defining the environment needed to run or build against catkin packages defined in that space, and the ones it extends (recursively)", and more concretely, it is a "folder containing caktin setup.* files". The nature of this "meachnism" (= sourcing setup.* files) is implicit with the proposed --extend argument, while it is explicit for preparing a shell for implicit workspace chainging and also for preparing a shell to run ros packages.

On a different but related note, has the use of the term "workspace" been discussed in the conference call? I think for catkin the workspace is the folder you typically execute catkin_make from (usually parent of the source/build/devel/install) or maybe for the future a folder that has the corresponding marker file. Whatever the precise definition for a catkin workspace, one unfortunate objection that was brought up is the conflicting use in wstool, where workspace refers to what is a source space for catkin.

[jbohren]

@NikolausDemmel What has come up previously, but not in the recent discussion, is the question whether there should be separate "run" and "build" setup files, i.e. setupfiles that create an environment to rosrun/roslaunch vs setup files that create an environment to build against. In the light of that overloaded and somewhat conflicting functionality of the setupfiles (highlighted nicely in the last part of Thibault's comment catkin/catkin_tools#67 (comment)), I strongly object to the names "run space" or "exec space" or the like (independent of whether splitting up the setup files into "run" and "build" setup files is feasable in a way that makes things less confusing, not more, given catkin's history).

I think that it's good that in order to "use" any part of a workspace, regardless if it's to build against or run from, a user needs to load just one shell script that corresponds to the result of building that workspace with a given configuration. I think trying to create setup files for different purposes will introduce unnecessary complexity. Really, this space equips you to use the workspace.

To that end, how about equip-space?

IMO the abstract definition is something like a "location that provides a mechanism for defining the environment needed to run or build against catkin packages defined in that space, and the ones it extends (recursively)", and more concretely, it is a "folder containing caktin setup.* files". The nature of this "meachnism" (= sourcing setup.* files) is implicit with the proposed --extend argument, while it is explicit for preparing a shell for implicit workspace chainging and also for preparing a shell to run ros packages.

Yeah this is all correct in that it defines the directory we're talking about. This is true for now, at least, but Dirk and Will mentioned at one point during the telecon that at some point these setup files are going to need to move out of the root of the FSH tree in order to be distributed through normal package managers.

@NikolausDemmel On a different but related note, has the use of the term "workspace" been discussed in the conference call? I think for catkin the workspace is the folder you typically execute catkin_make from (usually parent of the source/build/devel/install) or maybe for the future a folder that has the corresponding marker file.

Yeah, we all agreed that a "catkin workspace" is the directory containing the other "subspaces" from which you run one of these high-level build commands.

@NikolausDemmel Whatever the precise definition for a catkin workspace, one unfortunate objection that was brought up is the conflicting use in wstool, where workspace refers to what is a source space for catkin.

Yeah, we didn't discuss this at length, but I think the vcstools/wstool code needs a lot of re-branding (https://github.com/vcstools/wstool/issues/19). I'd be happy to discuss that after the dust settles on all this.

[dirk-thomas]

@jbohren Can you please create a separate issue to extend the REP with a definitions / terms on workspace chaining / overlaying? I have the feeling finding a single name might take a while even without further additions...

[dirk-thomas]

I will "rule out" some of the proposed names and give a very brief reason for each. This will reflect my personal opinion and does not imply any final decision - but we simply need to make progress here.

The name should end in space as do all the other terms defined in this REP - so these are "out":

• catkin context
• setup directory / setup dir
• setup tree
• workspace realization

The following don't provide a good / clear / precise association imo:

• app-space: app has a very different meaning in the software domain
• deploy-space: deploy is associated with deploying to a live system
• env-space: the folder provides far more than an environment
• execute space: arguably the folder can be used just to build the next workspace
• fhs space: fhs is to specific, most people don't know what it is
• functional-space: functional for what? may it the folder provides far more than an environment
• hier-space: there is no hierarchy here
• live-space: if it hasn't been sourced yet it is everything but live
• mock-space as in mockup of a FSH tree: there is no mocking happening, the folder is the real complete thing
• operation-space / op-space: what operation is done?
• prime-space: prime seems to not match that there are potentially multiple of them
• product-space: product of what? what are the factors?
• run space: arguably the folder can be used just to build the next workspace
• setup space: the folder is more the result of a source space, one the scope of a single workspace the setup phase is long passed whe this folder exists
• target-space: for pure CMake packages targets go to a very different location

After this "fast" reduction the following are left:

• result-space: the result of a build operation which is can be utilized afterwards (for whatever build more stuff on top as well as run / execute things from it)
• use-space: the usable space (again usable to build more stuff on top as well as run / execute things from it)

What are your opinions - especially on the last two terms?

Update based on feedback from @NikolausDemmel:

• use-space: it is not clear what it is used for: used for the build or after building the workspace

What is your opinion on result space?

[jack-oquin]

What are your opinions - especially on the last two terms?

All of Dirk's comments make sense, and result-space and use-space are both better than the others. I have no immediate preference between the two. I'll think about them some more.

[NikolausDemmel]

Out of the last two I prefer result-space. Catkin is a build tool that builds source code found in the source-space inside the build-space and puts the result in the result-space. In that light the trio source / build / result makes sense as a generalization of source / build / {devel | install}.

I dislike use-space. Is this a space that I use, or that catkin uses? Is it used before buidling, while building, after building the current workspace?

One remark about potential confusion: You mentioned that it shoudl be called space, since everything is called space. This analogy might lead to the conclusion that the user expects and actual folder named result in the catkin workspace. Is it possible to put something in the name that indicates that the "result space" is a "meta space", that somehow distinguishes it from the other "real" spaces. I have no good suggestions unfortunately. Maybe by convention it could be referred to in documentation as <result-space> or RESULT-SPACE only (or similar).

[jbohren]

+1 to result-space and also +1 to @NikolausDemmel's comment on sensitivity when talking about different types of result spaces.

[mikepurvis]

My immediate preference is toward result space, for the reason that it seems to work better to say things like:

Your catkin build invocation will always produce a devel result space, and optionally also an install result space, if you specify the install build target. To run stuff from either result space, first source the appropriate setup file, eg:

$ source <result-space>/setup.bash   # where <result-space> is either devel or install
$ roslaunch my-awesome-thing.launch

[jack-oquin]

+1 result-space

[wjwwood]

I like result space as well, and with respect to the result space getting confused with a "real" space like build, devel, or install, you could instead phrase it like this: "... then source a result space, devel or install, ..." once before from then on just using result space. This is often done for acronyms when they are first introduced.

[dirk-thomas]

Please see #79 for the added definition of a result space. I intentionally kept the definition short since both the development space as well as the install space already state the details about FHS layout and setup files.

If you have further feedback on the definition of a result space please propose complete paragraphs to alter/extend the referenced pull request.

[dirk-thomas]

I have updated the REP with the result space term.

Based on that I have added references to some wiki pages (see first comment).

@jack-oquin Can you please review the .rst based documentation if it uses the various terms of spaces as defined in the REP (and update it to use the new result space term where adequate)?

@wjwwood Can you please update the Python tools to use the term as defined in the REP - especially:

• catkin_tools
• wstool
• vcstools

[wjwwood]

Do you have some thing particular in mind for wstool or vcstools? To my mind I can't think of where they would refer to devel or install spaces.

I'll do a sweep through catkin_tools.

[dirk-thomas]

I am just collecting the things mentioned in this ticket: https://github.com/ros-infrastructure/rep/issues/78#issuecomment-45722388

It is more about checking if something needs to be changed.

[jack-oquin]

I'll do it, but not until next week.

[NikolausDemmel]

Are we adding some guidline like

with respect to the result space getting confused with a "real" space like build, devel, or install, you could instead phrase it like this: "... then source a result space, devel or install, ..." once before from then on just using result space. This is often done for acronyms when they are first introduced.

or

Maybe by convention it could be referred to in documentation as or RESULT-SPACE only (or similar).

to the PEP?

[dirk-thomas]

@NikolausDemmel The changes to REP 8 have been discussed in #79 and that PR has already been merged. If you think further changes are necessary please propose them by doing a pull request.

[jack-oquin]

@dirk-thomas: I don't see anywhere in the .rst documentation that needs to be updated for this.

[dirk-thomas]

@jack-oquin @wjwwood Please consider addressing the tasks mentioned in the first comment of this ticket.

[wjwwood]

vcstools and wstool are clean, and @jack-oquin already mentioned that the .rst is fine, I'll just check that off at the top...

[wjwwood]

I'll have a look at catkin_tools, but I think it has already been addressed as well.

[wjwwood]

Yeah @jbohren fixed that here: https://github.com/catkin/catkin_tools/commit/60b98119f9d601c27854a580b3717a1ef231b022

[dirk-thomas]

Ok, please update this (and likely close) once you had the time to check it. Thanks.