Closed mxgrey closed 9 years ago
I think the major motivation of this proposal is giving more flexibility to Skeleton in kinematic structure. In that sense, DART has only the ability of creating BodyNode (based on the current implementation of #369), and this proposal is for filling the missing abilities of removing, moving, and copying BodyNode between Skeletons. I have two additional idea based on this.
There could be more useful abilities for the flexibility. They are very similar to the operations we can apply to a general tree like data structure. Here is some (rough) prototypes of them:
void Skeleton::clear(); // or removeAll()
Skeleton* Skeleton::split(targetBodyNode); // the returning skeleton is the separated skeleton whose root body is targetBodyNode
void Skeleton::merge(targetBodyNode, otherSkeleton or bodyNodeFromOtherSkeleton); // merge other skeleton or body node from other skeleton into the target body node as a child, the other skeleton or body node will be removed
void Skeleton::insert(targetBodyNode, otherSkeleton or bodyNodeFromOtherSkeleton); // insert other skeleton or body node from other skeleton to the target body node as a child, the other skeleton or body node still exist
void Skeleton::swap(body1, body2); // the body nodes can be from either same skeleton or different skeletons
I just omitted considerations on Joint for simplicity. It would be similar to BodyNode::moveTo(~)
or BodyNode::copyTo(~)
.
As suggesting the prototype of API above, some API seems more reasonable when they come in Skeleton's member function, but still BodyNode level API looks useful.
CompositedSkeleton class contains two or multiple Skeletons, and does everything what Skeleton class can. It forms a new tree structure by connecting the Skeletons with certain Joints.
This is a possible solution for the case of two Skeletons are temporarily bonded kinematically. This means the two Skeletons are acting like a single skeleton in kinematics update for certain time duration and they need to be separated after all. For example, a robot (one Skeleton) that holds a cup (another Skeleton) with zero dof. In this case, we cannot use WeldJoint because they need to be separated at the end, nor WeldJointConstraint because we don't compute forward dynamics nor solve dynamic constraints here.
Skeleton::merge(~)
also can be an alternative solution but CompositedSkeleton stores the original Skeleton so that make it easy to split them back while Skeleton::merge(~)
doesn't remember the previous structure.
I definitely like the split
function idea, but I have a few concerns about the others.
(1) How should we deal with Skeletons that have multiple root nodes? Currently, multiple root nodes are still permitted in DART (and I don't see a particular reason that we couldn't keep that). So when we want to merge two entire Skeletons together, should all root nodes of the Skeleton become children of the new parent BodyNode? And should they all be given the specified Joint type when a new Joint type is specified? If that approach would be okay, then it's not a problem.
(2) You say that for merge
, "the other skeleton will be removed", but now Skeletons are managed by shared_ptrs. I'm thinking we should just clear everything out of the old Skeleton rather than attempt to delete it.
(3) I'm thinking about the CompositedSkeleton class... It might be tricky to implement, or it might not be; I'm honestly not sure. I'm having trouble thinking of an obvious way to implement it. Does it inherit the Skeleton class directly? Do we populate its mBodyNodes with all the BodyNodes of all the Skeletons it's composed of? If so, how do we handle things like indexing for BodyNodes, Joints, and DegreesOfFreedom? There would be index disagreement between the CompositedSkeleton and the original Skeletons that make it up. I definitely like the idea of being able to easily split apart Skeletons that were previously merged together, so I would like the CompositedSkeleton idea to work out in some way... it's just not immediately clear to me how.
Let me try to answer (1) and (2) first.
(1) When we merge two Skeletons into one Skeleton or put two Skeletons into a CompositedSkeleton, the root node is always single by adding one of the Skeleton's root node to a certain leaf node in another Skeleton as a child. So we always have single tree structure.
(2) You are correct. Actually I meant clearing the Skeleton so that it doesn't contain any BodyNode nor Joint.
For (1) what I'm referring to is the fact that DART currently allows us to have multiple independent trees within a single Skeleton. You can do this by creating multiple BodyNodes in a Skeleton that all have a nullptr for a parent BodyNode. Do we intend to strictly enforce that each Skeleton is a single tree? I'm not sure if that would be necessary or desirable.
We haven't discussed whether we should enforce that each Skeleton is a single tree or not, but it seems I just allowed it int this commit since there is no specific reason not to allow and no technical issue except for Skeleton::getRootBodyNode()
, which is not critical.
When would it be useful to allow a single Skeleton to have multiple trees?
One use case that I can imagine is if you want to enforce uniqueness in BodyNode and Joint names for multiple trees simultaneously. Or if you have constraint functions that can generalize to multiple trees of BodyNodes but you want your function to use a single Jacobian for all of them simultaneously, or have access to all those trees of BodyNodes through a single Skeleton API.
I have a proposal as an alternative to the CompositedSkeleton
class:
We could have a class called something like SubSkeleton
, ReferenceSkeleton
, or ProxySkeleton
which provides most of the same functionality as the Skeleton class (able to index to BodyNodes and Joints, compute Jacobians, compute mass matrices, etc) except it does not have authority over the BodyNode/Joint management and it does not have to represent a full Skeleton from the root to the leaves.
Instead, the SubSkeleton
would act like a proxy for accessing some arbitrary BodyNodes of an actual Skeleton. The SubSkeleton
would be agnostic to how the BodyNodes are really arranged or what actual Skeletons they belong to. This way you can mangle up Skeletons however you'd like, and as long as you hang onto a SubSkeleton
that matches the original layout, you can always recreate the original. Of course we can create convenience functions to facilitate the recreation.
Also, I'd be happy to hear better suggestions for the class name. I'm having trouble coming up with a name that sounds good to me.
Maybe MetaSkeleton
?
SubSkeleton
looks a useful class. But it seems not possible to compose distinct skeletons. Is this means that SubSkeleton
is not a solution for the case of temporary merger of distinct skeletons (like the case that a robot holds a cup temporarily)?
Here's how I imagine it working:
Let's say you start out with two skeletons named robot
and cup
. You can create MetaSkeleton
for each one: meta_robot
and meta_cup
. Then you could attach cup
to a BodyNode in robot
using a command like MetaSkeleton::merge
. All the contents of the cup
Skeleton will be moved into the robot
Skeleton, but meta_cup
will remain the same, because it only cares about pointers, which will not be changed. Later on when you want to split them, you could call MetaSkeleton::split
, and the contents will be transferred into a new Skeleton (or into the original, if you pass it as an argument to split
). Again, meta_cup
will remain unaffected.
I like the idea of MetaSkeleton
. It sounds like opposite approach to CompositedSkeleton
. Both approaches can be solutions for the robot-cup problem, but MetaSkeleton
seems more straightforward to implement.
I think your proposal is similar to how Eigen
concatenate matrices and manipulates block of a matrix. Eigen
creates a new actual matrix to concatenate matrices like using Skeleton::merge
, and uses Eigen::Matrix<~>::block(~)
to manipulate block of the matrix like using MetaSkeleton
.
This is minor check. merge
and split
shouldn't be a member function of Skeleton
instead of MetaSkeleton
? I thought MetaSkeleton
doesn't have the authority over the BodyNode/Joint management.
One concern is MetaSkeleton
shouldn't be regarded a independent skeleton. I mean if a controller attempts to control a MetaSkeleton
, which refers a sub-tree of a skeleton, it wouldn't act as the controller expected since the MetaSkeleton
will be interfered by the connected joints and bodies.
I think your proposal is similar to how Eigen concatenate matrices and manipulates block of a matrix.
The idea definitely draws inspiration from Eigen's block semantics.
This is minor check. merge and split shouldn't be a member function of Skeleton instead of MetaSkeleton?
The Skeletons would be responsible for managing the memory (as in, owning the BodyNodes and deleting them when the Skeleton's reference count drops to zero), but the MetaSkeleton would offer an API that facilitates moving the BodyNodes around between Skeletons. It exists for convenience.
The reason I personally wouldn't want something like Skeleton::merge
is because both Skeletons will still continue to exist afterwards (it's just that one of them would be left empty), so it might be deceptive to claim that we're "merging the Skeletons". That said, there's no reason we couldn't offer the API for it. If having it would make sense to other people, then I wouldn't be opposed to it.
I mean if a controller attempts to control a MetaSkeleton
Part of my motivation for the MetaSkeleton was to allow controllers that only operate on portions of a Skeleton to work in the same way that they would have operated on a full Skeleton. For example, let's say you write two different controllers for two different robotic arms, but now you want to attach both arms to the same Skeleton without needing to rewrite the controllers. If you design the controllers to work on a MetaSkeleton, then there won't be any problem there.
the MetaSkeleton will be interfered by the connected joints and bodies.
I'm guessing you're concerned about the scenario where we might have a structure like [link 1] -> [link 2] -> [link 3], but the MetaSkeleton only contains [link 1] and [link 3], so the resulting behavior of the control input might not be what the controller expected.
I'm thinking about offering a couple extensions of MetaSkeleton. I haven't decided on the names yet, but conceptually they'd be something like ConnectedMetaSkeleton
and SubtreeMetaSkeleton
.
ConnectedMetaSkeleton would enforce that all of its contents are fully connected. SubtreeMetaSkeleton would enforce that it contains all the BodyNodes that descend from an arbitrary BodyNode. So if your controller depends on one of those properties, you could select one of those Skeleton types for it to operate on.
Also, a Skeleton would qualify as a MetaSkeleton, ConnectedMetaSkeleton, and SubtreeMetaSkeleton.
Maybe ConnectedMetaSkeleton
could be called Linkage
and SubtreeMetaSkeleton
could be called SubSkeleton
.
The BodyNode transferring features have been finished in pull request #369. Now we just need to decide on the nature of the MetaSkeleton and its extensions. Here are my thoughts so far:
MetaSkeleton
will be a pure abstract base class that provides a common interface for setting and getting important data from Skeletons and subsets of Skeletons, such as (but definitely not limited to) the following:
MetaSkeleton::setPositions(Eigen::VectorXd)
MetaSkeleton::getPositions()
MetaSkeleton::setVelocities()
MetaSkeleton::getVelocities()
MetaSkeleton::getMass()
MetaSkeleton::getJacobian()
MetaSkeleton::getCOM()
MetaSkeleton::getCOMJacobian()
MetaSkeleton::getMassMatrix()
MetaSkeleton::getBodyNode(size_t)
MetaSkeleton::getJoint(size_t)
The current implementation of the Skeleton class will remain exactly the same, except that it inherits MetaSkeleton and overrides MetaSkeleton's pure virtual functions.
Alongside the standard Skeleton extension of MetaSkeleton, there will also be a ReferentialSkeleton that extends MetaSkeleton. ReferentialSkeleton will use a list of BodyNodePtrs where the BodyNodePtrs refer to BodyNodes that are managed by a standard Skeleton somewhere else. ReferentialSkeleton will only exist as a base class for implementing the referential versions of MetaSkeleton's functions. You will not be able to directly construct a ReferentialSkeleton; instead, there will be four extension classes that can be used to construct it:
(1) The Linkage
class will inherit ReferentialSkeleton, and it will enforce that all the BodyNodePtrs that it contains are fully connected during construction. It does not have to include all the descendents of the BodyNodePtrs it contains, but it must self-contain an unbroken kinematic chain between any two BodyNodePtrs that it holds. However, these rules would only apply during construction; it cannot block a user from rearranging the BodyNodes after it has been constructed. But it would offer a function, let's call it bool Linkage::disassembled() const
which returns true if the relevant parent/child relationships of its BodyNodes have changed, and returns false otherwise. It would also offer two functions to reconstruct the Linkage: [1] Linkage::reassemble()
which would return the BodyNodes to the assembly that they had when the Linkage was originally constructed (it would not, however, return any of their BodyNode Properties or Joint Properties/types to what they were originally), and then [2] Linkage::update()
would use the input that was provided during construction to reformulate what it considers the Linkage to be, without modifying the assembly of the BodyNodes. This class will offer a SkeletonPtr Linkage::clone() const
function which produces a cloned Skeleton that matches the original assembly that the Linkage had during construction (or during the last call to Linkage::update()
). A valid Linkage is able to include BodyNodePtrs from multiple different Skeletons, as long as a path to the root of its Skeleton is included for each BodyNodePtr.
(2) The Branch
class will inherit Linkage (because a Branch is always a Linkage, but a Linkage is not always a Branch), and it will enforce that all BodyNodePtrs that descend from a single specific BodyNode are included. That single BodyNode will always be the sole root node of the Branch. Unlike Linkage, adding new child BodyNodes to the members of a Branch will invalidate the Branch. Calling Branch::reassemble()
will cut off any new children that were not present when the Branch was constructed. Calling Branch::update()
will pull in any new children so that they are included in the Branch. In addition to the clone()
function offered by the Linkage class, Branch will also offer a Branch::moveTo()
function. It will be the same as using the BodyNode::moveTo()
function on the root BodyNode of the Branch. This means that any new children that were added after the Branch was constructed will also be moved, even if the Branch has not been updated (or as an alternative, we could make it so that the Branch calls reassemble()
before performing the move. Would that be preferable?).
(3) The Chain
class will also inherit Linkage
. It will not offer any new functionality; it will just have a stricter constructor. A Chain must consist of a direct path from one BodyNode to another BodyNode, and there must not be any branching taking place along that path.
(4) Finally, the Disjointment
class (the name is pending; I'm open to suggestions) will be a ReferentialSkeleton (not a Linkage), and it will just be an arbitrary collection of BodyNodePtrs. The BodyNodePtrs can be added or removed arbitrarily and without consequence. This class will not offer any cloning, moving, or reassembling features.
Linkage, Branch, and Disjointment can all be implicitly constructed by passing in a SkeletonPtr, so if you have an API that expects one of those three types, you can just pass a SkeletonPtr into it without needing to worry about constructing one of those types yourself.
In DART's custom file format (.skel), you will be able to specify Linkages, Branches, Chains, and Disjointments for each Skeleton, and after the Skeleton is loaded you can access them using:
LinkagePtr Skeleton::getLinkage(std::string)
ConstLinkagePtr Skeleton::getLinkage(std::string) const
BranchPtr Skeleton::getBranch(std::string)
ConstBranchPtr Skeleton::getBranch(std::string) const
ChainPtr Skeleton::getChain(std::string)
ConstChainPtr Skeleton::getChain(std::string) const
DisjointmentPtr Skeleton::getDisjointment(std::string)
ConstDisjointmentPtr Skeleton::getDisjointment(std::string) const
Each function will have a version that accepts a std::string (name uniqueness will be enforced for each type) or a size_t, and const-correctness will of course be exercised.
If anyone has feedback or suggestions, don't hesitate to share.
I have a question to pose: Should the ReferentialSkeleton class enforce name uniqueness among the BodyNodes and Joints that it refers to?
I'm thinking we would want to be able to access the members of a ReferentialSkeleton based on their names, but there are only two ways that could work: (1) ReferentialSkeleton must enforce name uniqueness, or (2) ReferentialSkeleton must only contain BodyNodes from one Skeleton at a time. I think (2) might be undesirable, because then there would be no way to group together BodyNodes from separate Skeletons, and the ReferentialSkeleton would need to be aware of whenever a BodyNode that it's referring to gets moved to another Skeleton. Option (1) wouldn't really be an issue, except that the ReferentialSkeleton would need to set up callbacks for whenever a name gets changed, which kills how light-weighted the class was meant to be.
Option (3) would be to not allow members of a ReferentialSkeleton to be accessed by name. Instead, the user will be able to arbitrarily set the indexing order to something of their choosing. Maybe letting the user set the indexing is enough to make up for not being able to access by name?
Overall, this design looks good. We like the generic ReferentialSkeleton
base class and concrete implementations for Skeleton
, Linkage
, Branch
, Chain
, and Disjointment
. We have a few questions:
ReferentialSkeleton
is no longer valid? For example, if I remove a BodyNode
from a Branch
. We would prefer that this fails gracefully, e.g. by returning an error code or throwing an exception, rather than SEGFAULTing or producing invalid output.update
, disassemble
, and reassemble
? These functions implement some non-trivial behavior. We are having trouble coming up with an example application where this is useful. Could you share your a use case?ReferentialSkeleton
be registered on a class? We like the ability to load named ReferentialSkeleton
s from .skel
files. However, we would also like the ability to create a "temporary" ReferentialSkeleton
without assigning it a unique name and attaching it to a Skeleton
. Is this possible in the proposed API?Finally, we agree that it is undesirable to register callbacks in each ReferentialSkeleton
to enforce name uniqueness. This may also make it difficult to generate unique names: the set of possible collisions may change every time a new ReferentialSkeleton
is constructed.
Instead, we propose the following:
Skeleton
ownerReferentalSkeleton
take:
std::string name
(required)Skeleton *owner
(optional, defaults to nullptr
)owner
is not specified and the name
is unique, return the resultowner
is not specified and the name
is not unique, return nullptr
ReferentialSkeleton
Here are our thoughts:
std::string
when names are unique.Tagging @psigen @siddhss5.
What will an operation do if its ReferentialSkeleton is no longer valid?
At the level of the ReferentialSkeleton
, there is no issue of whether or not it is valid. The ReferentialSkeleton will hold onto strong BodyNodePtrs which will ensure that the BodyNodes it refers to will stay alive. So even if the remove
operation is performed on a BodyNode that a ReferentialSkeleton refers to, that BodyNode will just get placed into its own Skeleton, and the ReferentialSkeleton
can still perform operations on it. The idea of validity only pertains to the more strict Linkage
, Branch
, and Chain
class. Probably 95% of the implementation of these classes will be done at the level of ReferentialSkeleton
, so they will just behave like a generic ReferentialSkeleton after they have gone "invalid" or been "disassembled". The thing is, these classes can't actually know that they've gone invalid without either (1) an O(n) operation, or (2) callbacks that monitor when BodyNodes get moved around. I'm trying to avoid option (2), because I'd prefer if the ReferentialSkeleton and its extensions are as light-weight as possible: ideally just a collection of BodyNodePtrs, some data caching, and a convenient API. Also, we don't want to perform option (1) every time one of the ReferentialSkeleton functions is called, because that seems frivolously wasteful.
I don't think we actually need to put much effort into monitoring the validity of Linkage and its extensions, because the DART library itself will never unexpectedly rearrange BodyNodes. Only the user can change the structural layout of a Skeleton. Of course, in a complex code base you can't necessarily know that some other component of the software didn't break apart your Linkage and invalidate it. To that end, if you both (1) believe that some other portion of your software might unexpectedly alter the layout of your Skeleton, and (2) you strictly require a certain layout for your Linkage, then at the start of your code you should perform the O(n) operation of option (1) in the previous paragraph to check whether it is still valid. If it is still valid, then lock the mutexes of all relevant Skeletons, do your work, and then release the mutexes. If it is not valid, then do whatever error reporting you would like. I think it makes sense to put this responsibility on the user, since only the user can change the layout of a Skeleton in the first place.
The main reason I would like to avoid anything that involves callbacks for ReferentialSkeletons is because I'd be worried about clogging up basic operations. If a BodyNode can have an arbitrary number of ReferentialSkeletons that refer to it, then every moveTo
, setName
, setMass
, setPosition
, etc operation might need to trickle through hundreds or thousands of these satellite ReferentialSkeleton objects. I think it would be better if they were more like detached observers where there isn't any overhead for their mere existence (besides taking up memory).
What is the motivation for update, disassemble, and reassemble?
Just to be clear, there isn't a disassemble
function, there's a bool disassembled
function which returns true
if the Linkage has been invalided. We should probably call it isDisassembled
instead. Or I'm open to other suggestions for the function name.
The easiest example that comes to mind is if you're interested in the right arm of a robot, and that right arm might pick something up. You could create two Branches
where the root BodyNode for each Branch
is the right shoulder. We'll name one Branch
"RightArmOriginal" and the other "RightArm". When the robot picks something up, you call update()
on RightArm while leaving RightArmOriginal alone, and then you can do your planning/controlling operations with RightArm and the newly picked up object will be included. Once you want to drop the item, you could call reassemble()
on RightArmOriginal and then update()
on RightArm, which would revert the BodyNode layout to what it was before the item was picked up. This would allow you to cheaply preserve a model of the original right arm layout and then easily revert to that layout once you want to undo any changes that have occurred.
Must a ReferentialSkeleton be registered on a class?
Nope, you can create ReferentialSkeletons however and wherever you'd like. There will be no naming restrictions on ReferentialSkeletons, except for those that are registered to a specific Skeleton.
we would also like the ability to create a "temporary" ReferentialSkeleton without assigning it a unique name
This will definitely be an option. And you can always register a ReferentialSkeleton with a Skeleton after you've created it.
Instead, we propose the following:
I liked the sound of the proposal at first, but then I realized there are a couple issues:
(1) In order for the ReferentialSkeleton to keep an up-to-date map of the BodyNode names, it would still need to use callbacks. The name of a BodyNode that it refers to could be changed at any moment.
(2) If you need a Skeleton pointer to access a BodyNode, then you may as well just call getBodyNode(std::string)
on the Skeleton directly. It wouldn't accomplish anything to wind it through the ReferentialSkeleton at that point.
I can offer a counter-proposal:
The ReferentialSkeleton could keep a list of Skeletons that its BodyNodes belong to. This list would get refreshed during construction of the ReferentialSkeleton as well as each time an update()
is called on the ReferentialSkeleton. It can then cycle through the list of Skeletons, calling Skeleton::getBodyNode(std::string)
on each one, and then use the behavior that you prescribed in your proposal. Honestly, I'm not sure how valuable this is, since it would entail an O(log(n)) nested within an O(n) operation, which might be more expensive than an end user expects for a simple BodyNode lookup.
I'm feeling somewhat inclined to just not offer the ability to lookup BodyNodes by name in ReferentialSkeletons, and make up for it by allowing users to set the indexing of ReferentialSkeletons however they'd like to. This feels like a fair trade off to me, but we can keep discussing it if it sounds bad to you guys.
I also think that it is important to keep ReferentialSkeleton
as lightweight as possible; i.e. not register callbacks. Manually calling disassembled()
when necessary seems like an acceptable trade off.
It would be nice to lookup BodyNode
s by name in a ReferentialSkeleton
, but this is by no means a deal breaker. I am fine proceeding without this functionality since there is no clear solution.
One question: What does it mean to load/save a ReferentialSkeleton
that contains multiple skeletons from/to a .skel
? I presume each .skel
only contains one Skeleton
, so this is a bit strange. Will the loading/saving function throw an error?
@psigen Do you have anything to add?
It would be nice to lookup BodyNodes by name in a ReferentialSkeleton, but this is by no means a deal breaker. I am fine proceeding without this functionality since there is no clear solution.
One viable option might be to allow ReferentialSkeleton to have name management that is independent of the Skeleton, and independent of the BodyNode's "official" names. When constructing a ReferentialSkeleton, it could automatically use whatever the BodyNodes' names are at that moment and then allow the user to change what the ReferentialSkeleton believes the names are. The obvious issue here is that refSkeleton->getBodyNode( bodyNode->getName() )
would fail if the ReferentialSkeleton doesn't agree with the BodyNode about its name, and that seems gross.
I presume each .skel only contains one
Skeleton
Somewhat unintuitively, a .skel file can contain a simulation::World that contains an array of Skeletons. I do wonder if it would make sense to have separate file extensions for a file containing a single Skeleton versus a file containing a World. That should probably be brought up in its own issue.
It's not at all obvious to me how to specify ReferentialSkeletons that encapsulate more than one Skeleton and then deliver them to the user in the current scheme. Most likely, we'll need a new file scheme. I'll start a new issue about this.
The features here have been finished in #369
An extremely useful potential feature currently missing from DART is the ability to freely move BodyNodes between Skeletons. I have a proposal for how this could be added to the latest memory_management branch #369. We would use the following functions:
The
BodyNode::remove()
function would remove the BodyNode and all its children (recursively) from their Skeleton, and delete them.The
BodyNode::moveTo(~)
function would move the BodyNode and all its children (recursively) to the specified Skeleton, giving it the specified BodyNode as its new parent. The specified new parent BodyNode must already be a member of the new Skeleton (or it must be a nullptr). This function can also be used to move a BodyNode around within a single Skeleton. Note that all the pointers for the BodyNodes and Joints will remain the same as they get moved from one Skeleton to another.The templated
BodyNode::moveTo<JointType>(~)
function would do the same thing as the non-templated version, except it would create a new parent Joint of the specified type. This function could also be used to simply change the existing Joint type of a BodyNode without changing its parent. Like the non-templated version, the pointers for all the BodyNodes and Joints will remain the same after being moved, except the new parent Joint will always use a new pointer.The
BodyNode::copyTo(~)
functions will behave like theBodyNode::moveTo(~)
functions, except it will create clones of the BodyNodes and attach the clones to the new Skeleton instead of moving the originals. Obviously the clones will all have new pointers.