L434

[studywolf]

https://github.com/tcstewar/2015-Embodied_Benchmarks/blob/master/paper/paper.tex#L434

Suggested change to keep the notation for velocity the same as above

"Let the system state $q$ and velocity $\dot{q}$ be vectors of length $N$, where $N$ is the number of joints. The force applied by each motor is related to the signal sent to the motor $u$, but will generally have some maximum value $T$, so we use $tanh(u)T$. For friction, we simply scale the velocity by some factor $F$ every time step.
This results in the simplistic simulation as follows:

\begin{align} \Delta \dot{q} & = -\dot{q} F + tanh(u)T \ \Delta q & = \dot{q} \end{align}"

also is there a $dt$ variable in the $\Delta q$ missing?

[tcstewar]

I was of two minds while describing this. If I call velocity \dot{q} then Equation 2 looks really weird (\Delta q & = \dot{q}) since the relationship is right there as part of the term. It's the same as writing "\Delta q = dq/dt", which can either be thought of as redundant or wrong (since it's a discrete-time simulation). So when writing an algorithm for saying what the simulation is doing, I think it's clearer to have q and v be separate variables, and then be explicit about the relationship between them.

I'd originally written it as v_{t+1} = v_t - (v_t F + tanh(ut)T)*dt q{t+1} = q_t + v_t * dt

But then rewrote it in the Delta form so as to not have to worry about all the subscripts. And I made the dt kind of implicit, just because I thought it was being distracting from the important part of the algorithm, which is the basic formula there.

But, I could also see that paring it down to those essentials might make it look like I forgot about dt and things like that in the actual implementation, so I could see going back to that.

Hm. Not sure how I feel on this one. but using $\dot{q}$ here instead of $v$ feels wrong to me. I think I'll poke around some other papers that describe algorithms like this and see what they do... I'd be more tempted to use $v$ elsewhere, for that sort of consistency....

[studywolf]

Right! Sorry i don't know why i left in \Delta, normally that would be written \ddot{q} = \dot{q}. Hm, i'm curious about $\dot{q}$ feeling wrong. I think what you get out of using it is the relationship between $q$ and $\dot{q}$ is for free, and you don't have to worry about keeping an extra variable. But people also use the separate variables to convert to a 1st order system, still with the \dot, and denotation \dot{v} = -v F+tanh(u) T \dot{q} = v but yeah as long as the $v$ or $\dot{q}$ is consistent it probably doesn't matter really, i'm not sure what would be more familiar to the field.

[tcstewar]

Right! Sorry i don't know why i left in \Delta, normally that would be written \ddot{q} = \dot{q}.

No, it'd be \dot{q} = \dot{q} .....

I think my problem is that \dot{q} doesn't have any meaning in a discrete-time-step simulation, since there are no instantaneous temporal derivatives. \dot{q} is dq/dt, which is only approximately the same as \Delta q / \Delta t. So I was fine referring to \dot{q} when talking about the PID controller, since things were pretty abstract then, and that's how people always write the PID controller equations, since they're generally thinking about controlling a realtime system, but now when timesteps start being involved it feels weird.

I think I'll try going through and replacing $\dot{q}$ with $v$ everywhere and see how that feels. If I don't like that then I'll try being very explicit that $v$ is the discrete-time simulation equivalent of $\dot{q}$....

[studywolf]

man yeah i am all over the place. jeeeeeeeezzzz right yeah d/dt [q // dq] = [dq // dq * F + tanh...] hmm, i'm not sure that that's the norm, usually \dot{q} is used and everyone accepts that if you're implementing it on a computer you'll just discretize (e.g. https://homes.cs.washington.edu/~todorov/papers/LiICINCO04.pdf where the controller description uses the discrete timesteps explicitly, but in describing the plant in section 3 they use \dot{x}), it might be an unnecessary formalization to distinguish between discrete and continuous. But either way! As long as it reads easy.