stanfordnmbl / osim-rl

Reinforcement learning environments with musculoskeletal models
http://osim-rl.stanford.edu/
MIT License
877 stars 248 forks source link

Constant force applied to a point #221

Closed annshorn closed 4 years ago

annshorn commented 4 years ago

Hi, I'm trying to make my own env based of your code. And here the question that I have - how can I deal with damping of the oscillations?

I took your code and added "CoordinateActuator" to shoulder (like pendulum):

import opensim as osim 
import numpy as np

#your code
arm = osim.Model('/Users/anna/open_sim/models/arm2dof6musc.osim')
arm.setUseVisualizer(True)

muscleSet = arm.getMuscles()
forceSet = arm.getForceSet()
bodySet = arm.getBodySet()
jointSet = arm.getJointSet()
markerSet = arm.getMarkerSet()
contactGeometrySet = arm.getContactGeometrySet()

# Add a controller 
brain = osim.PrescribedController()

# console reporter

reporter = osim.ConsoleReporter()
reporter.set_report_time_interval(0.1)
reporter.addToReport(jointSet.get(1).getCoordinate().getOutput("value"), 'r_shoulder')
reporter.addToReport(jointSet.get(2).getCoordinate().getOutput("value"), 'r_elbow')
arm.addComponent(reporter)

# that's what I added
# ----------------------------------------------------------------------------
coordAct = osim.CoordinateActuator('r_shoulder_elev')
coordAct.setName('coordAct_r_shoulder')           # Name
coordAct.setOptimalForce(1.0)            # Maximum generalized force

coordAct.setMinControl(-np.inf)              # Minimum control signal allowed
coordAct.setMaxControl(np.inf)               # Maximum control signal allowed

# Add the force to the model.
arm.addForce(coordAct)

brain.addActuator(coordAct)
brain.prescribeControlForActuator("coordAct_r_shoulder", osim.Constant(1.0))
arm.addController(brain)
# ----------------------------------------------------------------------------

#continue your code
istep = 0
stepsize = 0.1

state = arm.initSystem()
manager = osim.Manager(arm)

#state.setTime(istep)
manager.setIntegratorAccuracy(5e-5)
manager.initialize(state)

while istep < 1000:
    #function "OsimModel.actuator"
    brain = osim.PrescribedController.safeDownCast(arm.getControllerSet().get(0))
    functionSet = brain.get_ControlFunctions()
    func = osim.Constant.safeDownCast(functionSet.get(0))
    func.setValue( 1.2 )
    #function "OsimModel.integrate"
    istep = istep + 1
    state = manager.integrate(stepsize * istep)

But I have very strange plot, without controller it's just damping oscillation, with controller it looks so: (I suppose that shoulder started to move, then come back to previous point and started move again in the same way)

Снимок экрана 2020-04-30 в 7 59 07

My question: how can I get oscillation without damping (like pure sine wave)? Thank you in advance

kidzik commented 4 years ago

@carmichaelong do you know what might be happening here in the arm2dof6musc.osim model?

Regarding the 'strange plot', it seems like time returns to zero sometimes as if the environment had restarted. How do you define the time variable?

annshorn commented 4 years ago

I don't define time variable, all this data i got from this

# console reporter

reporter = osim.ConsoleReporter()
reporter.set_report_time_interval(0.1)
reporter.addToReport(jointSet.get(1).getCoordinate().getOutput("value"), 'r_shoulder')
reporter.addToReport(jointSet.get(2).getCoordinate().getOutput("value"), 'r_elbow')
arm.addComponent(reporter)

and then open the ".log" file for visualization

annshorn commented 4 years ago

The simple code without controller gave me it:

import opensim as osim 
import numpy as np

arm = osim.Model('/Users/anna/open_sim/models/arm2dof6musc.osim')
arm.setUseVisualizer(True)

muscleSet = arm.getMuscles()
forceSet = arm.getForceSet()
bodySet = arm.getBodySet()
jointSet = arm.getJointSet()
markerSet = arm.getMarkerSet()
contactGeometrySet = arm.getContactGeometrySet()

# ---------------------------------------------------------------------------
# We want to write our simulation results to the console.
# ---------------------------------------------------------------------------

reporter = osim.ConsoleReporter()
reporter.set_report_time_interval(0.1)
reporter.addToReport(jointSet.get(1).getCoordinate().getOutput("value"), 'r_shoulder')
reporter.addToReport(jointSet.get(2).getCoordinate().getOutput("value"), 'r_elbow')
arm.addComponent(reporter)

state = arm.initSystem()

# ---------------------------------------------------------------------------
# Simulate.
# ---------------------------------------------------------------------------

manager = osim.Manager(arm)
state.setTime(0)
manager.initialize(state)
state = manager.integrate(15.0)
Снимок экрана 2020-04-30 в 11 25 33

In some moment in the coordinate of shoulder I got zero value

Снимок экрана 2020-04-30 в 11 27 00
carmichaelong commented 4 years ago

My guess is that this is coming from the "spring-like" behavior of the muscles. Even when a muscle isn't activated, it will create force after it's stretched past a certain length. My guess is that the arm is bouncing up and down against these forces.

annshorn commented 4 years ago

this is strange because the force I apply constantly, that is, in theory, everything should be solved by increasing the applied force over time?

i = 1.0
while istep < 300:
    #function "OsimModel.actuator"
    brain = osim.PrescribedController.safeDownCast(arm.getControllerSet().get(0))
    functionSet = brain.get_ControlFunctions()
    func = osim.Constant.safeDownCast(functionSet.get(0))
    func.setValue( i  )
    i+=0.01
    #function "OsimModel.integrate"
    istep = istep + 1
    state = manager.integrate(stepsize * istep)

gives:

Снимок экрана 2020-05-01 в 3 59 44
carmichaelong commented 4 years ago

I'm not surprised this occurs with a constant force, and I don't think an increasing force would make the oscillations go away either (increasing force would not necessarily keep a spring from oscillating). I think the main issue is that there is minimal damping in the muscles (if any), and the stiffness of the joint will be small when the muscles are not activated. Another property of the muscle that might cause this is that the "spring-like" behavior is actually non-linear and only one way (i.e., it can only pull when its stretched too long, but it won't push if the muscle is too short), but I can't easily predict how that should change a typical spring's response.