Pusher#

This environment is part of the Mujoco environments. Please read that page first for general information.


Action Space	Box(-2.0, 2.0, (7,), float32)
Observation Shape	(23,)
Observation High	[inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf]
Observation Low	[-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf]
Import	`gym.make("Pusher-v4")`

Description#

“Pusher” is a multi-jointed robot arm which is very similar to that of a human. The goal is to move a target cylinder (called object) to a goal position using the robot’s end effector (called fingertip). The robot consists of shoulder, elbow, forearm, and wrist joints.

Action Space#

The action space is a Box(-2, 2, (7,), float32). An action (a, b) represents the torques applied at the hinge joints.

Num	Action	Control Min	Control Max	Name (in corresponding XML file)	Joint	Unit
0	Rotation of the panning the shoulder	-2	2	r_shoulder_pan_joint	hinge	torque (N m)
1	Rotation of the shoulder lifting joint	-2	2	r_shoulder_lift_joint	hinge	torque (N m)
2	Rotation of the shoulder rolling joint	-2	2	r_upper_arm_roll_joint	hinge	torque (N m)
3	Rotation of hinge joint that flexed the elbow	-2	2	r_elbow_flex_joint	hinge	torque (N m)
4	Rotation of hinge that rolls the forearm	-2	2	r_forearm_roll_joint	hinge	torque (N m)
5	Rotation of flexing the wrist	-2	2	r_wrist_flex_joint	hinge	torque (N m)
6	Rotation of rolling the wrist	-2	2	r_wrist_roll_joint	hinge	torque (N m)

Observation Space#

Observations consist of

Angle of rotational joints on the pusher
Angular velocities of rotational joints on the pusher
The coordinates of the fingertip of the pusher
The coordinates of the object to be moved
The coordinates of the goal position

The observation is a ndarray with shape (23,) where the elements correspond to the table below. An analogy can be drawn to a human arm in order to help understand the state space, with the words flex and roll meaning the same as human joints.

Num	Observation	Min	Max	Name (in corresponding XML file)	Joint	Unit
0	Rotation of the panning the shoulder	-Inf	Inf	r_shoulder_pan_joint	hinge	angle (rad)
1	Rotation of the shoulder lifting joint	-Inf	Inf	r_shoulder_lift_joint	hinge	angle (rad)
2	Rotation of the shoulder rolling joint	-Inf	Inf	r_upper_arm_roll_joint	hinge	angle (rad)
3	Rotation of hinge joint that flexed the elbow	-Inf	Inf	r_elbow_flex_joint	hinge	angle (rad)
4	Rotation of hinge that rolls the forearm	-Inf	Inf	r_forearm_roll_joint	hinge	angle (rad)
5	Rotation of flexing the wrist	-Inf	Inf	r_wrist_flex_joint	hinge	angle (rad)
6	Rotation of rolling the wrist	-Inf	Inf	r_wrist_roll_joint	hinge	angle (rad)
7	Rotational velocity of the panning the shoulder	-Inf	Inf	r_shoulder_pan_joint	hinge	angular velocity (rad/s)
8	Rotational velocity of the shoulder lifting joint	-Inf	Inf	r_shoulder_lift_joint	hinge	angular velocity (rad/s)
9	Rotational velocity of the shoulder rolling joint	-Inf	Inf	r_upper_arm_roll_joint	hinge	angular velocity (rad/s)
10	Rotational velocity of hinge joint that flexed the elbow	-Inf	Inf	r_elbow_flex_joint	hinge	angular velocity (rad/s)
11	Rotational velocity of hinge that rolls the forearm	-Inf	Inf	r_forearm_roll_joint	hinge	angular velocity (rad/s)
12	Rotational velocity of flexing the wrist	-Inf	Inf	r_wrist_flex_joint	hinge	angular velocity (rad/s)
13	Rotational velocity of rolling the wrist	-Inf	Inf	r_wrist_roll_joint	hinge	angular velocity (rad/s)
14	x-coordinate of the fingertip of the pusher	-Inf	Inf	tips_arm	slide	position (m)
15	y-coordinate of the fingertip of the pusher	-Inf	Inf	tips_arm	slide	position (m)
16	z-coordinate of the fingertip of the pusher	-Inf	Inf	tips_arm	slide	position (m)
17	x-coordinate of the object to be moved	-Inf	Inf	object (obj_slidex)	slide	position (m)
18	y-coordinate of the object to be moved	-Inf	Inf	object (obj_slidey)	slide	position (m)
19	z-coordinate of the object to be moved	-Inf	Inf	object	cylinder	position (m)
20	x-coordinate of the goal position of the object	-Inf	Inf	goal (goal_slidex)	slide	position (m)
21	y-coordinate of the goal position of the object	-Inf	Inf	goal (goal_slidey)	slide	position (m)
22	z-coordinate of the goal position of the object	-Inf	Inf	goal	sphere	position (m)

Rewards#

The reward consists of two parts:

*reward_near *: This reward is a measure of how far the fingertip of the pusher (the unattached end) is from the object, with a more negative value assigned for when the pusher’s fingertip is further away from the target. It is calculated as the negative vector norm of (position of the fingertip - position of target), or -norm(“fingertip” - “target”).
*reward_dist *: This reward is a measure of how far the object is from the target goal position, with a more negative value assigned for object is further away from the target. It is calculated as the negative vector norm of (position of the object - position of goal), or -norm(“object” - “target”).
reward_control: A negative reward for penalising the pusher if it takes actions that are too large. It is measured as the negative squared Euclidean norm of the action, i.e. as - sum(action²).

The total reward returned is reward = reward_dist + 0.1 * reward_ctrl + 0.5 * reward_near

Unlike other environments, Pusher does not allow you to specify weights for the individual reward terms. However, info does contain the keys reward_dist and reward_ctrl. Thus, if you’d like to weight the terms, you should create a wrapper that computes the weighted reward from info.

Starting State#

All pusher (not including object and goal) states start in (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0). A uniform noise in the range [-0.005, 0.005] is added to the velocity attributes only. The velocities of the object and goal are permanently set to 0. The object’s x-position is selected uniformly between [-0.3, 0] while the y-position is selected uniformly between [-0.2, 0.2], and this process is repeated until the vector norm between the object’s (x,y) position and origin is not greater than 0.17. The goal always have the same position of (0.45, -0.05, -0.323).

The default framerate is 5 with each frame lasting for 0.01, giving rise to a dt = 5 * 0.01 = 0.05

Episode End#

The episode ends when any of the following happens:

Truncation: The episode duration reaches a 100 timesteps.
Termination: Any of the state space values is no longer finite.

Arguments#

No additional arguments are currently supported (in v2 and lower), but modifications can be made to the XML file in the assets folder (or by changing the path to a modified XML file in another folder)..

env = gym.make('Pusher-v4')

There is no v3 for Pusher, unlike the robot environments where a v3 and beyond take gym.make kwargs such as xml_file, ctrl_cost_weight, reset_noise_scale etc.

Version History#

v4: all mujoco environments now use the mujoco bindings in mujoco>=2.1.3
v2: All continuous control environments now use mujoco_py >= 1.50
v1: max_time_steps raised to 1000 for robot based tasks (not including reacher, which has a max_time_steps of 50). Added reward_threshold to environments.
v0: Initial versions release (1.0.0)