Spaces#

Superclass that is used to define observation and action spaces.

Spaces are crucially used in Gym to define the format of valid actions and observations. They serve various purposes:

They clearly define how to interact with environments, i.e. they specify what actions need to look like and what observations will look like
They allow us to work with highly structured data (e.g. in the form of elements of Dict spaces) and painlessly transform them into flat arrays that can be used in learning code
They provide a method to sample random elements. This is especially useful for exploration and debugging.

Different spaces can be combined hierarchically via container spaces (Tuple and Dict) to build a more expressive space

Warning

Custom observation & action spaces can inherit from the Space class. However, most use-cases should be covered by the existing space classes (e.g. Box, Discrete, etc…), and container classes (:class`Tuple` & Dict). Note that parametrized probability distributions (through the Space.sample() method), and batching functions (in gym.vector.VectorEnv), are only well-defined for instances of spaces provided in gym by default. Moreover, some implementations of Reinforcement Learning algorithms might not handle custom spaces properly. Use custom spaces with care.

General Functions#

Each space implements the following functions:

gym.spaces.Space.sample(self, mask: Any | None = None) → T_cov#

Randomly sample an element of this space.

Can be uniform or non-uniform sampling based on boundedness of space.

Parameters:: mask – A mask used for sampling, expected dtype=np.int8 and see sample implementation for expected shape.
Returns:: A sampled actions from the space

gym.spaces.Space.contains(self, x) → bool#: Return boolean specifying if x is a valid member of this space.

property Space.shape: Tuple[int, ...] | None#: Return the shape of the space as an immutable property.

property gym.spaces.Space.dtype#: Return the data type of this space.

gym.spaces.Space.seed(self, seed: int | None = None) → list#: Seed the PRNG of this space and possibly the PRNGs of subspaces.

gym.spaces.Space.to_jsonable(self, sample_n: Sequence[T_cov]) → list#: Convert a batch of samples from this space to a JSONable data type.

gym.spaces.Space.from_jsonable(self, sample_n: list) → List[T_cov]#: Convert a JSONable data type to a batch of samples from this space.

Box#

class gym.spaces.Box(low: ~typing.SupportsFloat | ~numpy.ndarray, high: ~typing.SupportsFloat | ~numpy.ndarray, shape: ~typing.Sequence[int] | None = None, dtype: ~typing.Type = <class 'numpy.float32'>, seed: int | ~numpy.random._generator.Generator | None = None)#

A (possibly unbounded) box in \(\mathbb{R}^n\).

Specifically, a Box represents the Cartesian product of n closed intervals. Each interval has the form of one of \([a, b]\), \((-\infty, b]\), \([a, \infty)\), or \((-\infty, \infty)\).

There are two common use cases:

Identical bound for each dimension:

>>> Box(low=-1.0, high=2.0, shape=(3, 4), dtype=np.float32)
Box(3, 4)

Independent bound for each dimension:

>>> Box(low=np.array([-1.0, -2.0]), high=np.array([2.0, 4.0]), dtype=np.float32)
Box(2,)

__init__(low: ~typing.SupportsFloat | ~numpy.ndarray, high: ~typing.SupportsFloat | ~numpy.ndarray, shape: ~typing.Sequence[int] | None = None, dtype: ~typing.Type = <class 'numpy.float32'>, seed: int | ~numpy.random._generator.Generator | None = None)#

Constructor of Box.

The argument low specifies the lower bound of each dimension and high specifies the upper bounds. I.e., the space that is constructed will be the product of the intervals \([\text{low}[i], \text{high}[i]]\).

If low (or high) is a scalar, the lower bound (or upper bound, respectively) will be assumed to be this value across all dimensions.

Parameters:

low (Union[SupportsFloat, np.ndarray]) – Lower bounds of the intervals.
high (Union[SupportsFloat, np.ndarray]) – Upper bounds of the intervals.
shape (Optional[Sequence[int]]) – The shape is inferred from the shape of low or high np.ndarray`s with `low and high scalars defaulting to a shape of (1,)
dtype – The dtype of the elements of the space. If this is an integer type, the Box is essentially a discrete space.
seed – Optionally, you can use this argument to seed the RNG that is used to sample from the space.

Raises:

ValueError – If no shape information is provided (shape is None, low is None and high is None) then a value error is raised.

is_bounded(manner: str = 'both') → bool#

Checks whether the box is bounded in some sense.

Parameters:: manner (str) – One of "both", "below", "above".
Returns:: If the space is bounded
Raises:: ValueError – If manner is neither "both" nor "below" or "above"

sample(mask: None = None) → ndarray#

Generates a single random sample inside the Box.

In creating a sample of the box, each coordinate is sampled (independently) from a distribution that is chosen according to the form of the interval:

\([a, b]\) : uniform distribution
\([a, \infty)\) : shifted exponential distribution
\((-\infty, b]\) : shifted negative exponential distribution
\((-\infty, \infty)\) : normal distribution

Parameters:: mask – A mask for sampling values from the Box space, currently unsupported.
Returns:: A sampled value from the Box

Dict#

A dictionary of Space instances.

Elements of this space are (ordered) dictionaries of elements from the constituent spaces.

Example usage:

>>> from gym.spaces import Dict, Discrete
>>> observation_space = Dict({"position": Discrete(2), "velocity": Discrete(3)})
>>> observation_space.sample()
OrderedDict([('position', 1), ('velocity', 2)])

Example usage [nested]:

>>> from gym.spaces import Box, Dict, Discrete, MultiBinary, MultiDiscrete
>>> Dict(
...     {
...         "ext_controller": MultiDiscrete([5, 2, 2]),
...         "inner_state": Dict(
...             {
...                 "charge": Discrete(100),
...                 "system_checks": MultiBinary(10),
...                 "job_status": Dict(
...                     {
...                         "task": Discrete(5),
...                         "progress": Box(low=0, high=100, shape=()),
...                     }
...                 ),
...             }
...         ),
...     }
... )

It can be convenient to use Dict spaces if you want to make complex observations or actions more human-readable. Usually, it will not be possible to use elements of this space directly in learning code. However, you can easily convert Dict observations to flat arrays by using a gym.wrappers.FlattenObservation wrapper. Similar wrappers can be implemented to deal with Dict actions.

Constructor of Dict space.

This space can be instantiated in one of two ways: Either you pass a dictionary of spaces to __init__() via the spaces argument, or you pass the spaces as separate keyword arguments (where you will need to avoid the keys spaces and seed)

Example:

>>> from gym.spaces import Box, Discrete
>>> Dict({"position": Box(-1, 1, shape=(2,)), "color": Discrete(3)})
Dict(color:Discrete(3), position:Box(-1.0, 1.0, (2,), float32))
>>> Dict(position=Box(-1, 1, shape=(2,)), color=Discrete(3))
Dict(color:Discrete(3), position:Box(-1.0, 1.0, (2,), float32))

Parameters:

spaces – A dictionary of spaces. This specifies the structure of the Dict space
seed – Optionally, you can use this argument to seed the RNGs of the spaces that make up the Dict space.
**spaces_kwargs – If spaces is None, you need to pass the constituent spaces as keyword arguments, as described above.

sample(mask: Dict[str, Any] | None = None) → dict#

Generates a single random sample from this space.

The sample is an ordered dictionary of independent samples from the constituent spaces.

Parameters:: mask – An optional mask for each of the subspaces, expects the same keys as the space
Returns:: A dictionary with the same key and sampled values from :attr:`self.spaces`

Discrete#

class gym.spaces.Discrete(n: int, seed: int | Generator | None = None, start: int = 0)#

A space consisting of finitely many elements.

This class represents a finite subset of integers, more specifically a set of the form \(\{ a, a+1, \dots, a+n-1 \}\).

Example:

>>> Discrete(2)            # {0, 1}
>>> Discrete(3, start=-1)  # {-1, 0, 1}

class __init__(*args, **kwargs)#: Initialize self. See help(type(self)) for accurate signature.

sample(mask: ndarray | None = None) → int#

Generates a single random sample from this space.

A sample will be chosen uniformly at random with the mask if provided

Parameters:: mask – An optional mask for if an action can be selected. Expected np.ndarray of shape (n,) and dtype np.int8 where 1 represents valid actions and 0 invalid / infeasible actions. If there are no possible actions (i.e. np.all(mask == 0)) then space.start will be returned.
Returns:: A sampled integer from the space

Graph#

A space representing graph information as a series of nodes connected with edges according to an adjacency matrix represented as a series of edge_links.

Example usage:

self.observation_space = spaces.Graph(node_space=space.Box(low=-100, high=100, shape=(3,)), edge_space=spaces.Discrete(3))

Constructor of Graph.

The argument node_space specifies the base space that each node feature will use. This argument must be either a Box or Discrete instance.

The argument edge_space specifies the base space that each edge feature will use. This argument must be either a None, Box or Discrete instance.

Parameters:

node_space (Union[Box, Discrete]) – space of the node features.
edge_space (Union[None, Box, Discrete]) – space of the node features.
seed – Optionally, you can use this argument to seed the RNG that is used to sample from the space.

Generates a single sample graph with num_nodes between 1 and 10 sampled from the Graph.

Parameters:

mask – An optional tuple of optional node and edge mask that is only possible with Discrete spaces (Box spaces don’t support sample masks). If no num_edges is provided then the edge_mask is multiplied by the number of edges
num_nodes – The number of nodes that will be sampled, the default is 10 nodes
num_edges – An optional number of edges, otherwise, a random number between 0 and `num_nodes`^2

Returns:

A NamedTuple representing a graph with attributes .nodes, .edges, and .edge_links.

MultiBinary#

class gym.spaces.MultiBinary(n: ndarray | Sequence[int] | int, seed: int | Generator | None = None)#

An n-shape binary space.

Elements of this space are binary arrays of a shape that is fixed during construction.

Example Usage:

>>> observation_space = MultiBinary(5)
>>> observation_space.sample()
    array([0, 1, 0, 1, 0], dtype=int8)
>>> observation_space = MultiBinary([3, 2])
>>> observation_space.sample()
    array([[0, 0],
        [0, 1],
        [1, 1]], dtype=int8)

__init__(n: ndarray | Sequence[int] | int, seed: int | Generator | None = None)#

Constructor of MultiBinary space.

Parameters:

n – This will fix the shape of elements of the space. It can either be an integer (if the space is flat) or some sort of sequence (tuple, list or np.ndarray) if there are multiple axes.
seed – Optionally, you can use this argument to seed the RNG that is used to sample from the space.

sample(mask: ndarray | None = None) → ndarray#

Generates a single random sample from this space.

A sample is drawn by independent, fair coin tosses (one toss per binary variable of the space).

Parameters:: mask – An optional np.ndarray to mask samples with expected shape of space.shape. For mask == 0 then the samples will be 0 and mask == 1 then random samples will be generated. The expected mask shape is the space shape and mask dtype is np.int8.
Returns:: Sampled values from space

MultiDiscrete#

class gym.spaces.MultiDiscrete(nvec: ~numpy.ndarray | list, dtype=<class 'numpy.int64'>, seed: int | ~numpy.random._generator.Generator | None = None)#

This represents the cartesian product of arbitrary Discrete spaces.

It is useful to represent game controllers or keyboards where each key can be represented as a discrete action space.

Note

Some environment wrappers assume a value of 0 always represents the NOOP action.

e.g. Nintendo Game Controller - Can be conceptualized as 3 discrete action spaces:

Arrow Keys: Discrete 5 - NOOP[0], UP[1], RIGHT[2], DOWN[3], LEFT[4] - params: min: 0, max: 4
Button A: Discrete 2 - NOOP[0], Pressed[1] - params: min: 0, max: 1
Button B: Discrete 2 - NOOP[0], Pressed[1] - params: min: 0, max: 1

It can be initialized as MultiDiscrete([ 5, 2, 2 ]) such that a sample might be array([3, 1, 0]).

Although this feature is rarely used, MultiDiscrete spaces may also have several axes if nvec has several axes:

Example:

>> d = MultiDiscrete(np.array([[1, 2], [3, 4]]))
>> d.sample()
array([[0, 0],
       [2, 3]])

__init__(nvec: ~numpy.ndarray | list, dtype=<class 'numpy.int64'>, seed: int | ~numpy.random._generator.Generator | None = None)#

Constructor of MultiDiscrete space.

The argument nvec will determine the number of values each categorical variable can take.

Parameters:

nvec – vector of counts of each categorical variable. This will usually be a list of integers. However, you may also pass a more complicated numpy array if you’d like the space to have several axes.
dtype – This should be some kind of integer type.
seed – Optionally, you can use this argument to seed the RNG that is used to sample from the space.

sample(mask: tuple | None = None) → ndarray#

Generates a single random sample this space.

Parameters:: mask – An optional mask for multi-discrete, expects tuples with a np.ndarray mask in the position of each action with shape (n,) where n is the number of actions and dtype=np.int8. Only mask values == 1 are possible to sample unless all mask values for an action are 0 then the default action 0 is sampled.
Returns:: An `np.ndarray` of shape `space.shape`

Sequence#

class gym.spaces.Sequence(space: Space, seed: int | Generator | None = None)#

This space represent sets of finite-length sequences.

This space represents the set of tuples of the form \((a_0, \dots, a_n)\) where the \(a_i\) belong to some space that is specified during initialization and the integer \(n\) is not fixed

Example::

>>> space = Sequence(Box(0, 1))
>>> space.sample()
(array([0.0259352], dtype=float32),)
>>> space.sample()
(array([0.80977976], dtype=float32), array([0.80066574], dtype=float32), array([0.77165383], dtype=float32))

__init__(space: Space, seed: int | Generator | None = None)#

Constructor of the Sequence space.

Parameters:

space – Elements in the sequences this space represent must belong to this space.
seed – Optionally, you can use this argument to seed the RNG that is used to sample from the space.

sample(mask: Tuple[ndarray | int | None, Any | None] | None = None) → Tuple[Any]#

Generates a single random sample from this space.

Parameters:: mask – An optional mask for (optionally) the length of the sequence and (optionally) the values in the sequence. If you specify mask, it is expected to be a tuple of the form (length_mask, sample_mask) where length_mask is - None The length will be randomly drawn from a geometric distribution - np.ndarray of integers, in which case the length of the sampled sequence is randomly drawn from this array. - int for a fixed length sample The second element of the mask tuple sample mask specifies a mask that is applied when sampling elements from the base space. The mask is applied for each feature space sample.
Returns:: A tuple of random length with random samples of elements from the :attr:`feature_space`.

Text#

class gym.spaces.Text(max_length: int, *, min_length: int = 1, charset: Set[str] | str = frozenset({'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z'}), seed: int | Generator | None = None)#

A space representing a string comprised of characters from a given charset.

Example::

>>> # {"", "B5", "hello", ...}
>>> Text(5)
>>> # {"0", "42", "0123456789", ...}
>>> import string
>>> Text(min_length = 1,
...      max_length = 10,
...      charset = string.digits)

__init__(max_length: int, *, min_length: int = 1, charset: Set[str] | str = frozenset({'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z'}), seed: int | Generator | None = None)#

Constructor of Text space.

Both bounds for text length are inclusive.

Parameters:

min_length (int) – Minimum text length (in characters). Defaults to 1 to prevent empty strings.
max_length (int) – Maximum text length (in characters).
charset (Union[set], str) – Character set, defaults to the lower and upper english alphabet plus latin digits.
seed – The seed for sampling from the space.

sample(mask: Tuple[int | None, ndarray | None] | None = None) → str#

Generates a single random sample from this space with by default a random length between min_length and max_length and sampled from the charset.

Parameters:: mask – An optional tuples of length and mask for the text. The length is expected to be between the min_length and max_length otherwise a random integer between min_length and max_length is selected. For the mask, we expect a numpy array of length of the charset passed with dtype == np.int8. If the charlist mask is all zero then an empty string is returned no matter the min_length
Returns:: A sampled string from the space

Tuple#

class gym.spaces.Tuple(spaces: Iterable[Space], seed: int | Sequence[int] | Generator | None = None)#

A tuple (more precisely: the cartesian product) of Space instances.

Elements of this space are tuples of elements of the constituent spaces.

Example usage:

>>> from gym.spaces import Box, Discrete
>>> observation_space = Tuple((Discrete(2), Box(-1, 1, shape=(2,))))
>>> observation_space.sample()
(0, array([0.03633198, 0.42370757], dtype=float32))

__init__(spaces: Iterable[Space], seed: int | Sequence[int] | Generator | None = None)#

Constructor of Tuple space.

The generated instance will represent the cartesian product \(\text{spaces}[0] \times ... \times \text{spaces}[-1]\).

Parameters:

spaces (Iterable[Space]) – The spaces that are involved in the cartesian product.
seed – Optionally, you can use this argument to seed the RNGs of the spaces to ensure reproducible sampling.

sample(mask: Tuple[ndarray | None, ...] | None = None) → tuple#

Generates a single random sample inside this space.

This method draws independent samples from the subspaces.

Parameters:: mask – An optional tuple of optional masks for each of the subspace’s samples, expects the same number of masks as spaces
Returns:: Tuple of the subspace’s samples

Utility Functions#

gym.spaces.utils.flatdim(space: Space) → int#

gym.spaces.utils.flatdim(space: Box | MultiBinary) → int

gym.spaces.utils.flatdim(space: Discrete) → int

gym.spaces.utils.flatdim(space: MultiDiscrete) → int

gym.spaces.utils.flatdim(space: Tuple) → int

gym.spaces.utils.flatdim(space: Dict) → int

gym.spaces.utils.flatdim(space: Graph)

gym.spaces.utils.flatdim(space: Text) → int

Return the number of dimensions a flattened equivalent of this space would have.

Example usage:

>>> from gym.spaces import Discrete
>>> space = Dict({"position": Discrete(2), "velocity": Discrete(3)})
>>> flatdim(space)
5

Parameters:

space – The space to return the number of dimensions of the flattened spaces

Returns:

The number of dimensions for the flattened spaces

Raises:

NotImplementedError – if the space is not defined in gym.spaces.
ValueError – if the space cannot be flattened into a Box

gym.spaces.utils.flatten_space(space: Space) → Dict | Sequence | Tuple | Graph#

gym.spaces.utils.flatten_space(space: Box) → Box

gym.spaces.utils.flatten_space(space: Discrete | MultiBinary | MultiDiscrete) → Box

gym.spaces.utils.flatten_space(space: Tuple) → Box | Tuple

gym.spaces.utils.flatten_space(space: Dict) → Box | Dict

gym.spaces.utils.flatten_space(space: Graph) → Graph

gym.spaces.utils.flatten_space(space: Text) → Box

gym.spaces.utils.flatten_space(space: Sequence) → Sequence

Flatten a space into a space that is as flat as possible.

This function will attempt to flatten space into a single Box space. However, this might not be possible when space is an instance of Graph, Sequence or a compound space that contains a Graph or Sequence`space. This is equivalent to :func:`flatten, but operates on the space itself. The result for non-graph spaces is always a Box with flat boundaries. While the result for graph spaces is always a Graph with node_space being a Box with flat boundaries and edge_space being a Box with flat boundaries or None. The box has exactly flatdim() dimensions. Flattening a sample of the original space has the same effect as taking a sample of the flattenend space.

Example:

>>> box = Box(0.0, 1.0, shape=(3, 4, 5))
>>> box
Box(3, 4, 5)
>>> flatten_space(box)
Box(60,)
>>> flatten(box, box.sample()) in flatten_space(box)
True

Example that flattens a discrete space:

>>> discrete = Discrete(5)
>>> flatten_space(discrete)
Box(5,)
>>> flatten(box, box.sample()) in flatten_space(box)
True

Example that recursively flattens a dict:

>>> space = Dict({"position": Discrete(2), "velocity": Box(0, 1, shape=(2, 2))})
>>> flatten_space(space)
Box(6,)
>>> flatten(space, space.sample()) in flatten_space(space)
True

Example that flattens a graph:

>>> space = Graph(node_space=Box(low=-100, high=100, shape=(3, 4)), edge_space=Discrete(5))
>>> flatten_space(space)
Graph(Box(-100.0, 100.0, (12,), float32), Box(0, 1, (5,), int64))
>>> flatten(space, space.sample()) in flatten_space(space)
True

Parameters:: space – The space to flatten
Returns:: A flattened Box
Raises:: NotImplementedError – if the space is not defined in gym.spaces.

gym.spaces.utils.flatten(space: Space[T], x: T) → ndarray | Dict | tuple | GraphInstance#

gym.spaces.utils.flatten(space: MultiBinary, x) → ndarray

gym.spaces.utils.flatten(space: Box, x) → ndarray

gym.spaces.utils.flatten(space: Discrete, x) → ndarray

gym.spaces.utils.flatten(space: MultiDiscrete, x) → ndarray

gym.spaces.utils.flatten(space: Tuple, x) → tuple | ndarray

gym.spaces.utils.flatten(space: Dict, x) → dict | ndarray

gym.spaces.utils.flatten(space: Graph, x) → GraphInstance

gym.spaces.utils.flatten(space: Text, x: str) → ndarray

gym.spaces.utils.flatten(space: Sequence, x) → tuple

Flatten a data point from a space.

This is useful when e.g. points from spaces must be passed to a neural network, which only understands flat arrays of floats.

Parameters:

space – The space that x is flattened by
x – The value to flatten

Returns:

- For ``Box`` and ``MultiBinary``, this is a flattened array
- For ``Discrete`` and ``MultiDiscrete``, this is a flattened one-hot array of the sample
- For ``Tuple`` and ``Dict``, this is a concatenated array the subspaces (does not support graph subspaces)
- For graph spaces, returns `GraphInstance` where –
- nodes are n x k arrays
- edges are either:
  - m x k arrays
  - None
- edge_links are either:
  - m x 2 arrays
  - None

Raises:

NotImplementedError – If the space is not defined in gym.spaces.

gym.spaces.utils.unflatten(space: Space[T], x: ndarray | Dict | tuple | GraphInstance) → T#

gym.spaces.utils.unflatten(space: Box | MultiBinary, x: ndarray) → ndarray

gym.spaces.utils.unflatten(space: Discrete, x: ndarray) → int

gym.spaces.utils.unflatten(space: MultiDiscrete, x: ndarray) → ndarray

gym.spaces.utils.unflatten(space: Tuple, x: ndarray | tuple) → tuple

gym.spaces.utils.unflatten(space: Dict, x: ndarray | Dict) → dict

gym.spaces.utils.unflatten(space: Graph, x: GraphInstance) → GraphInstance

gym.spaces.utils.unflatten(space: Text, x: ndarray) → str

gym.spaces.utils.unflatten(space: Sequence, x: tuple) → tuple

Unflatten a data point from a space.

This reverses the transformation applied by flatten(). You must ensure that the space argument is the same as for the flatten() call.

Parameters:

space – The space used to unflatten x
x – The array to unflatten

Returns:

A point with a structure that matches the space.

Raises:

NotImplementedError – if the space is not defined in gym.spaces.

Spaces#

General Functions#

Box#

Dict#

Discrete#

Graph#

MultiBinary#

MultiDiscrete#

Sequence#

Text#

Tuple#

Utility Functions#

Gym Documentation