Utilities¶
Environment checker¶
check_env(env: Env[Any, Any], *, n_steps: int = 100, warn_on_anomaly: bool = True, check_obs_space: bool = True, check_action_space: bool = True, check_reset_seed: bool = True, check_determinism: bool = False, check_async: bool = True) -> list[str]
Validate that env correctly implements the sekai Env API.
Returns a list of issue strings, each prefixed "ERROR:" or "WARNING:". An empty list means the env passed every check.
This is a testing utility — call it in your test suite, not in production. Unlike gymnasium's PassiveEnvChecker, it adds zero overhead to normal training.
Exceptions from reset(), step() and action_mask() are reported as ERROR lines, not raised. Every reset() is checked, including those that start a new episode during the rollout (which must not overwrite the previous episode's terminal observation in place).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env
|
Env[Any, Any]
|
The environment to check. It is reset and stepped, so pass a fresh one. |
required |
n_steps
|
int
|
Number of random-action steps to take. |
100
|
warn_on_anomaly
|
bool
|
Report WARNING-level issues (ERRORs are always reported). |
True
|
check_obs_space
|
bool
|
Check every observation is inside observation_space. |
True
|
check_action_space
|
bool
|
Check action_space.sample() is inside action_space. |
True
|
check_reset_seed
|
bool
|
Check reset(seed=0) twice gives the same observation. |
True
|
check_determinism
|
bool
|
Check two rollouts with the same seed and actions match. |
False
|
check_async
|
bool
|
Check async_reset()/async_step() return valid results. Skipped when called from inside a running event loop. |
True
|
Running statistics¶
Used by the normalisation wrappers; see Non-stationary statistics.
Running mean and variance over batches, using the parallel-variance update.
Shared by the observation/reward normalisation wrappers (single-env and
vectorised). Starts at mean 0, var 1 with a tiny pseudo-count (epsilon) so the
first update dominates: the first batch sets the mean outright, and the prior's
unit variance keeps only its pseudo-count's weight (so a single first sample does
not give a zero variance). Pooling the first batch with the prior mean of 0
instead would add about mean**2 * epsilon / count to the variance for good, a
large error for data far from zero, such as prices. Empty batches are ignored.
Exponentially weighted mean and variance, a drop-in for RunningMeanStd.
RunningMeanStd weights every sample ever seen equally, so on non-stationary
data (volatility regimes, trending prices) its statistics drift towards a
long-run average that no longer describes the current data. This tracks
the recent distribution instead: a sample's weight halves every halflife
samples. Batches count as len(batch) samples arriving together.
Weights are bias-corrected (like pandas' ewm(adjust=True)): early on, before
a halflife's worth of data has arrived, samples are weighted almost equally,
so the first batch does not dominate the estimates.
Observation normalisation¶
The engine shared by NormalizeObservation and VecNormalizeObservation.
ObservationNormalizer(space: Space[Any], *, stats: StatsKind = 'running', halflife: float = 1000.0, epsilon: float = 1e-08, clip: float | None = None)
Normalises batches of observations of space (leading batch axis on every leaf).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
space
|
Space[Any]
|
The (unbatched) observation space. |
required |
stats
|
StatsKind
|
"running" for equal-weight statistics over all samples (RunningMeanStd), "ewm" for exponentially weighted ones that follow non-stationary data (ExponentialMovingStats). |
'running'
|
halflife
|
float
|
Halflife for stats="ewm", in calls: each normalised batch (one env step, or one step of a vector env) counts once. |
1000.0
|
epsilon
|
float
|
Added to the variance before dividing. |
1e-08
|
clip
|
float | None
|
If given, normalised values are clipped to [-clip, clip]. |
None
|
Action masks¶
Exported from sekai.utils (and get_action_mask from sekai). See
Action masks.
Action masking: which actions are valid in the current state.
Convention: an environment that restricts its actions in some states (a halted
stock cannot be traded, a short is not available, an inventory limit is hit)
implements a method action_mask() returning a mask in the format that its
action_space.sample(mask=...) accepts:
| Action space | Mask |
|---|---|
Discrete(n) |
int8 array (n,), 1 = allowed |
MultiDiscrete(nvec) |
tuple of int8 arrays (one per entry), or one array padded to max(nvec) |
Box |
array broadcastable to the shape; 0 = this component must stay at zero |
A MultiDiscrete mask entry with the wrong number of flags raises ValueError, in
sample(mask=...) and in flatten_action_mask() alike (and check_env reports it).
"Zero" in a Box mask is action_space.zero() (zero clipped into the bounds) of the
space the mask is applied in. RescaleAction changes the space, so it sets the
masked components of every action to the env's zero itself: a policy sampling in
[-1, 1] with the env's mask sends 0 (not the midpoint of the env's range) for an
asset that cannot be traded.
Vector envs expose it with vec.call("action_mask"), sekai.ActionMaskObservation
adds it to the observation, and sekai.compat.to_gymnasium() forwards it as
action_masks() for learners such as sb3-contrib's MaskablePPO.
The env's current action mask, or None if it does not provide one.
Whether env (or any layer of its wrapper stack) defines action_mask().
Convert a mask to the flat boolean layout used by masked-policy learners.
Discrete: (n,). MultiDiscrete: the per-entry masks concatenated, length
sum(nvec). Box: the mask broadcast to the shape and flattened. A mask of the
wrong shape raises ValueError.
action_mask_space(space: Space[Any]) -> MultiBinary
The MultiBinary space of flattened masks for an action space (see flatten_action_mask).
Seeding¶
Exported from sekai and sekai.utils.
Create a numpy Generator and return (rng, seed_used).
If seed is already a Generator, returns it with -1 as the seed (unknown). If seed is None, generates a random 128-bit seed from OS entropy.