The finance environments keep the account in weight space: positions are fractions of equity, the rest is cash, and equity starts at 1. One decision runs through three modules:
sekai.finance.account: the action becomes target weights that satisfy the account’s
rules (long-only or long/short, a leverage cap, nothing in untradable assets);sekai.finance.costs: moving from the current (drifted) weights to the targets costs a
fraction of equity;sekai.finance.account again: over the period the positions earn the assets’ returns
and cash earns (or pays) interest; the equity grows by a factor $g_t$ and the weights
drift;sekai.finance.rewards: a reward function turns the step’s outcome into the reward.Everything here broadcasts over leading batch dimensions, so the same objects serve a single environment and every sub-env of a native vector env at once.
A CostModel is called with trades, the signed change in each asset’s weight (shape
(..., n)), and returns the cost as a fraction of equity (shape (...)):
| Model | Cost as a fraction of equity | requires |
|---|---|---|
NoCost() |
$0$ | |
ProportionalCost(rate) |
$\sum_i \kappa_i\,\lvert\Delta w_i\rvert$, one rate or one per asset | |
SpreadCost(half_spread) |
$\sum_i h_i\,\lvert\Delta w_i\rvert$: crossing half the bid-ask spread | |
SpreadCost(fallback_spread=None) |
the same with $h_i$ = half the data’s quoted spread (the cross-sectional median where an asset’s is missing or negative; a zero spread costs nothing) |
spread |
FixedCost(amount, threshold=1e-6) |
amount per asset traded (weight change above threshold), divided by equity in currency |
equity |
SquareRootImpact(coefficient=0.5, fallback_volatility=None) |
square-root market impact (below) | equity, volatility, volume |
Missing inputs are never free. Only a non-finite value (NaN) counts as missing; a
finite zero is a valid input, so a zero quoted spread costs nothing and a zero volatility
(a flat window) gives zero impact. A negative value is invalid and treated like a missing
one. A quoted spread or a volatility that is missing for one asset is replaced by the
cross-sectional median of the others. When no asset on a row has one (no spreads quoted
that day, no return history yet), there is nothing to take a median of, and trading on
that row raises a ValueError rather than costing nothing. To price such trades anyway,
give the model an explicit value: fallback_spread (a full spread, as a fraction of the
price) or fallback_volatility (per step). Untraded rows never need one.
A plain number passed as transaction_cost= to an environment means
ProportionalCost(number); the default 1e-3 is 10 bp of every unit traded.
Square-root impact. Empirically, trading $q$ in currency against a traded value $V$ per period moves the price by about $c\,\sigma\sqrt{q/V}$, where $\sigma$ is the per-period volatility and $c$ a coefficient of order 0.1–1 (Almgren et al. 2005; Tóth et al. 2011). The trade pays that move on its whole size, so with equity $E$ in currency and $q_i = \lvert\Delta w_i\rvert E$,
\[\text{cost} = \sum_i c\,\sigma_i \sqrt{\frac{\lvert\Delta w_i\rvert\,E}{V_i}}\;\lvert\Delta w_i\rvert ,\]which grows as $\lvert\Delta w\rvert^{3/2}$: large rebalances in illiquid names get
expensive, which a proportional cost cannot express. The impact per unit traded,
$c\,\sigma_i\sqrt{q_i/V_i}$, is capped at 1 (a trade never costs more than its value); an
asset with zero or missing volume has no liquidity, so trading it costs the full traded
value; a missing volatility is replaced by the median volatility of the other assets (or
fallback_volatility when no asset has one); and a zero volatility gives zero impact.
Any trading cost of 100% of equity or more wipes the account out: the environments terminate the episode rather than let equity change sign.
import numpy as np
import sekai
from sekai.finance import FixedCost, ProportionalCost, SpreadCost, SquareRootImpact
trades = np.array([0.10, -0.05, 0.0]) # buy 10% of equity in asset 0, sell 5% of asset 1
context = dict(equity=1e6, # account value in currency
volatility=np.array([0.02, 0.01, 0.015]), # per-step
volume=np.array([5e7, 2e7, 1e7])) # traded value per step, in currency
for model in (ProportionalCost(1e-3), SpreadCost(half_spread=2e-4), FixedCost(5.0),
SquareRootImpact(0.5)):
print(f"{model!r:31s} {float(model(trades, **context)) * 1e4:.3f} bp")
ProportionalCost(0.001) 1.500 bp
SpreadCost(half_spread=0.0002) 0.300 bp
FixedCost(5.0) 0.100 bp
SquareRootImpact(0.5) 0.572 bp
Models compose with + into a CompositeCost whose requires is the union of its
parts’, so a realistic schedule reads like its definition:
costs = SpreadCost(half_spread=2e-4) + SquareRootImpact(0.5) + FixedCost(5.0)
print(costs) # SpreadCost(half_spread=0.0002) + SquareRootImpact(0.5) + FixedCost(5.0)
print(sorted(costs.requires)) # ['equity', 'volatility', 'volume']
print(f"{float(costs(trades, **context)) * 1e4:.3f} bp") # 0.972 bp
Batches work the same way, one row per sub-env:
batch = np.array([[0.1, -0.05, 0.0], [0.5, 0.5, 0.0]])
cost = costs(batch, equity=np.array([1e6, 1e6]), volatility=np.full((2, 3), 0.02),
volume=np.full((2, 3), 5e7))
print((cost * 1e4).round(3)) # [ 1.005 12.1 ]
Environments compute only the inputs named in the cost model’s requires:
| Input | What Trading-v0, Portfolio-v0 and Market-v0 pass |
|---|---|
equity |
account value in currency: capital= (default 1,000,000) times the current relative equity |
volatility |
per-step standard deviation of each asset’s log returns over the observation’s lookback window |
volume |
the data field "volume" (units traded per step) times the current price, i.e. traded value |
spread |
the data field "spread": the quoted bid-ask spread as a fraction of the price |
volume and spread therefore need historical MarketData with those fields; asking for a
model whose inputs the data cannot supply raises a ValueError naming the missing field
when the environment is built. So does a volatility-based model (SquareRootImpact) with
lookback=1: one return gives no volatility estimate, so every trade would be unpriced;
use lookback >= 2 (a fallback does not help, since a one-return window can look flat).
Here a four-asset history with volumes and spreads, and the cost of the same first
rebalance (cash to 20% in each asset) for accounts of different sizes:
from sekai.envs.finance import simulate_gbm
from sekai.finance import MarketData
prices = simulate_gbm(np.random.default_rng(0), 1000, s0=[50, 120, 30, 80],
mu=0.05, sigma=[0.25, 0.18, 0.4, 0.08])
data = MarketData(prices, fields={"volume": np.full(prices.shape, 2e6),
"spread": np.full(prices.shape, 5e-4)})
for capital in (1e6, 1e8, 1e9):
env = sekai.make("Portfolio-v0", prices=data, lookback=60, episode_length=252,
transaction_cost=SpreadCost() + SquareRootImpact(0.5), capital=capital)
env.reset(seed=0)
info = env.step(np.ones(5, np.float32)).info
print(f"capital {capital:>13,.0f}: turnover {info['turnover']:.1f}, cost {info['cost'] * 1e4:6.1f} bp")
capital 1,000,000: turnover 0.8, cost 5.3 bp
capital 100,000,000: turnover 0.8, cost 34.9 bp
capital 1,000,000,000: turnover 0.8, cost 106.1 bp
The spread costs the same 2 bp at every size; impact grows with the square root of size.
try:
sekai.make("Portfolio-v0", transaction_cost=SpreadCost()) # simulated: no spread data
except ValueError as error:
print(error)
SpreadCost(half_spread=from data) needs a 'spread' data field (fraction of the price), which a simulated market does not provide.
Subclass CostModel, implement __call__(trades, *, equity=None, volatility=None,
volume=None, spread=None) returning an array of the batch shape, and set requires to the
inputs you read. A broker commission of 1 bp with a minimum ticket of 10 currency units:
from sekai.finance import CostModel
class MinimumCommission(CostModel):
requires = frozenset({"equity"})
def __init__(self, rate=1e-4, minimum=10.0):
self.rate, self.minimum = rate, minimum
def __call__(self, trades, *, equity=None, **context):
notional = np.abs(trades) * np.asarray(equity)[..., None]
fees = np.where(notional > 0, np.maximum(self.rate * notional, self.minimum), 0.0)
return fees.sum(axis=-1) / equity
print(f"{float(MinimumCommission()(trades, equity=1e6)) * 1e4:.2f} bp") # 0.20 bp
print(f"{float(MinimumCommission()(trades, equity=1e5)) * 1e4:.2f} bp") # 2.00 bp
AccountConfig holds the constraints and financing terms:
| Field | Default | Meaning |
|---|---|---|
allow_short |
False |
allow negative weights |
max_leverage |
1.0 |
cap on gross exposure $\sum_i \lvert w_i\rvert$ |
cash_rate |
0.0 |
annual interest earned on positive cash |
margin_rate |
None |
annual interest paid on borrowed (negative) cash; None means cash_rate |
borrow_rate |
0.0 |
annual fee on the value of short positions |
Market-v0 takes these as constructor arguments of the same names (its long/short action
space also caps each weight at ±1, whatever max_leverage); Portfolio-v0 has
allow_short with a leverage of 1 and no financing.
Projection. project_weights(scores, tradable, config, *, cash_score=None) maps raw
action values to admissible target weights. Long-only, scores (and an optional cash score)
are clipped at zero and normalised to sum to max_leverage; all zeros means all cash.
Long/short, values are clipped to $\pm$max_leverage and scaled down if gross exposure
exceeds it. Untradable assets get weight 0 either way:
from sekai.finance import AccountConfig
from sekai.finance.account import project_weights, settle
long_only = AccountConfig()
everything = np.array([True, True, True])
print(project_weights([2.0, 1.0, 1.0], everything, long_only)) # [0.5 0.25 0.25]
print(project_weights([2.0, 1.0, 1.0], everything, long_only, cash_score=4.0)) # [0.25 0.125 0.125]
print(project_weights([2.0, 1.0, 1.0], [True, False, True], long_only)) # [0.66666667 0. 0.33333333]
levered = AccountConfig(allow_short=True, max_leverage=2.0, cash_rate=0.04,
margin_rate=0.06, borrow_rate=0.01)
w = project_weights([1.5, -1.0, 0.5], everything, levered)
print(w.round(4), np.abs(w).sum().round(4)) # [ 1. -0.6667 0.3333] 2.0
Settlement. settle(weights, returns, *, cost, dt, config) holds the weights over one
period of length dt years, after paying cost. With cash weight
$c = 1 - \sum_i w_i$ and asset returns $r_i$:
where $b$ is borrow_rate, $g$ the equity growth factor (growth), $f$ the financing
(financing) and $\tilde w$ the drifted weights the next rebalance starts from
(drifted). If $g \le 0$ the account is wiped out (wiped_out) and the drifted weights are
zero. Financing accrues once per step of dt years: on daily rows, weekends and holidays
accrue nothing extra. The cash weight includes the proceeds of short sales, which therefore
earn the full cash_rate.
s = settle(w, [0.01, -0.02, 0.0], cost=5e-4, dt=1 / 252, config=levered)
print(f"{float(s.growth):.6f} {float(s.financing):.2e}", s.drifted.round(4), bool(s.wiped_out))
1.022848 2.65e-05 [ 0.9869 -0.6384 0.3257] False
The net exposure is $1 - 2/3 + 1/3 = 2/3$, so a third of equity is cash earning 4%,
less the 1% fee on the 2/3 short. turnover(target, current) gives
$\sum_i \lvert w_i - \tilde w_i\rvert$, the quantity proportional costs charge.
Delisting. In Market-v0, an asset whose price disappears during a period earns
delisting_return (default 0) over that period and its position becomes cash. Here BBB
delists after the second decision and recovers half its value:
prices = np.array([[100.0, 50.0], [100.0, 50.0], [101.0, 50.0], [102.0, np.nan]])
env = sekai.make("Market-v0", data=MarketData(prices, assets=["AAA", "BBB"]), lookback=1,
episode_length=2, transaction_cost=0.0, delisting_return=-0.5)
result = env.reset(seed=0)
for _ in range(2):
result = env.step(np.array([1.0, 1.0, 0.0], np.float32)) # half in each, no cash
print(result.observation["portfolio"].round(4), round(result.info["equity"], 4))
[0.5025 0.4975 0. ] 1.005
[0.6689 0. 0.3311] 0.7587
The portfolio observation lists the weights per slot with cash last; BBB’s slot is
empty after the delisting, and it is masked out of the next action.
A RewardFunction maps the step’s StepOutcome to a reward. The outcome holds
log_return ($\log g_t$, net of all costs), simple_return ($g_t - 1$), equity (after the
step, relative to the start of the episode), turnover, cost and pnl (the change in
equity, $E_t - E_{t-1}$); build one with
StepOutcome.from_growth(growth, equity=..., turnover=..., cost=..., previous_equity=...).
The built-in environments always pass previous_equity. Without it pnl is None, and
ProfitAndLoss recovers the change from the equity after the step, which is impossible
on a wipe-out (equity 0), so it raises there instead of rewarding the loss with 0.
| Reward | Per step | Summed over an episode |
|---|---|---|
LogReturn() (default) |
$\log g_t$ | $\log(E_T / E_0)$, the Kelly objective |
SimpleReturn() |
$g_t - 1$ | |
ProfitAndLoss() |
$E_t - E_{t-1}$, exact (a wipe-out step loses all of $E_{t-1}$) | $E_T - 1$, P&L in units of initial equity |
MeanVariance(risk_aversion=1.0) |
$r_t - \tfrac{\lambda}{2} r_t^2$ on the log return | |
TurnoverPenalty() |
$\tau_t$ (subtract it) | total turnover |
DrawdownPenalty() |
how much the drawdown deepened (subtract it) | total deepening of drawdowns |
DifferentialSharpe(eta=0.01) |
the differential Sharpe ratio | approximately maximises the Sharpe ratio |
Drawdown penalty. With running peak $P_t = \max(1, \max_{s \le t} E_s)$ and drawdown $D_t = 1 - E_t / P_t$, the penalty is $\max(D_t - D_{t-1}, 0)$: zero while equity is at a peak or recovering, positive only when the drawdown gets deeper.
Differential Sharpe ratio (Moody & Saffell 2001). Exponential moving estimates of the first and second moments of the step returns $R_t$ (simple returns),
\[A_t = A_{t-1} + \eta\,(R_t - A_{t-1}), \qquad B_t = B_{t-1} + \eta\,(R_t^2 - B_{t-1}),\]define a Sharpe ratio $S_t = A_t / \sqrt{B_t - A_t^2}$; the reward is its sensitivity to the latest return,
\[D_t = \frac{B_{t-1}\,\Delta A_t - \tfrac12 A_{t-1}\,\Delta B_t}{\big(B_{t-1} - A_{t-1}^2\big)^{3/2}}, \qquad \Delta A_t = R_t - A_{t-1},\quad \Delta B_t = R_t^2 - B_{t-1},\]an online, per-step signal whose sum approximately maximises the Sharpe ratio. It is 0 until the variance estimate is positive; $\eta$ is about one over the averaging window.
Rewards compose with +, -, * and constants, and reset() reaches every stateful
part:
from sekai.finance import DrawdownPenalty, LogReturn, StepOutcome, TurnoverPenalty
reward = LogReturn() - 0.5 * DrawdownPenalty() - 1e-3 * TurnoverPenalty()
print(reward) # LogReturn() - 0.5 * DrawdownPenalty() - 0.001 * TurnoverPenalty()
drawdown, equity = DrawdownPenalty(), 1.0
for growth in (1.02, 0.97, 0.99, 1.05, 0.98):
equity *= growth
outcome = StepOutcome.from_growth(growth, equity=equity, turnover=0.1, cost=1e-4)
print(f"equity {equity:.4f} drawdown penalty {float(drawdown(outcome)):.4f} "
f"reward {float(reward(outcome)):+.5f}")
equity 1.0200 drawdown penalty 0.0000 reward +0.01970
equity 0.9894 drawdown penalty 0.0300 reward -0.04556
equity 0.9795 drawdown penalty 0.0097 reward -0.01500
equity 1.0285 drawdown penalty 0.0000 reward +0.04869
equity 1.0079 drawdown penalty 0.0200 reward -0.03030
Pass a reward to any of Trading-v0, Portfolio-v0 or Market-v0 (and their native vector
versions) with reward=. Each environment keeps its own copy (a deep copy, so one
stateful reward passed to several environments, or to make_vec, never shares state
between them) and resets it at every episode start:
from sekai.finance import DifferentialSharpe
env = sekai.make("Portfolio-v0", reward=DifferentialSharpe(eta=0.01))
result, total = env.reset(seed=0), 0.0
while not result.done:
result = env.step(np.ones(6, np.float32))
total += result.reward
print(f"{total:.2f}, final equity {result.info['equity']:.4f}") # 37.52, final equity 1.0203
The scale of a differential-Sharpe episode return is not that of a log return; compare policies on the metrics of the next page, not on rewards of different kinds.
Batches and partial resets. Outcome fields may be arrays with one entry per sub-env.
Stateful rewards (DrawdownPenalty, DifferentialSharpe) then keep per-env state, and
reset(mask) restarts only the sub-envs where mask is True, which is what a native
vector env does when some of its sub-envs finish:
sharpe = DifferentialSharpe(eta=0.1)
rng = np.random.default_rng(0)
for _ in range(5):
growth = 1.0 + rng.normal(0.001, 0.01, size=4)
rewards = sharpe(StepOutcome.from_growth(growth, equity=np.ones(4), turnover=np.zeros(4),
cost=np.zeros(4)))
print(rewards.round(3)) # [-0.427 -0.464 0.849 2.467]
sharpe.reset(np.array([True, False, False, False])) # sub-env 0 starts a new episode
Writing a reward. Subclass RewardFunction and implement __call__(outcome) (with
NumPy operations, so it batches); override reset(mask=None) if it keeps state. A
downside-only penalty:
from sekai.finance import RewardFunction
class Downside(RewardFunction):
"""The step's log return when negative, else 0."""
def __call__(self, outcome):
return np.minimum(outcome.log_return, 0.0)
reward = LogReturn() + 0.5 * Downside() # losses count 1.5 times
outcome = StepOutcome.from_growth(0.98, equity=0.98, turnover=0.0, cost=0.0)
print(round(float(reward(outcome)), 5)) # -0.0303
transaction_cost=, reward=
and the account options in each environment.sekai.finance API.