This is the full developer documentation for Probatio
# Probatio
> Put your data to the proof.
**Probatio** is a modern, maintained data validation library for Python, and a clean-room reimplementation of [voluptuous](https://github.com/alecthomas/voluptuous): the same API, written fresh. Change the import, keep your schemas, and get a library that is actively maintained, MIT licensed, and validated against voluptuous’s own behavior. Then, when you want more, it is already there.
The model stays the same as voluptuous: a schema is data, and data describes data. You compose schemas from plain Python types, dicts, lists, and small helpers, then call the schema with a value to validate it.
Created and written by **[Franck Nijhof](https://www.linkedin.com/in/frenck)**, also known as [Frenck](https://frenck.dev).
Pre-1.0 project
Probatio is early software. The goal is to be a drop-in replacement for voluptuous, validated against the voluptuous behavior. Some internals may still change before 1.0 while the project gathers production feedback.
Drop-in compatible
The public API mirrors voluptuous: `Schema`, `Required`, `Optional`, `All`, `Any`, `Coerce`, `Invalid`, and the rest. Change the import, keep your schemas. voluptuous’s own 0.16.0 test suite runs against it, and every divergence is a deliberate, documented deviation.
More than a replacement
It also clears voluptuous’s backlog: cross-field rules, dataclass and TypedDict schemas, network and format validators, and errors that carry a path and suggest the key you meant. Several are voluptuous’s own open requests, now implemented.
Proven, not promised
100% line and branch coverage, two type checkers, and continuous fuzzing on every untrusted-input surface. It is also safer than the original: leaks and `RecursionError` edges voluptuous has are fixed, and documented.
Typed and modern
Ships `py.typed`, checked under mypy and ty, pure Python with no native build. Exports JSON Schema and OpenAPI from a schema, and reads them back in.
It is also fast for pure Python: roughly two to three times voluptuous’s speed on a representative config schema, and about seven times once a schema [compiles itself](/guides/compiled-schemas/). The numbers, and how to reproduce them, are on the [performance page](/reference/performance/).
## At a glance
[Section titled “At a glance”](#at-a-glance)
Define a schema, then call it with a value to validate it:
```python
from probatio import Schema, Required, Optional
schema = Schema(
{
Required("name"): str,
Optional("age"): int,
}
)
schema({"name": "Frenck", "age": 40})
# {'name': 'Frenck', 'age': 40}
```
When a value does not match, the schema raises `Invalid` (or `MultipleInvalid`) with a path to the offending value, however deep it sits:
```python
from probatio import Schema, Invalid
schema = Schema({"server": {"port": int}})
try:
schema({"server": {"port": "nope"}})
except Invalid as err:
print(err)
# expected int at 'server.port'
```
## Why trust it
[Section titled “Why trust it”](#why-trust-it)
It validates untrusted input, config, an API body, a device response, so the bar is whether you would hand it exactly that. The evidence, not the adjectives:
* **voluptuous 0.16.0 test suite:** runs against Probatio, with every divergence a deliberate, documented deviation. voluptuous’s own authors’ notion of the contract, checked.
* **Home Assistant `config_validation`:** the full suite passes, with voluptuous swapped out for Probatio.
* **100% line and branch coverage**, type-checked under both mypy and ty, in CI.
* **Fuzzed on every untrusted-input surface.** The first fuzzing pass found hundreds of exception leaks on hostile input. All fixed; none since.
* **Safer than the original.** A built-in validator only ever raises `Invalid`, never a raw exception, and that is enforced, not hoped for.
## Migrating from voluptuous
[Section titled “Migrating from voluptuous”](#migrating-from-voluptuous)
Probatio aims to be a drop-in replacement. In most cases the migration is a one-line import change. See [migrating from voluptuous](/getting-started/migrating-from-voluptuous/) for the details and the current compatibility status.
# Compatibility
> How Probatio stays compatible with voluptuous, the compatibility shim, and where it deviates on purpose.
Probatio targets behavioral compatibility with voluptuous 0.16.0. The public surface mirrors it: the same names, the same signatures, the same semantics. See the [API reference](/reference/) for the full surface. For most code, the only change is the import line.
## The compatibility shim
[Section titled “The compatibility shim”](#the-compatibility-shim)
Most code uses voluptuous through its public API. There, swapping `import voluptuous` for `import probatio` is enough, and your schemas keep working.
Some dependencies are harder. They import voluptuous *internals*, not just the public names. A real example is `annotatedyaml`, which imports `voluptuous.schema_builder._compile_scalar`. You cannot edit that code, and a plain import swap does not reach it.
For that case, Probatio ships `install_as_voluptuous` in `probatio.compat`. Call it once, very early in process startup, before anything imports voluptuous. It registers Probatio (and a small set of voluptuous-shaped submodules, including `schema_builder` with the `_compile_scalar` internal) into `sys.modules` under the `voluptuous` name. Every later `import voluptuous`, including the ones inside your dependencies, then resolves to Probatio for the rest of the process.
```python
from probatio.compat import install_as_voluptuous
install_as_voluptuous()
```
The call is idempotent but has no teardown. It shadows a real voluptuous that was already imported, and it aliases process-wide, so it is meant for an application or a test harness that owns the process. Do not call it from library code that others import.
Process-wide
`install_as_voluptuous` mutates `sys.modules` for the whole process. Call it from your application entry point, not from a library, and call it before the first `import voluptuous` anywhere in the process.
## Intentional deviations
[Section titled “Intentional deviations”](#intentional-deviations)
Compatibility is the target. A few behaviors still differ on purpose, and each is a deliberate improvement over a sharp edge:
* `str(error)` renders a dotted path and drops the error-type clause (ADR-015). voluptuous’s `expected int for dictionary value @ data['server']['port']` reads `expected int at 'server.port'` in Probatio, with sequence indices as `[n]`. A non-mapping is rejected as “expected a mapping” rather than “expected a dictionary”. The attributes (`path`, `msg`, `error_message`, `error_type`) are unchanged, so only code that string-matches `str(error)` notices.
* Recursive `Self` on cyclic or very deep data raises a clean `Invalid` instead of letting Python crash with a `RecursionError`. You get a validation error with a path, not a stack overflow.
* `from_json_schema` treats its input as untrusted. It refuses a catastrophically backtracking `pattern` and a pathologically deep document rather than hanging or overflowing the stack.
* A missing complex required key (`Required(Any("a", "b"))`, “at least one of these keys”) raises one clear error. voluptuous 0.16.0 additionally emits a redundant `required key not provided` for the same key; Probatio reports it once. The first, meaningful error matches voluptuous.
* Any `Mapping` is accepted, not only `dict`. A `MappingProxyType`, a multidict, or any custom mapping validates and returns a plain `dict`, where voluptuous rejects it. A genuine `dict` subclass is preserved as its own class, matching voluptuous, so a Home Assistant `NodeDictClass` keeps its type (and the source line it carries) across validation. A strict superset, so dict code is unaffected.
* A callable validator that raises `ValueError("reason")` keeps the reason in the error (“not a valid value: reason”), where voluptuous drops it. A `ValueError` with no message still reads “not a valid value”.
* An `enum` member works as a scalar schema and as a mapping key (matched by equality), where voluptuous 0.16.0 rejects it with `SchemaError`. Upstream adds this in [PR #537](https://github.com/alecthomas/voluptuous/pull/537); Probatio already has it, which Home Assistant relies on for its `StrEnum`/`IntEnum` service and attribute names.
* Built-in validators never leak a raw exception on a wrong-typed value: `Replace("a", "b")(42)` raises `MatchInvalid` rather than the `TypeError` from `re.sub`, and `Number()` on `None`/`dict`/`set`/`bytes`/an empty sequence raises `Invalid`. voluptuous fixes the same edges upstream in [PR #540](https://github.com/alecthomas/voluptuous/pull/540) and [PR #539](https://github.com/alecthomas/voluptuous/pull/539).
* `extend` accepts another `Schema`, merging its keys and preserving its `required` intent (its `extra` must match the result’s). voluptuous 0.16.0 raises an `AssertionError`; it is added upstream in [PR #538](https://github.com/alecthomas/voluptuous/pull/538).
* Validating a list reports every failing item, each with its index, where voluptuous 0.16.0 stops at the first failing nested item (open request, [issue #171](https://github.com/alecthomas/voluptuous/issues/171)). The diagnostics are more complete; error-iterating code is unaffected.
* A set schema (`{Coerce(int)}`) transforms its elements like a list schema does, so a `Coerce` coerces. voluptuous returns the set untransformed (bug, [issue #400](https://github.com/alecthomas/voluptuous/issues/400)).
* A failing `Any` with concrete branches lists them (“expected int or str or None”) as `AnyInvalid`, where voluptuous reports only the first branch or “not a valid value” ([issue #412](https://github.com/alecthomas/voluptuous/issues/412)). A validator branch keeps surfacing its own error.
Where Probatio diverges, it tells you with a normal error, not a crash.
## How compatibility is checked
[Section titled “How compatibility is checked”](#how-compatibility-is-checked)
The drop-in claim is tested, not asserted. voluptuous 0.16.0 is pinned as a dev-only oracle in Probatio’s development dependencies (no code is copied from it), and three layers of evidence back the claim:
* **voluptuous’s own test suite runs against Probatio.** Through the shim, upstream `tests.py` at 0.16.0 imports Probatio instead of voluptuous. A clean run is all green: every divergence is one of the documented deviations above, marked as expected with a reason, so any new break shows up loudly. This is voluptuous’s own authors’ notion of the contract, checked.
* **Differential tests in Probatio’s suite** (`tests/test_conformance.py`, `tests/test_fidelity.py`, `tests/test_v0_16.py`). The same schemas and inputs run through both libraries and the results are compared: the same accepted or normalized value, the same error paths, and the same bare error messages where downstream code string-matches them. A divergence is treated as a Probatio bug unless it is one of the documented deviations above.
* **Home Assistant’s `config_validation` test suite passes** with voluptuous swapped out for Probatio through the shim. That suite exercises voluptuous hard across a large, real configuration surface, so it is a strong signal that the drop-in promise holds in practice.
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Migrating from voluptuous](/getting-started/migrating-from-voluptuous/): the step-by-step swap.
* [API reference](/reference/): the full public surface.
# Installation
> Install Probatio, the Python versions it supports, and the optional speed backends.
Probatio is pure Python, and the core has zero runtime dependencies. There is nothing to compile, no build step, and no native extension to match against your platform. Install it and import it.
## Install
[Section titled “Install”](#install)
Pick your package manager:
* uv
```sh
uv add probatio
```
* pip
```sh
pip install probatio
```
* Poetry
```sh
poetry add probatio
```
## Supported Python versions
[Section titled “Supported Python versions”](#supported-python-versions)
Probatio needs Python 3.12 or newer. It is tested on CPython 3.12, 3.13, 3.14, and 3.15. If you can run those, you can run Probatio.
## No runtime dependencies
[Section titled “No runtime dependencies”](#no-runtime-dependencies)
The core has no runtime dependencies, and there are no extras to install. Probatio validates parsed Python objects: parsing and serialization stay with the caller. Parse your JSON, YAML, or TOML with whatever library you already use, then hand the result to a schema. See the [loading and dumping guide](/guides/loading-and-dumping/) for the pattern.
## Confirm it works
[Section titled “Confirm it works”](#confirm-it-works)
A tiny schema, validated:
```python
from probatio import Schema, Required, Optional
schema = Schema(
{
Required("name"): str,
Optional("port", default=8080): int,
}
)
schema({"name": "app"}) # {'name': 'app', 'port': 8080}
```
If that returns the dict with `port` filled in, you are set. Read the [quick start](/getting-started/quick-start/) next.
# Migrating from voluptuous
> Probatio aims to be a drop-in replacement for voluptuous. Here is what that means and how to switch.
Probatio is a clean-room reimplementation of [voluptuous](https://github.com/alecthomas/voluptuous) with the same public API. The goal is simple: you should be able to switch with a one-line import change and keep your existing schemas.
Behavioral compatibility with voluptuous is the correctness target, checked against voluptuous itself. If you hit a difference that is not listed below as intentional, treat it as a bug and [open an issue](https://github.com/frenck/probatio/issues).
## The short version
[Section titled “The short version”](#the-short-version)
Replace the import:
```python
# Before
from voluptuous import Schema, Required, Optional, All, Any, Coerce, Invalid
# After
from probatio import Schema, Required, Optional, All, Any, Coerce, Invalid
```
Your schemas stay the same:
```python
from probatio import Schema, Required, Optional, All, Range
schema = Schema(
{
Required("name"): str,
Optional("port"): All(int, Range(min=1, max=65535)),
}
)
```
## What carries over
[Section titled “What carries over”](#what-carries-over)
The names you reach for in voluptuous are present in Probatio with the same behavior and the same signatures (including positional `required`/`extra` on `Schema`):
* `Schema`, its calling convention, and `extend`.
* Markers: `Required`, `Optional`, `Remove`, `Extra`, `Inclusive`, `Exclusive`.
* Combinators: `All`, `Any`, `Union`, `Switch`, `SomeOf` (and `And`/`Or`).
* Validators: `Coerce`, `Range`, `Clamp`, `Length`, `In`, `NotIn`, `Match`, `Contains`, `Equal`, `Literal`, `Set`, `Unordered`, `Object`, `Maybe`, `ExactSequence`, `Unique`, the string transforms, `Email`/`Url`/`FqdnUrl`, `Datetime`/`Date`, `IsDir`/`IsFile`/`PathExists`, and the rest.
* Helpers: the `validate` decorator and the `raises` guard.
* Errors: `Invalid`, `MultipleInvalid`, and the semantic subclasses, including the error path, and `humanize_error` / `validate_with_humanized_errors` in `probatio.humanize`.
Probatio ships three submodules you can import from directly: `probatio.error`, `probatio.validators`, and `probatio.humanize`. Deeper `voluptuous.*` imports, including `voluptuous.util` and `voluptuous.schema_builder`, are covered by the [compatibility shim](/getting-started/compatibility/#the-compatibility-shim), which registers voluptuous-shaped modules under the `voluptuous` name.
## When a dependency imports voluptuous
[Section titled “When a dependency imports voluptuous”](#when-a-dependency-imports-voluptuous)
Your own code migrates with an import swap. A dependency you do not control may still `import voluptuous`, and editing your imports does not reach it. For that case, call `install_as_voluptuous()` from `probatio.compat` once, early at process startup; every later `import voluptuous` then resolves to Probatio, including the ones inside your dependencies. The [compatibility shim](/getting-started/compatibility/#the-compatibility-shim) section has the details and the one sharp edge (it aliases process-wide).
## Known differences
[Section titled “Known differences”](#known-differences)
Every intentional deviation is a deliberate improvement, not a regression, and the [compatibility page](/getting-started/compatibility/#intentional-deviations) owns the complete list. The ones that bite in practice:
* **The rendered error string differs (ADR-015).** voluptuous renders `expected int for dictionary value @ data['server']['port']`; Probatio renders the same error as `expected int at 'server.port'`. The attributes (`path`, `msg`, `error_message`, `error_type`) keep their voluptuous semantics, so only code that string-matches `str(error)` notices; switch that code to `path` and `error_message`, or to `as_dict()`.
* **Every failing list item is reported**, not just the first, so `MultipleInvalid.errors` can hold more entries than under voluptuous. Code that asserts an exact error count on list input notices; code that iterates the errors is unaffected.
* **Set schemas transform their elements.** A set schema like `{Coerce(int)}` runs the element schema on each item, so a `Coerce` actually coerces, where voluptuous returns the set untransformed.
Anything else that differs and is not on the [deviations list](/getting-started/compatibility/#intentional-deviations) is a bug; [open an issue](https://github.com/frenck/probatio/issues).
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Compatibility](/getting-started/compatibility/): the shim in full, the complete list of intentional deviations, and how the drop-in claim is tested.
* [Error handling](/guides/error-handling/): paths, multiple errors at once, and human-readable messages.
* [Dict schemas and markers](/guides/dict-schemas-and-markers/): required and optional keys, defaults, and extra-key policy.
# Quick start
> From nothing to a working schema, and what to read next.
A schema describes the shape of valid data. You build one from plain Python types and a few helpers, then call it with a value. A valid value comes back (possibly normalized); an invalid one raises an error that points at what went wrong.
Probatio not installed yet? Start with [installation](/getting-started/installation/).
## Your first schema
[Section titled “Your first schema”](#your-first-schema)
```python
from probatio import Schema, Required, Optional
schema = Schema(
{
Required("name"): str,
Optional("port", default=8080): int,
}
)
schema({"name": "app"}) # {'name': 'app', 'port': 8080}
```
Two things happened there. `name` was required, so leaving it out would fail. `port` was optional with a default, so it was filled in. The result is a new dict; the input is not mutated.
## Normalizing as you validate
[Section titled “Normalizing as you validate”](#normalizing-as-you-validate)
A schema does not only check, it can convert. `Coerce` turns the value into a type, and the converted value is what you get back:
```python
from probatio import Schema, Coerce, All, Range
port = Schema(All(Coerce(int), Range(min=1, max=65535)))
port("443") # 443
```
`All` runs its validators in order and feeds each result into the next, so the string becomes an int and then the int is range-checked.
## Handling failures
[Section titled “Handling failures”](#handling-failures)
When validation fails, the schema raises `MultipleInvalid` (a subclass of `Invalid`) with a path to the value that did not match, rendered as a dotted trail into the nested data:
```python
from probatio import Schema, Invalid
schema = Schema({"server": {"port": int}})
try:
schema({"server": {"port": "nope"}})
except Invalid as err:
print(err) # expected int at 'server.port'
```
Catching `Invalid` catches every validation error, because everything Probatio raises for bad data is a subclass of it. The [error handling guide](/guides/error-handling/) goes deeper into paths, multiple errors at once, and human-readable messages.
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Migrating from voluptuous](/getting-started/migrating-from-voluptuous/): swap your imports and keep your schemas.
* [The validation model](/guides/validation-model/): how schemas are built and what can go inside one.
* [Dict schemas and markers](/guides/dict-schemas-and-markers/): required and optional keys, defaults, and extra-key policy.
* [Validating a config file](/recipes/config-file/): a worked end-to-end example.
* [Dataclasses](/guides/dataclasses/) and [TypedDict](/guides/typeddict/): skip writing the schema when the type already says it.
* [JSON Schema](/guides/json-schema/) and [OpenAPI](/guides/openapi/): export what you validate.
* [Performance](/reference/performance/): fast for pure Python, with an opt-in compiled mode.
* [API reference](/reference/): the full public surface.
# Combinators
> Build validators from other validators with All, Any, Union, and SomeOf.
Combinators take other validators and combine them. They are how you express “all of these”, “any of these”, or “enough of these”, and they nest freely inside dict and list schemas.
## All: every validator, in order
[Section titled “All: every validator, in order”](#all-every-validator-in-order)
`All` runs its validators left to right, feeding each result into the next. It is both an “and” and a pipeline: the value is transformed as it passes through.
```python
from probatio import Schema, All, Coerce, Range
schema = Schema(All(Coerce(int), Range(min=1, max=65535)))
schema("443") # 443
```
Order matters. `All(Coerce(int), Range(min=0))` coerces the string to an int and then range-checks it; swapping them would range-check a string. `And` is an alias for `All`.
## Any: the first that matches
[Section titled “Any: the first that matches”](#any-the-first-that-matches)
`Any` returns the result of the first validator that accepts the value, and fails only when none do.
```python
from probatio import Schema, Any
schema = Schema(Any(int, str))
schema(5) # 5
schema("a") # 'a'
```
When nothing matches and every branch names something concrete (a type, `None`, or a literal), `Any` lists the alternatives, so the message tells you what was expected:
```python
from probatio import Schema, Any, Invalid
try:
Schema(Any(int, str, None))(1.5)
except Invalid as err:
print(err) # expected int or str or None
```
This is a Probatio improvement over voluptuous, which reports only the first branch (see [issue #412](https://github.com/alecthomas/voluptuous/issues/412)). When a branch is an arbitrary validator (a `Range`, a callable) there is no clean label, so `Any` falls back to the error from the branch that got furthest (the deepest path), which is usually the most useful one. Pass `msg` to replace the message with your own:
```python
from probatio import Schema, Any, Invalid
schema = Schema(Any("red", "green", "blue", msg="not a known color"))
try:
schema("mauve")
except Invalid as err:
print(err) # not a known color
```
`Or` is an alias for `Any`.
## Union and Switch: pick a branch
[Section titled “Union and Switch: pick a branch”](#union-and-switch-pick-a-branch)
`Union` is like `Any`, but a `discriminant` can narrow which branches to try instead of attempting every one. The discriminant is called as `discriminant(value, validators)` and returns the subset to try, which is how you validate a tagged union by its tag:
```python
from probatio import Schema, Union
def by_type(value, alternatives):
return [a for a in alternatives if a["type"] == value.get("type")]
schema = Schema(
Union(
{"type": "point", "x": int, "y": int},
{"type": "label", "text": str},
discriminant=by_type,
)
)
schema({"type": "point", "x": 1, "y": 2}) # {'type': 'point', 'x': 1, 'y': 2}
```
The win is the error message: with a discriminant, a bad `point` reports the problem with its own fields, rather than “matched none of the alternatives.” Note that the discriminant sees the raw input, and the one above assumes a mapping; a production discriminant should also handle a non-mapping value. `Switch` is an alias for `Union`.
For the common case, routing on one key’s value, reach for `TaggedUnion` instead of writing the discriminant by hand:
```python
from probatio import Schema, TaggedUnion
schema = Schema(
TaggedUnion(
"type",
{
"point": {"type": "point", "x": int, "y": int},
"label": {"type": "label", "text": str},
},
)
)
schema({"type": "point", "x": 1, "y": 2}) # {'type': 'point', 'x': 1, 'y': 2}
```
`TaggedUnion` reads `value["type"]`, validates against the one listed schema, and on an unknown tag rejects with the valid tags named. Pass `default=` for a fallback. It is the direct equivalent of Home Assistant’s `cv.key_value_schemas`.
When each branch pins its tag as a literal, pass a list instead of a mapping and the tag is read from each branch (written once, not repeated as the key); the routing table is still built once, so it is no slower:
```python
from probatio import Schema, TaggedUnion, Required
POINT = Schema({Required("type"): "point", Required("x"): int, Required("y"): int})
LABEL = Schema({Required("type"): "label", Required("text"): str})
schema = Schema(TaggedUnion("type", [POINT, LABEL]))
schema({"type": "point", "x": 1, "y": 2}) # {'type': 'point', 'x': 1, 'y': 2}
```
## SomeOf: enough of them
[Section titled “SomeOf: enough of them”](#someof-enough-of-them)
`SomeOf` requires the value to pass a bounded number of its validators. Give at least one of `min_valid` and `max_valid`.
```python
from probatio import Schema, SomeOf, Range
schema = Schema(SomeOf(min_valid=2, validators=[Range(1, 5), int, 3]))
schema(3) # 3
```
Too few passes raises [`NotEnoughValid`](/reference/errors/); too many raises [`TooManyValid`](/reference/errors/). Like `All`, each validator’s output feeds the next.
## Combinators nest
[Section titled “Combinators nest”](#combinators-nest)
Combinators are validators, so they go anywhere a validator goes: as a dict value, a list element, or inside another combinator.
```python
from probatio import Schema, All, Any, Coerce, Range, Required
schema = Schema(
{
Required("level"): Any("low", "high", All(Coerce(int), Range(min=0, max=10))),
}
)
schema({"level": "high"}) # {'level': 'high'}
schema({"level": "7"}) # {'level': 7}
```
## Passing options through
[Section titled “Passing options through”](#passing-options-through)
`All`, `Any`, `Union`, and `SomeOf` accept `required=True`, which propagates into any mapping sub-schemas they wrap (so the nested dicts treat their keys as required). Other keyword arguments are accepted and ignored, matching voluptuous.
# Compiled schemas
> How Probatio compiles a hot schema into a specialized validator, on its own by default, and how to control or turn it off.
Probatio validates with one interpreted engine. On top of that, it can compile a schema into a specialized validator: a flat function generated for that exact schema, with the common per-key work unrolled and the common validators inlined. It is faster, and it is the same result: on a representative config schema, compiling takes a validation from roughly 2 µs to under 1 µs, about seven times faster than the same schema in voluptuous (see [Numbers](#numbers)). Compiling only changes speed, never what a schema accepts, rejects, or how it reports an error.
By default this happens on its own. You do not have to do anything to get it, and in most cases you should not have to think about it at all. This page explains what it does, how to steer it when you want to, and the one case where you should turn it off.
## The default: it compiles itself
[Section titled “The default: it compiles itself”](#the-default-it-compiles-itself)
The process starts on the `AUTO` policy. Under `AUTO`, a schema validates interpreted and quietly counts how often it is called. Once it has proven hot (it has been validated enough times to be worth the cost), it compiles itself and runs through the generated validator from then on. The line is currently drawn at roughly 50 validations, an implementation detail that may move. A schema you build and call once, or a handful of times, never crosses it, so it stays interpreted and never pays for code generation.
There is no public way to ask whether a schema has compiled; the observable effect is throughput, so measure that instead. [Performance](/reference/performance/) shows how to benchmark it yourself.
So the practical rule is the same as it has always been: build a schema once, at module scope, and reuse it. The hot ones speed themselves up.
```python
from probatio import Schema, All, Coerce, Range
# Built once, reused for every value. After enough calls it compiles itself.
PORT = Schema(All(Coerce(int), Range(min=1, max=65535)))
PORT("443") # 443
PORT(8080) # 8080
```
## Asking for it explicitly
[Section titled “Asking for it explicitly”](#asking-for-it-explicitly)
When you know a schema is hot and you do not want to wait for the warmup, opt in per schema. The `compile` flag and `Schema.compile()` both do that; `compile()` is spelled to mirror `re.compile`, and unlike the flag it generates right away rather than on first use.
```python
from probatio import Schema, Required, Optional
# Opt in at construction (generated on first validation).
schema = Schema({Required("name"): str, Optional("age"): int}, compile=True)
# Or opt in eagerly, generating now. Returns the same schema.
schema = Schema({Required("name"): str, Optional("age"): int}).compile()
```
`DataclassSchema` and `TypedDictSchema` take the same flag, and they are the clearest win: compiling fuses field validation and object construction into one function with no intermediate dict.
```python
from dataclasses import dataclass
from probatio import DataclassSchema
@dataclass
class Point:
x: int
y: int
POINT = DataclassSchema(Point).compile()
POINT({"x": 1, "y": 2}) # Point(x=1, y=2)
```
To opt a schema out, pass `compile=False`. It then stays interpreted whatever the process policy is.
## The process-wide policy
[Section titled “The process-wide policy”](#the-process-wide-policy)
The default for any schema that does not set its own flag is the process policy. Set it once, early, from deliberate startup code. There is no environment variable on purpose: this is an architectural choice, not a deployment toggle that flips behavior invisibly.
```python
from probatio import CompilePolicy, set_compile_policy
set_compile_policy(CompilePolicy.OFF) # never compile unless a schema opts in
set_compile_policy(CompilePolicy.ON) # compile every eligible schema on first use
set_compile_policy(CompilePolicy.AUTO) # the default: compile a schema once it is hot
```
A per-schema `compile` flag always wins over the policy, in either direction. The order of precedence is: `schema.compile()`, then the `compile=True`/`compile=False` flag, then the process policy.
Applications set this, not libraries
`set_compile_policy` is a single process-wide switch, so only an application (the process that owns the run) should call it. A library that sets it changes compilation for every schema in the process, the application’s own and those of unrelated dependencies included, which is not a library’s call to make. A library that wants a particular schema compiled (or kept interpreted) sets the per-schema flag instead, `Schema(..., compile=True)` or `compile=False`, and even then sparingly: whether compilation pays depends on how the application drives the schema, so the `AUTO` default is usually the right answer.
## What compiles, and what does not
[Section titled “What compiles, and what does not”](#what-compiles-and-what-does-not)
The generator handles the common shapes: dict (mapping) schemas, dataclasses, and TypedDicts, including nested mappings, and single-element list schemas (`[str]`, `[All(Coerce(int), Range(...))]`) both inside a mapping and at the top level. It inlines the validators that show up most: `Coerce` to `int`/`float`, numeric `Range`, plain `In` membership, type checks, an `Any` of literal choices (`Any("on", "off")`, validated as a membership test), `Maybe(X)` (an optional validated value), and `All`/`Any` chains of those.
Anything it does not handle is simply left interpreted. A recursive schema (`Self` or a recursive JSON Schema `$ref`), an exclusive or inclusive group, a multi-element list (an “each item matches one of these” union), a list whose element does not inline, an exotic key: the schema works exactly as before, just without the speedup. You never lose correctness by compiling; at worst you do not gain speed.
This is the heart of the design. The generated function is a fast **success path only**. The moment anything is off (a missing required key, a type mismatch, a validator that raises, an unexpected key) it hands the value back to the interpreted engine, which produces every error, path, code, and ordering. So there is one set of validation semantics, not two, and the compiled path differs only by speed (with one caveat for side-effecting validators on a failing validation, noted below).
Note
Because failures run through the interpreted engine, an error-heavy workload sees interpreted speed, not compiled speed. Compilation accelerates the happy path, which is the one that runs in a steady system.
Keep validators pure
The fast path runs fields optimistically. When a validation fails, a validator or a `default` factory that already ran before a later field bailed runs a second time in the interpreted re-run. The returned value and the raised error are always identical; only the count of side effects differs, and only on a failing validation. So a validator or default factory that mutates state (a counter, a log line, an external call) can fire more than once for input that is then rejected. Keep them pure, which is the right shape for a validator regardless of compilation.
## The one time to turn it off
[Section titled “The one time to turn it off”](#the-one-time-to-turn-it-off)
Each compiled schema holds a generated code object and a small namespace capturing its validators. For a fixed set of schemas (the normal case, even a large one built once at import) that is bounded and cheap, and schemas of the same shape share a single code object behind the scenes.
The exception is a workload that builds an unbounded stream of **unique** schemas, a new distinct schema per request that is never reused. There, compiling each one grows memory without bound. `AUTO` already protects you (a one-shot schema never crosses the warmup threshold, so it is never compiled), but if you run such a workload under `ON`, turn it off:
```python
from probatio import CompilePolicy, set_compile_policy
set_compile_policy(CompilePolicy.OFF)
```
If your schemas are a fixed set built once, which is almost always the case, you do not need to think about this.
## Numbers
[Section titled “Numbers”](#numbers)
On the bundled benchmarks, compiled validation runs up to roughly seven times faster than the same schema in voluptuous, and the interpreted engine already runs about one and a half to three and a half times faster. The exact ratio moves with the machine, the Python version, and the schema. See [Performance](/reference/performance/) for the cost model, the honest caveats, and how to measure it yourself.
# Custom error messages
> Override a validator's message with msg, raise your own Invalid carrying structured data, and render validation errors in your own words.
Probatio’s default messages are written for humans, but they are still Probatio’s words. Sometimes you want your own: a schema author wants one rule to explain itself better, an application wants to render failures in its own tone, its own UI, or its own language. This guide covers both ends: changing the message where the schema is defined, and ignoring the message entirely to build your own output from the structured data every error carries.
## Override the message with msg
[Section titled “Override the message with msg”](#override-the-message-with-msg)
Most validators and markers accept a `msg` argument that replaces the default message wholesale. This is the one-line tool for the spot where the default wording does not fit your users:
```python
from probatio import Schema, Required, Range, MultipleInvalid
schema = Schema(
{
Required("port", msg="a port is required"): Range(
min=1, max=65535, msg="not a valid port number"
),
}
)
try:
schema({})
except MultipleInvalid as err:
# error_message is the bare message, without the path
print(err.errors[0].error_message) # a port is required
try:
schema({"port": 0})
except MultipleInvalid as err:
# printing the error itself appends the path
print(err.errors[0]) # not a valid port number at 'port'
```
The replacement is complete: `msg` becomes the error’s message, and the path is still appended by `str(error)`. Everything else about the error is unchanged; it keeps its class (`RangeInvalid` here) and its machine-readable `code`, so code that branches on those keeps working no matter what the message says.
`msg` changes one validator’s words. If you find yourself passing `msg` to every validator in a schema to restyle all output, you are working at the wrong end: skip to [rendering errors your way](#render-errors-your-way) instead.
## Raise your own Invalid
[Section titled “Raise your own Invalid”](#raise-your-own-invalid)
A [custom validator](/guides/custom-validators/) rejects a value by raising `Invalid` (or a subclass), and the message is yours from the start. Beyond the message, `Invalid` accepts the structured fields, so your error carries data for a consumer instead of only prose:
* `code`: a stable machine-readable identifier for this kind of failure.
* `context`: a dict of structured detail, such as the limit that was exceeded.
* `translation_key` and `placeholders`: a lookup key plus the values to interpolate, for a consumer that renders localized messages.
```python
from probatio import Schema, Invalid, MultipleInvalid
def quiet_hour(value):
if not isinstance(value, int) or not 0 <= value <= 23:
raise Invalid(
"expected an hour between 0 and 23",
code="quiet_hour",
context={"min": 0, "max": 23},
translation_key="quiet_hour_out_of_range",
placeholders={"min": 0, "max": 23},
)
return value
schema = Schema({"start": quiet_hour})
try:
schema({"start": 25})
except MultipleInvalid as err:
error = err.errors[0]
print(error) # expected an hour between 0 and 23 at 'start'
print(error.code) # quiet_hour
print(error.translation_key) # quiet_hour_out_of_range
print(error.placeholders) # {'min': 0, 'max': 23}
```
The message is the fallback for anyone who just prints the error; the structured fields are the contract for anyone building their own output. Fill both and your validator works in either world.
## Render errors your way
[Section titled “Render errors your way”](#render-errors-your-way)
The rendered string is for humans and makes no stability promise; the structured attributes are the API. A consumer that wants its own output (a CLI with its own formatting, a web API returning JSON, a UI translating messages) reads those attributes and never touches Probatio’s strings.
The pattern: catch `MultipleInvalid`, walk `errors`, and build output from `path`, `code`, `context`, and `error_message`. `render_path` renders a path the same way `str(error)` does, so your output stays consistent with the library’s:
```python
from probatio import Schema, MultipleInvalid
from probatio.error import render_path
schema = Schema({"server": {"port": int, "host": str}})
data = {"server": {"port": "nope", "host": 42}}
try:
schema(data)
except MultipleInvalid as err:
for error in sorted(err.errors, key=lambda e: str(e.path)):
where = render_path(error.path) or "the top level"
print(f"Problem with option '{where}': {error.error_message}.")
# Problem with option 'server.host': expected str.
# Problem with option 'server.port': expected int.
```
For an API, `as_dict()` serializes the structured layer of every error in one call, ready for a JSON response body:
```python
import json
from probatio import Schema, MultipleInvalid
schema = Schema({"port": int})
try:
schema({"port": "nope"})
except MultipleInvalid as err:
body = json.dumps(err.as_dict())
print(body)
# {"errors": [{"code": "type", "message": "expected int", "path": ["port"],
# "secret": false, "context": {"expected": "int"},
# "translation_key": "expected_type",
# "placeholders": {"expected": "int"}}]}
```
For translations, branch on `translation_key`. Every built-in error carries one, naming the exact sentence, together with `placeholders`, the raw values that sentence interpolates (the [translation keys reference](/reference/translation-keys/) lists every key and its English template). A lookup table maps sentences to your own language, with the original message as the fallback for keys you have not translated:
```python
from probatio import Schema, MultipleInvalid
DUTCH = {
"expected_type": "verwacht een waarde van type {expected}",
"required": "deze optie is verplicht",
"length_min": "lengte moet minimaal {min} zijn",
}
def render_dutch(error):
template = DUTCH.get(error.translation_key)
if template is None:
return error.error_message
return template.format(**error.placeholders)
schema = Schema({"port": int, "host": str})
try:
schema({"port": "nope"})
except MultipleInvalid as err:
print(render_dutch(err.errors[0])) # verwacht een waarde van type int
```
One sentence, one key: where two validators produce the same wording (the dict engine and the key groups both say “expected a mapping”), they share the key, so one translation covers both. `code` still identifies the *kind* of failure and is the better branch point for behavior; `translation_key` identifies the *sentence* and is the branch point for wording.
### Suggestions compose on top
[Section titled “Suggestions compose on top”](#suggestions-compose-on-top)
Some errors carry a “did you mean …?” suggestion (an unknown key close to a known one, for example; the [errors reference](/reference/errors/) marks which classes). The suggestion is *not* part of the base sentence: the key names the sentence without it, and the close matches live on `context["candidates"]` as a raw list. Probatio’s own English output appends the fragment (its template is the `did_you_mean` key), and `error_message` always includes it, so the fallback path keeps suggestions for free. A renderer working from keys has to compose the fragment itself, or the suggestion is silently dropped:
```python
from probatio import Schema, MultipleInvalid
DUTCH = {
"not_a_valid_option": "geen geldige optie",
"did_you_mean": ", bedoelde je {candidates}?",
}
def render_dutch(error):
text = DUTCH.get(error.translation_key)
if text is None:
return error.error_message
text = text.format(**error.placeholders)
candidates = error.context.get("candidates")
if candidates:
joined = " of ".join(repr(name) for name in candidates)
text += DUTCH["did_you_mean"].format(candidates=joined)
return text
schema = Schema({"name": str, "email": str})
try:
schema({"nmae": "app"})
except MultipleInvalid as err:
print(render_dutch(err.errors[0])) # geen geldige optie, bedoelde je 'name'?
```
The list joining (the “of”/“or” between candidates) is deliberately yours: it is language, and the raw list gives you full control over it.
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Custom validators](/guides/custom-validators/): the validator side of raising your own errors.
* [Error handling](/guides/error-handling/): paths, `MultipleInvalid`, and catching specific kinds.
* [Errors reference](/reference/errors/): every error class, its `code`, and the fields on `Invalid`.
* [Translation keys](/reference/translation-keys/): every key the built-ins emit, with its English template.
# Custom validators
> Write your own validators as plain callables, reject with Invalid, transform on the way through, and reshape failure messages.
The built-in toolbox covers a lot, but eventually you need a rule that is yours. In Probatio that rule is just a function. Any callable is a validator: it receives the value and returns the result, or raises to reject it. There is no base class to inherit and no interface to implement.
## A callable is a validator
[Section titled “A callable is a validator”](#a-callable-is-a-validator)
Put a function anywhere a validator goes. It is called with the value, and what it returns becomes the validated result.
```python
from probatio import Schema
def double(value):
return value * 2
schema = Schema(double)
schema(21) # 42
```
That is the whole contract. Return a value to accept, raise to reject.
## Rejecting with Invalid
[Section titled “Rejecting with Invalid”](#rejecting-with-invalid)
To reject a value, raise `Invalid` with a message. The message is what the caller sees, and Probatio attaches the path to the offending value for you.
```python
from probatio import Schema, Invalid, Required
def even(value):
if value % 2 != 0:
raise Invalid("must be even")
return value
schema = Schema({Required("count"): even})
schema({"count": 4}) # {'count': 4}
```
When the value fails, the message lands inside the usual error, pointing at the key that broke:
```python
from probatio import Schema, Invalid, Required, MultipleInvalid
def even(value):
if value % 2 != 0:
raise Invalid("must be even")
return value
schema = Schema({Required("count"): even})
try:
schema({"count": 3})
except MultipleInvalid as err:
print(err) # must be even at 'count'
```
## A plain ValueError works too
[Section titled “A plain ValueError works too”](#a-plain-valueerror-works-too)
You do not have to import `Invalid` for the simple cases. If your callable raises a `ValueError`, Probatio catches it and turns it into an `Invalid`. This is on purpose: a lot of standard-library and third-party functions already raise `ValueError` on bad input, so they drop straight in as validators.
```python
from probatio import Schema
def port(value):
number = int(value) # raises ValueError on "nope"
if not 0 < number <= 65535:
raise ValueError("out of range")
return number
schema = Schema(port)
schema("443") # 443
```
The catch keeps the `ValueError` reason, appending it after `not a valid value: `(a carry-forward of voluptuous [issue #417](https://github.com/alecthomas/voluptuous/issues/417), where the reason was dropped):
```python
from probatio import Schema, MultipleInvalid
def port(value):
number = int(value)
if not 0 < number <= 65535:
raise ValueError("out of range")
return number
schema = Schema(port)
try:
schema("99999")
except MultipleInvalid as err:
print(err) # not a valid value: out of range
```
Tip
Raise `ValueError` for the convenience: the reason is preserved, but it always carries the `not a valid value: `prefix. Raise `Invalid` when you want full control of the message the caller reads, with no prefix.
## Wrapping a predicate with truth
[Section titled “Wrapping a predicate with truth”](#wrapping-a-predicate-with-truth)
Some checks already exist as predicates: a function that answers `True` or `False`, like `os.path.isdir` or `str.isidentifier`. A predicate is not a validator, because it returns the answer instead of the value. `truth` (a voluptuous carry-over) bridges that: it wraps the predicate into a validator that passes the value through unchanged when the predicate is truthy, and rejects it when it is falsy.
```python
from probatio import Schema, truth
@truth
def positive(value):
return value > 0
schema = Schema(positive)
schema(5) # 5
```
A falsy result raises a bare `ValueError` under the hood, so the failure reads `not a valid value`, with no reason attached. Wrap the result in [`Msg`](#reshaping-the-message-with-msg) when the caller deserves a better message.
```python
from probatio import Schema, MultipleInvalid, truth
@truth
def positive(value):
return value > 0
schema = Schema(positive)
try:
schema(-1)
except MultipleInvalid as err:
print(err) # not a valid value
```
## Transforming the value
[Section titled “Transforming the value”](#transforming-the-value)
A validator does not have to return the input untouched. Returning a different value is how you normalize: trim, lowercase, parse, canonicalize. The schema hands back what you return, and the original input stays as it was (see [the validation model](/guides/validation-model/)).
```python
from probatio import Schema
def to_slug(value):
return value.strip().lower().replace(" ", "-")
schema = Schema(to_slug)
schema(" Hello World ") # 'hello-world'
```
## Reshaping the message with Msg
[Section titled “Reshaping the message with Msg”](#reshaping-the-message-with-msg)
Sometimes the validator you want to reuse already raises a fine error, just not the one you want here. `Msg(validator, msg, cls=None)` wraps a validator and replaces its failure message. Pass `cls` to also swap the error class, so callers can catch your failure by type.
```python
from probatio import Schema, Msg, Match
schema = Schema(Msg(Match(r"^[a-z]+$"), "lowercase letters only"))
schema("frenck") # 'frenck'
```
On failure the wrapped validator’s own message is gone, replaced by yours:
```python
from probatio import Schema, Msg, Match, MultipleInvalid
schema = Schema(Msg(Match(r"^[a-z]+$"), "lowercase letters only"))
try:
schema("ABC")
except MultipleInvalid as err:
print(err) # lowercase letters only
```
With `cls`, the raised error is your own subclass, which a caller can catch specifically:
```python
from probatio import Schema, Msg, Match, Invalid, MultipleInvalid
class BadName(Invalid):
"""The name does not look right."""
schema = Schema(Msg(Match(r"^[a-z]+$"), "lowercase letters only", cls=BadName))
try:
schema("ABC")
except MultipleInvalid as err:
print(type(err.errors[0]).__name__) # BadName
```
## Composing with All
[Section titled “Composing with All”](#composing-with-all)
Custom validators are validators, so they nest like any other. Drop them into `All` to chain a normalize step in front of your check, and the result of each validator feeds the next (see [combinators](/guides/combinators/)).
```python
from probatio import Schema, All, Strip, Lower, Invalid, Required
def not_empty(value):
if not value:
raise Invalid("must not be empty")
return value
schema = Schema({Required("name"): All(Strip, Lower, not_empty)})
schema({"name": " Frenck "}) # {'name': 'frenck'}
```
`Strip` and `Lower` normalize the value before `not_empty` ever sees it, so a field of only spaces collapses to an empty string and is rejected:
```python
from probatio import Schema, All, Strip, Lower, Invalid, Required, MultipleInvalid
def not_empty(value):
if not value:
raise Invalid("must not be empty")
return value
schema = Schema({Required("name"): All(Strip, Lower, not_empty)})
try:
schema({"name": " "})
except MultipleInvalid as err:
print(err) # must not be empty at 'name'
```
## Parameterizing a validator
[Section titled “Parameterizing a validator”](#parameterizing-a-validator)
A validator that needs a setting is just a function that returns a function. The outer call captures the parameter, the inner function does the check. This is how every built-in factory works, `Range(min=...)` and `Length(max=...)` included.
```python
from probatio import Schema, Invalid, MultipleInvalid
def at_least(minimum):
def check(value):
if value < minimum:
raise Invalid(f"must be at least {minimum}")
return value
return check
schema = Schema(at_least(18))
schema(21) # 21
try:
schema(16)
except MultipleInvalid as err:
print(err) # must be at least 18
```
A class with a `__call__` method works the same way and is handier when the validator carries several settings or wants a readable `repr`:
```python
from probatio import Schema, Invalid
class AtLeast:
def __init__(self, minimum):
self.minimum = minimum
def __call__(self, value):
if value < self.minimum:
raise Invalid(f"must be at least {self.minimum}")
return value
Schema(AtLeast(18))(21) # 21
```
## A type that validates itself
[Section titled “A type that validates itself”](#a-type-that-validates-itself)
A bare type used as a schema validates by `isinstance`. For a type whose runtime value differs from its raw form, that is the wrong check. An enum is the clear case: its value is the string a loader gives you, but `isinstance` accepts only an already-built member. So Probatio treats an enum class as a schema specially: it accepts a member or any of the enum’s values, and returns the member.
```python
import enum
from probatio import Schema
class Color(enum.Enum):
RED = "red"
BLUE = "blue"
schema = Schema(Color)
schema("red") # Color.RED
schema(Color.BLUE) # Color.BLUE
```
Your own types can opt into the same treatment with a `__probatio_validate__` classmethod. When that type is a schema, Probatio calls the method instead of the `isinstance` check. The method validates (and may normalize) the raw value, and raises `Invalid` to reject it, like any validator.
```python
from typing import Any
from probatio import Schema, Invalid
class Slug:
def __init__(self, value: str) -> None:
self.value = value
@classmethod
def __probatio_validate__(cls, value: Any) -> "Slug":
if not isinstance(value, str):
raise Invalid("expected a string slug")
return cls(value.lower())
Schema(Slug)("Hello").value # 'hello'
```
This keeps the “how do I validate a raw value of myself” knowledge on the type, instead of wrapping every use in a `Coerce`. It is the principled way to make a domain type a first-class schema. For a type you do not own, reach for `Coerce` or a small validator function instead. See [ADR-007](https://github.com/frenck/probatio/blob/main/adr/007-self-validation-protocol.md) for the design.
## Validating against call-time context
[Section titled “Validating against call-time context”](#validating-against-call-time-context)
Some checks depend on state known only when you validate, not when you build the schema: a set of allowed values from a request, the current user’s permissions, a list from the database. Pass it as `context` to the call, and a validator reads it with `current_context()`. One compiled schema then serves many calls with different state, instead of rebuilding the schema each time.
```python
from probatio import Schema, Required, current_context, Invalid
def allowed(value):
context = current_context() or {}
if value not in context.get("allowed", ()):
raise Invalid(f"{value!r} is not allowed")
return value
schema = Schema({Required("entity"): allowed})
schema({"entity": "light.kitchen"}, context={"allowed": {"light.kitchen"}})
```
The context is set for the duration of the call and visible to every validator it reaches, including nested schemas. A nested schema that passes its own `context` overrides it for that subtree; one that passes none inherits the enclosing call’s. Without a `context`, `current_context()` is `None`, so a validator that reads it decides what an absent context means. It is async- and thread-safe, and a plain `schema(data)` call sets nothing.
## Validating function arguments
[Section titled “Validating function arguments”](#validating-function-arguments)
The `probatio` decorator checks a function’s arguments against its type annotations, the same way a `Schema` checks data: it reads the signature, infers a validator for each annotated parameter, and validates (and coerces) the call before the body runs. A bad argument raises, so the body only ever sees valid input.
```python
from probatio import probatio, MultipleInvalid
@probatio
def area(width: int, height: int) -> int:
return width * height
area(3, 4) # 12
try:
area("wide", 4)
except MultipleInvalid as err:
print(err) # expected int at 'width'
```
Layer extra rules with a `constraints` map, and validate the result against the `-> R` annotation with `returns=True`:
```python
from probatio import probatio, Length
@probatio({"name": Length(min=2)}, returns=True)
def greet(name: str) -> str:
return "hi " + name
greet("ada") # 'hi ada'
```
An unannotated parameter (`self`, `cls`, a bare `*args`) is left alone, and a coroutine function is validated the same way, awaiting the call before the result schema runs. See [the probatio decorator](/guides/probatio-decorator/) for the full picture. The voluptuous-style `validate` decorator, which names a schema per argument (`@validate(width=int, __return__=int)`), stays available for drop-in compatibility.
The companion `raises` context manager asserts that a block raises a given error, optionally matching the message. It is a test helper, so it lives with the rest of the testing story: see [Testing with pytest](/guides/testing-with-pytest/).
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Custom error messages](/guides/custom-error-messages/): carrying structured data (`code`, `context`, localization slots) on the errors you raise, and rendering errors your way as a consumer.
* [The validation model](/guides/validation-model/): why a returned value can differ from the input.
* [Combinators](/guides/combinators/): `All`, `Any`, and friends.
* [Recursive schemas](/guides/recursive-schemas/): validators that refer back to their own schema.
# Schemas from dataclasses
> Generate a validating schema straight from a dataclass, with the field annotations driving the validators.
If your data already has a dataclass, you should not have to write the schema twice. `DataclassSchema` reads a dataclass’s fields and builds a schema from the annotations: it validates a plain mapping and hands you back a constructed instance. This is the same idea as voluptuous [PR #533](https://github.com/alecthomas/voluptuous/pull/533), with a richer type mapping (Probatio descends into element types instead of stopping at the container).
The same engine reads a `TypedDict`. If your shape is a dict rather than a constructed object, see [Schemas from TypedDicts](/guides/typeddict/); this page is about dataclasses.
## The basics
[Section titled “The basics”](#the-basics)
Pass a dataclass type to `DataclassSchema`. A field without a default is required; a field with a default (or `default_factory`) is optional, and the default fills in when the key is absent.
```python
from dataclasses import dataclass
from probatio import DataclassSchema
@dataclass
class User:
name: str
age: int = 18
schema = DataclassSchema(User)
schema({"name": "ada"}) # User(name='ada', age=18)
```
The result is a real `User`, not a dict. A missing required field is reported like any other validation error:
```python
from dataclasses import dataclass
from probatio import DataclassSchema
@dataclass
class User:
name: str
age: int = 18
DataclassSchema(User)({}) # required key not provided at 'name'
```
## Annotations drive the validators
[Section titled “Annotations drive the validators”](#annotations-drive-the-validators)
The mapping is deep, not just the container type. A parameterized generic keeps its element types, a union with `None` becomes `Maybe`, and a nested dataclass recurses into its own schema.
```python
from dataclasses import dataclass, field
from probatio import DataclassSchema
@dataclass
class Address:
street: str
number: int = 0
@dataclass
class Person:
name: str
tags: list[str] = field(default_factory=list)
home: Address | None = None
schema = DataclassSchema(Person)
schema({"name": "ada", "tags": ["x"], "home": {"street": "Main", "number": 5}})
# Person(name='ada', tags=['x'], home=Address(street='Main', number=5))
```
The full set of mappings:
| Annotation | Schema |
| -------------------------- | ----------------------------------------- |
| `int`, `str`, a plain type | the type (an `isinstance` check) |
| `float` | the numeric tower (`int` in, `float` out) |
| `list[T]` | `[T]` (element type validated) |
| `set[T]`, `frozenset[T]` | `{T}` / `frozenset([T])` |
| `dict[K, V]` | `{K: V}` (key and value validated) |
| `tuple[X, Y]` | `ExactSequence([X, Y])` (positional) |
| `tuple[X, ...]` | a homogeneous sequence of `X` |
| `X \| None` | `Maybe(X)` |
| `X \| Y` | `Any(X, Y)` |
| `Literal["a", "b"]` | `In(["a", "b"])` |
| a nested dataclass | its own generated schema |
| `Any` | accepts any value |
A tuple field accepts a list or a tuple, since a sequence arrives as a list from JSON, and keeps whichever you gave it.
A `float` field is the one plain type that is not a bare `isinstance`. It honors the PEP 484 numeric tower: an `int` is a valid `float` value, so `5` is accepted and normalized to `5.0`, while `bool`, a string, and `None` are still rejected. See [ADR-017](https://github.com/frenck/probatio/blob/main/adr/017-numeric-tower-for-float.md).
## Layering extra rules
[Section titled “Layering extra rules”](#layering-extra-rules)
The annotation gives you the type check. For anything beyond that (a length, a range, a regular expression), pass `additional_constraints`: a map from field name to a validator that runs after the type check.
```python
from dataclasses import dataclass
from probatio import DataclassSchema, Length
@dataclass
class User:
name: str
schema = DataclassSchema(User, {"name": Length(min=2)})
schema({"name": "ada"}) # User(name='ada')
```
Every key must name a field. A key that matches nothing (a typo, or a dotted path like `"address.number"`, which is not resolved) would never run, so the schema is refused with a `SchemaError` when it is built.
The same rule can live on the field itself with `Annotated`, keeping the constraint next to the field instead of in a separate map. The rules:
* The first argument is the type; every callable after it is applied as a validator, in order.
* The type is checked on the result, not the raw input, so the annotation says what the field *is*: a coercer runs first and the type confirms what it produced.
* It composes inside containers: `list[Annotated[int, Range(min=1)]]` checks every element.
* A producing base (an enum, a nested dataclass, a container) coerces first, so an asserting constraint after it checks the produced value. A coercer in the metadata (`Coerce` or `Map`) still runs ahead of it, so you can reshape a raw value, like folding an enum sentinel to `None`, before the base sees it (see the [cookbook](/recipes/cookbook/#device-and-sensor-conversions)).
* Metadata that is not callable is left alone, so an `Annotated` value you share with another tool passes through untouched.
That second rule is what keeps a coerced field honestly typed. `Annotated[datetime, AsDatetime()]` accepts the string, parses it, and confirms a `datetime`, instead of annotating `str` and hiding the real type in the validator. To find the right callable to drop in here, whether it checks the value or transforms it, see [Built-ins by role](/reference/builtins-by-role/).
```python
from dataclasses import dataclass
from typing import Annotated
from probatio import DataclassSchema, Length, Range
@dataclass
class User:
name: Annotated[str, Length(min=2)]
age: Annotated[int, Range(min=0)] = 0
schema = DataclassSchema(User)
schema({"name": "ada", "age": 30}) # User(name='ada', age=30)
```
Here is that coercer-plus-type pairing in practice:
```python
from dataclasses import dataclass
from datetime import datetime
from typing import Annotated
from probatio import DataclassSchema, AsDatetime
@dataclass
class Event:
when: Annotated[datetime, AsDatetime()]
DataclassSchema(Event)({"when": "2020-01-01T12:00"})
# Event(when=datetime.datetime(2020, 1, 1, 12, 0))
```
A `NewType` is followed to the type it wraps, so a field typed `UserId = NewType("UserId", int)` validates as an `int`.
## Key facets on fields
[Section titled “Key facets on fields”](#key-facets-on-fields)
A plain type says what a field *is*. To give a field a key facet, redact it, accept it under other names, forbid it, group it, add `Key` to its `Annotated` metadata. It sits next to any value validators; the two do not interfere.
```python
from dataclasses import dataclass
from typing import Annotated
from probatio import DataclassSchema, Key, Length
@dataclass
class Account:
name: str
password: Annotated[
str, Key(secret=True), Length(min=8)
] # redacted, length-checked
user_name: Annotated[str, Key(alias=["user-name", "userName"])] = (
"" # accept aliases
)
is_admin: Annotated[bool, Key(forbidden=True)] = (
False # reject if the caller sends it
)
schema = DataclassSchema(Account)
schema({"name": "a", "password": "secret123", "user-name": "frenck"})
# Account(name='a', password='secret123', user_name='frenck', is_admin=False)
```
`Key(secret=True)` redacts the field’s value in validation errors. It shows up when an error renders the offending value: the message stays useful, the secret does not leak.
```python
from probatio import MultipleInvalid
from probatio.humanize import humanize_error
data = {"name": "a", "password": "short", "user-name": "frenck"}
try:
schema(data)
except MultipleInvalid as err:
print(humanize_error(data, err))
# length of value must be at least 8 at 'password'. Got
```
`Key(alias=[...])` accepts the field under alternate input names (a bare string works for one), emitting it under the field name; `accept_canonical=False` makes it a strict rename. `Key(inclusive="grp")` / `Key(exclusive="grp")` group fields the way the [dict form](/guides/dict-schemas-and-markers/) does. `Key(required=True)` (or `required=False`) overrides the presence the field’s default would imply. `required=True` on a field that has a default matches the dict form’s `Required(key, default=...)`: the key is guaranteed in the result, filled from the default when the input omits it. Without a default there is nothing to fall back on, so the key must be supplied. `required=False` marks a field optional, so on a dataclass it still needs a default for the constructor to fall back on; without one it raises `SchemaError`.
`Key` is a field-only spec; a plain dict schema keeps using the markers directly (`{Secret("password"): str}`, `{Alias("user_name", "user-name"): str}`). It works the same on a [TypedDict](/guides/typeddict/).
`forbidden` and `remove` make a key contribute nothing to the result, so on a dataclass (whose constructor needs a value for every field) such a field must have a default; without one it raises `SchemaError`. A TypedDict constructs nothing, so there they need no default.
One boundary to know, on which defaults pass through the schema:
* An `optional` field, a `required` field that has one, or a selected `exclusive` member, carries its default through the schema, so it is validated and coerced like input.
* A field the schema keeps out of the input (a `forbidden` field, a `remove` field, an unselected `exclusive` member) takes the dataclass’s own default exactly as declared: no validation, no coercion.
A nested dataclass field can default to a built instance, the idiomatic “always-present, defensive empty object”: `inner: Inner = field(default_factory=Inner)`. When the key is absent, the `Inner()` fills in as-is; it is not re-validated as a mapping. A caller may also hand in an already-built instance for such a field, and it passes through untouched; only a mapping is validated and constructed.
The schema validates input, and that second group’s values are not input, so write an already-typed default for such a field; a wrong-typed one is a type error your type-checker flags. See [ADR-013](https://github.com/frenck/probatio/blob/main/adr/013-markers-on-annotated-fields.md) for the model.
## The functional form and the helper
[Section titled “The functional form and the helper”](#the-functional-form-and-the-helper)
`create_dataclass_schema(dataclass_type, additional_constraints=None)` builds the same schema without the class wrapper, and `is_dataclass` is the standard-library check, re-exported so you do not have to import it separately.
Both `DataclassSchema` and `create_dataclass_schema` also take keyword-only `required` and `extra` arguments, passed straight through to `Schema`, so you can, for example, accept and keep unknown keys with `extra=ALLOW_EXTRA`.
```python
from dataclasses import dataclass
from probatio import create_dataclass_schema, is_dataclass
@dataclass
class User:
name: str
is_dataclass(User) # True
create_dataclass_schema(User)({"name": "ada"}) # User(name='ada')
```
## A `from_dict` mixin
[Section titled “A from_dict mixin”](#a-from_dict-mixin)
Parsing an API payload into a dataclass tree is the common shape: a `DataclassSchema` built once, then called, usually behind a `from_dict` classmethod. `SchemaMixin` bundles that into the class. Inherit it, pass the extra-key policy as a class argument, and the dataclass gets a cached, validating `from_dict`, with no separate module-level schema and no hand-written classmethod.
```python
from dataclasses import dataclass
from probatio import REMOVE_EXTRA, SchemaMixin
@dataclass
class Server(SchemaMixin, extra=REMOVE_EXTRA):
host: str = ""
port: int = 80
Server.from_dict({"host": "nas", "port": 8080, "unmodeled": 1})
# Server(host='nas', port=8080)
```
`from_dict` returns an instance of the class, typed as such, so your editor and type-checker know the result. The `DataclassSchema` is built on the first call and reused after, and `extra` is inherited: a subclass that sets none keeps the parent’s policy. The class stays an ordinary dataclass; the mixin adds `from_dict` and records `extra`, and leaves the fields alone. A `TypedDict` cannot carry methods, so it keeps using `TypedDictSchema` directly.
`from_dict` is the validation boundary, not the class. Coming from pydantic, this is the one thing to know: `SchemaMixin` does not make the dataclass validate itself. Constructing one directly (`Server(host="nas", port="no")`) runs no validation, the same as any plain dataclass, and a type-checker already flags the wrong type there. Validate untrusted input at the edge with `from_dict`, and treat an instance you already hold as trusted. The schema stays separate from the data, so the same dataclass keeps working with probatio nowhere in sight.
Note
Because a default flows back through the schema when its key is absent, a `Coerce` on a defaulted field must be idempotent: it has to take its own already-final default (a `datetime`, a `None`) and return it unchanged, rather than convert it a second time.
## Extra keys, all the way down
[Section titled “Extra keys, all the way down”](#extra-keys-all-the-way-down)
`extra` propagates. Set `extra=REMOVE_EXTRA` (or `ALLOW_EXTRA`) and the policy applies at every level: nested dataclasses, nested TypedDicts, and dataclasses inside a `list`, `dict`, `tuple`, or union all follow it. This is the case for parsing a third-party API response into a tree of dataclasses while dropping the fields you do not model, one flag, not a wrapper per level.
```python
from dataclasses import dataclass, field
from probatio import DataclassSchema, REMOVE_EXTRA
@dataclass
class Server:
host: str = ""
@dataclass
class Config:
server: Server = field(default_factory=Server)
data = {"server": {"host": "nas", "unmodeled": 9}, "toplevel_junk": 1}
DataclassSchema(Config, extra=REMOVE_EXTRA)(data) # Config(server=Server(host='nas'))
```
To pin a different policy for one field’s subtree, put `Key(extra=...)` in its `Annotated` metadata. It overrides the inherited policy for that field only, so you can keep one noisy sub-object loose while the rest stays strict, or the reverse:
```python
from dataclasses import dataclass, field
from typing import Annotated
from probatio import DataclassSchema, Key, PREVENT_EXTRA
@dataclass
class Strict:
id: int = 0
@dataclass
class Doc:
meta: Annotated[Strict, Key(extra=PREVENT_EXTRA)] = field(default_factory=Strict)
```
The schema is loose, but `meta` stays strict and rejects the unknown key:
```python
DataclassSchema(Doc, extra=REMOVE_EXTRA)({"meta": {"id": 1, "typo": 2}})
# MultipleInvalid: not a valid option at 'meta.typo'
```
The default (`PREVENT_EXTRA`) is unchanged: an unknown key, at any level, is rejected. `Key(extra=...)` on a plain (leaf) field has nothing to apply to and is ignored.
## Coercing a field’s type
[Section titled “Coercing a field’s type”](#coercing-a-fields-type)
A field annotated with `datetime` validates by `isinstance`, so a string from JSON or YAML is rejected. That is the safe default: probatio validates, it does not silently transform. When you do want a field coerced, say so on the field with an `Annotated` hint, and the coercion is scoped to exactly that field:
```python
from dataclasses import dataclass
from datetime import datetime
from typing import Annotated
from probatio import DataclassSchema, Coerce
@dataclass
class Event:
when: Annotated[datetime, Coerce(datetime.fromisoformat)]
DataclassSchema(Event)(
{"when": "2020-01-01T12:00"}
) # Event(when=datetime(2020, 1, 1, 12, 0))
```
The `Coerce` runs first and the `datetime` type confirms the result, so the field stays honestly typed (see the [annotation mapping table](#annotations-drive-the-validators)). This works the same way in a nested dataclass field, and `additional_constraints` still layers on top. The hand-written `Schema(datetime)` path is never affected.
It holds for a nullable or union field too: the coercer runs before the `X | None` (or `X | Y`) base confirms the result, so `Annotated[int | None, Coerce(...)]` coerces a raw string and `Annotated[datetime | None, Coerce(FromEpoch())]` turns a timestamp into a `datetime`, keeping the field’s real `int | None` / `datetime | None` type instead of falling back to `Any`.
Reach for a built-in coercer before writing a lambda. `AsTimedelta` reads a count of seconds, a `H:MM:SS` string, or an ISO 8601 duration into a `timedelta`, and `Maybe(Coerce(float))` coerces a value while letting `None` through. Both compose straight onto a field, so a “seconds or a `timedelta`” field is `Annotated[timedelta, AsTimedelta()]` and an “optional, coerce if present” field is `Annotated[float | None, Maybe(Coerce(float))]`, with no coercer of your own to write or test:
```python
from dataclasses import dataclass
from datetime import timedelta
from typing import Annotated
from probatio import AsTimedelta, Coerce, DataclassSchema, Maybe
@dataclass
class Job:
timeout: Annotated[timedelta, AsTimedelta()] = timedelta()
ratio: Annotated[float | None, Maybe(Coerce(float))] = None
DataclassSchema(Job)({"timeout": 90, "ratio": "0.5"})
# Job(timeout=datetime.timedelta(seconds=90), ratio=0.5)
```
The `As*` coercers return their native type unchanged (`AsTimedelta` on a `timedelta`, `AsDatetime` on a `datetime`), so they are idempotent in the sense above: one sits on a defaulted field without re-converting the default when it flows back through the schema.
## Recursive dataclasses
[Section titled “Recursive dataclasses”](#recursive-dataclasses)
A dataclass whose field refers back to itself (a tree node, a linked list) is supported: the field validates against the same schema, all the way down, and each level is constructed. Mutually recursive dataclasses (two types that point at each other) work the same way.
```python
from __future__ import annotations
from dataclasses import dataclass, field
from probatio import DataclassSchema
@dataclass
class Tree:
name: str
children: list[Tree] = field(default_factory=list)
DataclassSchema(Tree)({"name": "root", "children": [{"name": "leaf"}]})
# Tree(name='root', children=[Tree(name='leaf', children=[])])
```
Recursion follows the data, with the same depth guard as `Self`: cyclic or pathologically deep input raises a clean `Invalid`, never a `RecursionError`. A recursive dataclass level does more work than a bare `Self` level, so it bottoms out sooner; raise the limit with `sys.setrecursionlimit()` if you genuinely need to go deeper.
## Discriminated unions
[Section titled “Discriminated unions”](#discriminated-unions)
A field that is a union of dataclasses sharing a literal tag field becomes a discriminated union: the tag picks the one branch to validate, rather than trying each member in order. The branch is chosen by the tag, so a failure reports that branch’s error instead of a vague “matched no member”.
```python
from __future__ import annotations
from dataclasses import dataclass
from typing import Literal
from probatio import DataclassSchema
@dataclass
class Circle:
kind: Literal["circle"]
radius: int
@dataclass
class Square:
kind: Literal["square"]
side: int
@dataclass
class Shape:
shape: Circle | Square
DataclassSchema(Shape)({"shape": {"kind": "square", "side": 3}})
# Shape(shape=Square(kind='square', side=3))
```
The tag field must be a single-value `Literal` present on every member, with a distinct value per member. Without one (or if a member is not a dataclass), the union stays an ordinary “try each” `Any`. An unknown tag value falls back to trying every member, so it still fails cleanly rather than silently.
## Speed
[Section titled “Speed”](#speed)
A `DataclassSchema` is the clearest case for the compiled engine. When a schema compiles, it fuses field validation and object construction into a single generated function with no intermediate dict, which is most of why a hot dataclass schema validates several times faster than the interpreted one and lands close to a pure deserializer like mashumaro while still checking every field. This happens on its own once the schema proves hot; see [Compiled schemas](/guides/compiled-schemas/) to opt in eagerly or read the trade-offs.
## Trusted construction, without validation
[Section titled “Trusted construction, without validation”](#trusted-construction-without-validation)
Probatio is a validation library, but there are spots where validation is not needed: the input is your own data round-tripping back in, or it was already validated upstream. For those, `DataclassSchema.construct` builds the instance from trusted data and skips validation entirely.
```python
from dataclasses import dataclass
from probatio import DataclassSchema
@dataclass
class Point:
x: int
y: int
@dataclass
class Line:
start: Point
end: Point
schema = DataclassSchema(Line)
# Trusted input: build straight through, no type checks, no coercion.
schema.construct({"start": {"x": 0, "y": 0}, "end": {"x": 3, "y": 4}})
# Line(start=Point(x=0, y=0), end=Point(x=3, y=4))
```
It reads each field straight from the dict, recursing into nested dataclasses, lists of them, `Optional` fields, and a single dataclass inside a union (like `Comment | str`), and filling defaults, then constructs. With no checks it is faster than validating, fast enough to beat dedicated deserializers like mashumaro on the [performance page](/reference/performance/), because it is a purpose-built constructor and pure Python.
The catch is in the name: it trusts you. A wrong type lands in the instance unchecked, and it does not convert: a field typed `datetime` will hold whatever the dict held (a string stays a string), where validation would coerce it. For input that still needs decoding, validate.
Caution
Use `construct` only for input you already know is correct. For anything from outside your boundary, call the schema (`schema(data)`), which validates. A shape the fast path does not handle (a recursive dataclass, for instance) falls back to validating, so `construct` always returns a correct instance for trusted input, just not always faster. `TypedDictSchema.construct` does the same, returning the trusted dict unchanged.
## Limits
[Section titled “Limits”](#limits)
A field with `init=False` is left out of the schema, since it is not a constructor argument; it keeps whatever its default or `__post_init__` gives it. An `InitVar` goes the other way: it is a constructor argument, so it becomes a schema key, validated against its annotation like any field, and it feeds `__post_init__` as usual.
```python
from dataclasses import InitVar, dataclass, field
from probatio import DataclassSchema
@dataclass
class Rect:
width: int
height: int
scale: InitVar[int] = 1
area: int = field(init=False, default=0)
def __post_init__(self, scale: int) -> None:
self.area = self.width * self.height * scale
DataclassSchema(Rect)({"width": 2, "height": 3, "scale": 10})
# Rect(width=2, height=3, area=60)
```
The value is not coerced between container types: a list stays a list even where the annotation says `tuple`.
# Dict schemas and markers
> Required and optional keys, defaults, extra-key policy, and key groups.
A `dict` schema validates a mapping: each key in the schema describes a key that may appear in the data, and its value is the schema for that key’s value. Markers are special keys that carry intent, such as “this key is required” or “fill this in when it is absent.”
## Required and optional keys
[Section titled “Required and optional keys”](#required-and-optional-keys)
By default a literal key is allowed but not required, and an unknown key is rejected. `Required` and `Optional` make the intent explicit:
```python
from probatio import Schema, Required, Optional
schema = Schema(
{
Required("name"): str,
Optional("nickname"): str,
}
)
schema({"name": "Frenck"}) # {'name': 'Frenck'}
```
Leaving out a required key fails:
```python
from probatio import Schema, Required, Invalid
schema = Schema({Required("name"): str})
try:
schema({})
except Invalid as err:
print(err) # required key not provided at 'name'
```
A marker compares and hashes by its key, so `Required("name")` and `"name"` are the same dict key. You annotate a key without changing how it is matched.
## Defaults
[Section titled “Defaults”](#defaults)
`Optional` (and `Required`) take a `default`. When the key is absent, the default fills it in. A callable default is called each time, which is how you get a fresh list or dict per validation rather than a shared one:
```python
from probatio import Schema, Optional
schema = Schema(
{
Optional("port", default=8080): int,
Optional("tags", default=list): [str],
}
)
schema({}) # {'port': 8080, 'tags': []}
```
A callable default may return `UNDEFINED` to decline: the key is then left absent, exactly as if it had no default. This is useful for a default that depends on runtime context (the active platform, which plugins are loaded), where sometimes no value should be supplied:
```python
from probatio import Schema, Optional, UNDEFINED
context = {"fast": True}
def speed_default():
return 80 if context["fast"] else UNDEFINED
schema = Schema({Optional("speed", default=speed_default): int})
print(schema({})) # {'speed': 80}
context["fast"] = False
print(schema({})) # {}
```
For a `Required` key a declining default leaves the key missing, so it is reported as a missing required key.
## The extra-key policy
[Section titled “The extra-key policy”](#the-extra-key-policy)
What happens to a key the schema does not mention is controlled by the `extra` argument to `Schema`. By default unknown keys are rejected:
```python
from probatio import Schema, Invalid
schema = Schema({"name": str})
try:
schema({"name": "app", "debug": True})
except Invalid as err:
print(err) # not a valid option at 'debug'
```
When a rejected key looks like a misspelling of one the schema knows, the error points at the closest matches and carries them on the error as `ExtraKeysInvalid.candidates`, so a tool can show “did you mean …?” without recomputing it:
```python
from probatio import Schema, Invalid
schema = Schema({"name": str, "email": str})
try:
schema({"nmae": "app"})
except Invalid as err:
error = err.errors[0]
print(error) # not a valid option, did you mean 'name'? at 'nmae'
print(error.candidates) # ['name']
```
The bar is a real misspelling, not a family resemblance. Two keys that share a prefix and differ after it (`device_class` against `device_info`) are different fields, not a typo of one another, so no suggestion is offered: a hint pointing at the wrong field costs more than no hint at all.
The three policies are `PREVENT_EXTRA` (the default), `ALLOW_EXTRA` (keep unknown keys untouched), and `REMOVE_EXTRA` (drop them from the result):
```python
from probatio import Schema, ALLOW_EXTRA, REMOVE_EXTRA
Schema({"name": str}, extra=ALLOW_EXTRA)(
{"name": "app", "x": 1}
) # {'name': 'app', 'x': 1}
Schema({"name": str}, extra=REMOVE_EXTRA)({"name": "app", "x": 1}) # {'name': 'app'}
```
For finer control than a whole-schema policy, `Extra` is a catch-all key: `{Extra: validator}` validates every otherwise-unmatched key against `validator`. Use `{Extra: object}` to allow anything.
```python
from probatio import Schema, Extra
schema = Schema({"name": str, Extra: int})
schema({"name": "app", "a": 1, "b": 2}) # {'name': 'app', 'a': 1, 'b': 2}
```
## Removing keys
[Section titled “Removing keys”](#removing-keys)
`Remove` drops matching keys from the output. The value is still validated; only a value that validates is dropped, so `Remove` does not hide a type error.
```python
from probatio import Schema, Remove
schema = Schema({"keep": int, Remove("drop"): str})
schema({"keep": 1, "drop": "gone"}) # {'keep': 1}
```
## Forbidding keys
[Section titled “Forbidding keys”](#forbidding-keys)
`Forbidden` is the inverse of `Required`: the key must *not* be present. If it appears, validation fails with “key not allowed”. The mapped value is never looked at, so the idiom is to map it to `object`.
```python
from probatio import Schema, Required, Forbidden
schema = Schema({Required("id"): int, Forbidden("password"): object})
schema({"id": 1}) # {'id': 1}
```
A present forbidden key fails:
```python
from probatio import Schema, Forbidden
schema = Schema({Forbidden("password"): object})
schema({"password": "secret"}) # key not allowed at 'password'
```
It composes with `extend`, so a base schema can be tightened to forbid a key it used to allow.
## Aliasing keys
[Section titled “Aliasing keys”](#aliasing-keys)
`Alias` accepts a value under more than one name and stores it under a single canonical name. It is the answer when a source spells a key differently from your target: a kebab-case config key for a snake_case field, or a reserved word like `class`. The first argument is the canonical name (used in the output), and the rest are aliases accepted in the input.
```python
from probatio import Schema, Alias
schema = Schema({Alias("user_name", "user-name", "userName"): str})
schema({"user-name": "ada"}) # {'user_name': 'ada'}
schema({"userName": "ada"}) # {'user_name': 'ada'}
schema({"user_name": "ada"}) # {'user_name': 'ada'}
```
The canonical name is accepted as an input name too, and leads the search, so it wins when it and an alias both appear. Among aliases, the first one in the order you listed wins. Pass `accept_canonical=False` for a strict rename that accepts only the aliases:
```python
from probatio import Schema, Alias
schema = Schema({Alias("name", "alias", accept_canonical=False): str})
schema({"alias": "ada"}) # {'name': 'ada'}
schema({"name": "ada"}) # {} (the canonical name is not an input name here)
```
Note what the second call does: the canonical key is still recognized, so it is not reported as an unknown key, but its value is dropped from the output. An actually unknown key would raise.
An aliased key is optional by default (its `default` applies when absent under every name); pass `required=True` to demand one of its names. An alias that names another key in the schema, or that two keys share, is rejected at build time, so an ambiguous schema fails fast rather than at validation. So is a name that is not a literal: a type or a validator (`Alias(Any("a", "b"), "x")`) has no single spelling to look up in the input or to emit under.
## Redacting secret values
[Section titled “Redacting secret values”](#redacting-secret-values)
`Secret` marks a key whose value is a credential, so a validation failure under it is redacted: the error still reports the path and the reason, but shows `` instead of the offending value (in `Invalid` rendering and `humanize_error`, see [error handling](/guides/error-handling/)). The value passes through validation unchanged; `Secret` marks the key, it does not transform the value.
```python
from probatio import Schema, Required, Secret, MultipleInvalid
from probatio.humanize import humanize_error
schema = Schema({Required(Secret("password")): int})
data = {"password": "hunter2"}
try:
schema(data)
except MultipleInvalid as err:
print(humanize_error(data, err))
# expected int at 'password'. Got
```
Secrecy is an independent facet, so it composes with the presence markers by nesting: `Optional(Secret("password"))` (equivalently `Secret(Optional("password"))`) is an optional, redacted key. Order does not matter. The marker names a concrete key, so `Secret(str)` (a type key) is refused at build time.
Redaction covers validation errors only, the output a schema controls, not values you log yourself elsewhere. The `secret` flag also rides on each error (`err.errors[0].secret`, and in `as_dict()`), so a consumer building its own output can redact the same values. The [security guide](/project/security/) covers the threat model.
## Type and callable keys
[Section titled “Type and callable keys”](#type-and-callable-keys)
A key does not have to be a literal. A type key validates *every* key of that type, which is how you describe an open mapping like “string keys, integer values”:
```python
from probatio import Schema
schema = Schema({str: int})
schema({"a": 1, "b": 2}) # {'a': 1, 'b': 2}
```
A key that fails the type is reported against that key, the same way voluptuous does:
```python
from probatio import Schema, Invalid
schema = Schema({str: int})
try:
schema({1: 2})
except Invalid as err:
print(err) # expected str at '[1]'
```
## Co-dependent and exclusive keys
[Section titled “Co-dependent and exclusive keys”](#co-dependent-and-exclusive-keys)
`Inclusive` ties keys into a group that must appear together, all or none. `Exclusive` ties keys into a group where at most one may appear. Both take the group name as their second argument.
```python
from probatio import Schema, Inclusive
schema = Schema(
{
Inclusive("lat", "coords"): float,
Inclusive("lon", "coords"): float,
}
)
schema({"lat": 52.1, "lon": 5.1}) # {'lat': 52.1, 'lon': 5.1}
```
Providing only part of an inclusive group fails, and the error points at the group, not a single key:
```python
from probatio import Schema, Inclusive, Invalid
schema = Schema(
{
Inclusive("lat", "coords"): float,
Inclusive("lon", "coords"): float,
}
)
try:
schema({"lat": 52.1})
except Invalid as err:
print(
err
) # some but not all values in the same group of inclusion 'coords' at ''
```
Since no single key is at fault, a group error reports a synthetic path segment named after the group, in angle brackets: `` here, `` below.
An exclusive group is optional by default (none of its keys is fine). Two flags change what an empty group does. `required=True` makes the group demand exactly one key:
```python
from probatio import Schema, Exclusive
schema = Schema(
{
Exclusive("token", "auth", required=True): str,
Exclusive("password", "auth", required=True): str,
}
)
schema({}) # exactly one of ['token', 'password'] is required at ''
```
A `default` fills its member in when the group is empty (and wins over `required`, since a default already satisfies the group):
```python
from probatio import Schema, Exclusive
schema = Schema(
{
Exclusive("mode", "m", default="auto"): str,
Exclusive("custom", "m"): str,
}
)
schema({}) # {'mode': 'auto'}
```
Both are group-level: set either on any member and it governs the whole group.
A group member does not have to be a literal name. The first argument is a key schema like any other, so it can be an `Any` of names, a type, or any validator, and the member counts as present when some key in the data matches it. That makes `Any("hours", "minutes")` one member, satisfied by either name:
```python
from probatio import Any, Inclusive, Schema
schema = Schema(
{
Inclusive(Any("hours", "minutes"), "duration"): int,
Inclusive("name", "duration"): str,
}
)
schema({"hours": 1, "name": "tea"}) # {'hours': 1, 'name': 'tea'}
```
Both of the member’s names may appear. They still count as the one member, so they neither stand in for the rest of the group nor collide with each other in an exclusive group:
```python
from probatio import Any, Inclusive, Schema
schema = Schema(
{
Inclusive(Any("hours", "minutes"), "duration"): int,
Inclusive("name", "duration"): str,
}
)
value = {"hours": 1, "minutes": 5, "name": "tea"}
schema(value) # {'hours': 1, 'minutes': 5, 'name': 'tea'}
```
Either side alone still fails the group. voluptuous reads a member like this as never present, which leaves the group impossible to satisfy; see the [intentional deviations](/reference/compatibility-matrix/#intentional-deviations).
## Marking required across a whole schema
[Section titled “Marking required across a whole schema”](#marking-required-across-a-whole-schema)
Instead of wrapping every key in `Required`, pass `required=True` to `Schema` to make every key required by default; individual `Optional` keys still opt out.
```python
from probatio import Schema, Optional
schema = Schema({"a": int, Optional("b"): int}, required=True)
schema({"a": 1}) # {'a': 1}
```
# Error handling
> Invalid, MultipleInvalid, error paths, humanized messages, and the structured layer.
When data does not match, Probatio raises. This page covers what it raises, how to read the path to the offending value, how to collect every error at once, and how to turn an error into something a person or a machine can use.
## One base: Invalid
[Section titled “One base: Invalid”](#one-base-invalid)
Everything Probatio raises for bad data is an `Invalid` or a subclass. Catching `Invalid` catches them all, so you rarely need to name a specific subclass unless you want to branch on the kind of failure.
```python
from probatio import Schema, Invalid
schema = Schema(int)
try:
schema("nope")
except Invalid as err:
print(err) # expected int
```
A broken schema *definition* raises `SchemaError` instead: you wrote the schema wrong, and no input would make it right. The two never overlap: `SchemaError` means fix the code, `Invalid` means reject the data.
## The path to the value
[Section titled “The path to the value”](#the-path-to-the-value)
An error knows where in the data it happened. `str(error)` appends that path as a dotted trail (mapping keys joined with dots, sequence indices as `[n]`), and `error.path` is the list of keys and indices to walk:
```python
from probatio import Schema, Invalid
schema = Schema({"server": {"ports": [int]}})
try:
schema({"server": {"ports": [80, "nope"]}})
except Invalid as err:
print(err) # expected int at 'server.ports[1]'
print(err.path) # ['server', 'ports', 1]
```
`error.path` is the machine-readable form; follow it into the original data to find the exact value that failed.
## Many errors at once
[Section titled “Many errors at once”](#many-errors-at-once)
A dict schema does not stop at the first failing key (and a list schema not at the first failing element). It collects every failure into a `MultipleInvalid`, which is itself an `Invalid`. Its `errors` list holds the individual failures:
```python
from probatio import Schema, MultipleInvalid
schema = Schema({"a": int, "b": int})
try:
schema({"a": "x", "b": "y"})
except MultipleInvalid as err:
print(len(err.errors)) # 2
for sub in err.errors:
print(sub.path) # ['a'] then ['b']
```
For convenience, a `MultipleInvalid` proxies its first error, so `error.msg` and `error.path` read the first failure without reaching into `errors`.
## Readable messages
[Section titled “Readable messages”](#readable-messages)
`humanize_error` renders an error against the data, naming the value that failed. It lives in `probatio.humanize`:
```python
from probatio import Schema, Invalid
from probatio.humanize import humanize_error
data = {"port": "nope"}
schema = Schema({"port": int})
try:
schema(data)
except Invalid as err:
print(humanize_error(data, err)) # expected int at 'port'. Got 'nope'
```
`validate_with_humanized_errors(data, schema)` (also in `probatio.humanize`) does both steps: it validates and, on failure, raises `Error` (importable from `probatio` or `probatio.error`) carrying the humanized message. The [errors reference](/reference/errors/) has the class.
## Catching a specific kind
[Section titled “Catching a specific kind”](#catching-a-specific-kind)
The subclasses let you branch on what went wrong. They mirror voluptuous, so `TypeInvalid`, `RangeInvalid`, `LengthInvalid`, `CoerceInvalid`, and the rest are all there. A schema always raises `MultipleInvalid` at the top, so you branch on the individual errors inside it:
```python
from probatio import Schema, Range, MultipleInvalid
from probatio.error import RangeInvalid
schema = Schema(Range(min=0, max=10))
try:
schema(99)
except MultipleInvalid as err:
first = err.errors[0]
print(isinstance(first, RangeInvalid)) # True
print(first.error_message) # value must be at most 10
```
`error_message` is the bare message without the path; `msg` is the same text. The [errors reference](/reference/errors/) lists the whole hierarchy.
## The structured layer
[Section titled “The structured layer”](#the-structured-layer)
On top of the voluptuous-compatible fields, every error carries a structured, machine-readable layer: a stable `code`, a `context` dict, and the pair that identifies the message itself: `translation_key` (a stable key naming the sentence, like `length_min`) and `placeholders` (the raw values the message interpolates, like `{"min": 10}`). The built-in validators fill all of them; the [translation keys reference](/reference/translation-keys/) lists every key. `as_dict()` serializes the whole layer, which is handy for an API that returns validation errors as JSON.
```python
from probatio import Schema, Invalid
schema = Schema({"port": int})
try:
schema({"port": "nope"})
except Invalid as err:
first = err.errors[0]
print(first.code) # type
print(first.as_dict()["path"]) # ['port']
```
The legacy fields (`msg`, `path`, `error_message`) are untouched by this layer, so nothing about the voluptuous-compatible behavior changes; the structured data is purely additive.
To override a validator’s message, or to build your own output (or translations) from this layer, see [Custom error messages](/guides/custom-error-messages/).
# Field lists
> Render a Probatio schema as the flat field list config frontends and tool exporters consume.
Some tools do not want a schema document, they want a flat list of fields to render as a form. `to_field_list(schema)` produces exactly that. It is exported from the top level: `from probatio import to_field_list`.
Unlike [JSON Schema](/guides/json-schema/) and [OpenAPI](/guides/openapi/), this is not a published standard. It is the shape [voluptuous-serialize](https://github.com/home-assistant-libs/voluptuous-serialize) emits, an internal format from the Home Assistant ecosystem, where the config-flow frontend turns a schema into a form. Probatio matches it so anything built against voluptuous-serialize keeps working on a Probatio schema, apart from [a couple of deliberate differences](#where-probatio-differs). That is who this codec is for: Home Assistant and the libraries around it. If you are not in that world, reach for JSON Schema or OpenAPI instead.
## The field list
[Section titled “The field list”](#the-field-list)
A mapping becomes a list of field dicts, one per key, carrying the type, the name, and whether it is required:
```python
from probatio import Schema, Required, Optional, In, to_field_list
schema = Schema(
{
Required("name"): str,
Optional("port", default=8080): int,
Required("mode"): In(["auto", "manual"]),
}
)
to_field_list(schema)
# [{'type': 'string', 'name': 'name', 'required': True}, {'type': 'integer', 'name': 'port', 'required': False, 'optional': True, 'default': 8080}, {'type': 'select', 'options': [('auto', 'auto'), ('manual', 'manual')], 'name': 'mode', 'required': True}]
```
Each field carries what the frontend needs to render it: the `type`, the `name`, `required`, an `optional` flag and a `default` when present, and bounds (such as `valueMin`/`valueMax` for a `Range`) where the validator implies them. An `In(...)` becomes a `select` field with its `options` as (value, label) pairs, which is how a config-flow form renders a dropdown; pass `In` a mapping to give each value its own label.
## Where Probatio differs
[Section titled “Where Probatio differs”](#where-probatio-differs)
voluptuous-serialize raises on a `Match`, which takes down the whole field list. That hurts most where the regex is a detail of a field the rest of the schema already described:
```python
from probatio import All, Match, Required, Schema, to_field_list
schema = Schema({Required("pin"): All(str, Match(r"^\d{6}$"))})
to_field_list(schema)
# [{'type': 'string', 'name': 'pin', 'required': True}]
```
A text regex only ever accepts a string, so Probatio says `string` and moves on. The pattern itself is dropped: the field list has no key to carry one. Use [JSON Schema](/guides/json-schema/) or [OpenAPI](/guides/openapi/) if you need the pattern to survive; both emit it. A `bytes` pattern is the exception. It rejects every string a form can submit, so it still raises rather than describing a field nothing typed into it could satisfy.
The other difference is nullability: `Any(X, None)` renders as a nullable field, where the oracle only recognizes the shape when `None` comes first. Both orders mean the same thing, so both serialize.
Bounds inside an `All` are the third. Every member of an `All` has to accept the value, so repeated bounds intersect: the narrowest lower bound and the narrowest upper bound reach the field, in whatever order the members are written. voluptuous-serialize merges members with `dict.update`, so there the last one wins and a wider bound can overwrite a narrower one:
```python
from probatio import All, Length, NonEmpty, Range, Schema, to_field_list
to_field_list(Schema(All(Length(min=5), NonEmpty())))
# {'lengthMin': 5} the oracle says lengthMin 1
to_field_list(Schema(All(Range(min=5), Range(min=1))))
# {'valueMin': 5} the oracle says valueMin 1
```
A field that advertises a bound the schema rejects sends the user back a form error for a value the form invited, which is why Probatio narrows instead.
`Clamp` is the exception, because it bends a value into range rather than rejecting it. Its bounds describe what comes out, not what may be submitted, so a `Clamp` replaces the bounds it follows instead of narrowing them, exactly as the oracle does.
## A custom-serializer hook
[Section titled “A custom-serializer hook”](#a-custom-serializer-hook)
`to_field_list` and `to_openapi` take a `custom_serializer` hook, called first for each node. It returns a dict to override that node, or the `UNSUPPORTED` sentinel to defer to the default handling. `UNSUPPORTED` is exported from the top level and prints as itself:
```python
from probatio import UNSUPPORTED
UNSUPPORTED # UNSUPPORTED
```
The hook is a plain function taking one schema node. Return a dict to render that node yourself; return `UNSUPPORTED` for everything else. Here a `Port` renders as a dedicated `port` field type instead of the default bounded integer:
```python
from probatio import Schema, Required, Port, to_field_list, UNSUPPORTED
def render_port(node):
if isinstance(node, Port):
return {"type": "port"}
return UNSUPPORTED
schema = Schema({Required("host"): str, Required("port"): Port()})
to_field_list(schema, custom_serializer=render_port)
# [{'type': 'string', 'name': 'host', 'required': True}, {'type': 'port', 'name': 'port', 'required': True}]
```
The hook overrides the value part only; `to_field_list` still adds the key facets (`name`, `required`, `default`) around whatever the hook returns.
Tip
Return `UNSUPPORTED` from your hook for the nodes you do not care about. That keeps your custom logic small: handle the one or two constructs you need, defer everything else.
# JSON Schema
> Render a Probatio schema as JSON Schema, build one back, and stay safe doing it.
A Probatio schema is Python data. JSON Schema is the lingua franca other tools speak. The two codecs translate between them, for the constructs that map cleanly. They are exported from the top level, so `from probatio import to_json_schema, from_json_schema` is all you need. That unlocks the places JSON Schema already lives: editor validation of config files, form generation in a frontend, and LLM tool definitions (worked through in the [LLM tool recipe](/recipes/llm-tools/)). Decoded input is treated as untrusted by default; the [guards](#untrusted-input-is-the-default-assumption) are covered at the bottom of this page.
The same decoder backs OpenAPI (see [OpenAPI](/guides/openapi/)), and a third codec renders the flat shape config frontends consume (see [Field lists](/guides/field-lists/)).
## Both directions
[Section titled “Both directions”](#both-directions)
`to_json_schema(schema)` renders a schema as a JSON Schema dictionary. `from_json_schema(dict)` is the inverse: it builds a `Schema` back. Together they round trip the parts that have a clean mapping.
```python
from probatio import Schema, Required, Optional, to_json_schema
schema = Schema({Required("name"): str, Optional("port", default=8080): int})
to_json_schema(schema)
# {'type': 'object', 'properties': {'name': {'type': 'string'}, 'port': {'type': 'integer', 'default': 8080}}, 'additionalProperties': False, 'required': ['name']}
```
Going the other way, a JSON Schema becomes a working validator. `required` controls which keys must be present, `minimum` becomes a `Range`, and so on:
```python
from probatio import from_json_schema
document = {
"type": "object",
"properties": {
"name": {"type": "string"},
"age": {"type": "integer", "minimum": 0},
},
"required": ["name"],
}
schema = from_json_schema(document)
schema({"name": "Ada", "age": 37}) # {'name': 'Ada', 'age': 37}
```
A small round trip stays intact. Render a schema, build it back, validate:
```python
from probatio import Schema, Required, to_json_schema, from_json_schema
original = Schema({Required("name"): str, Required("age"): int})
rebuilt = from_json_schema(to_json_schema(original))
rebuilt({"name": "Ada", "age": 37}) # {'name': 'Ada', 'age': 37}
```
Note
The mapping is not lossless. `to_json_schema` renders what JSON Schema can express and turns a construct it cannot into an open schema (`{}`). Pass `strict=True` to make that a `SchemaError` instead, so a lossy conversion is caught rather than accepted silently. `from_json_schema` ignores purely descriptive keywords but refuses a restrictive keyword it cannot honor (see below). Treat a round trip as “the validatable shape survives”, not “byte for byte identical”.
## Widening the encoder, and overriding it
[Section titled “Widening the encoder, and overriding it”](#widening-the-encoder-and-overriding-it)
By default `to_json_schema` never rejects an input the schema accepts: where it cannot express a construct exactly it widens (an open `{}`, or a looser keyword), so the emitted schema stays a superset of what Probatio validates. Two options tune that:
* `strict=True` raises `SchemaError` for a construct with no JSON Schema form (an unknown validator, a `Coerce` with a non-type target, an `enum` member with no JSON representation), instead of widening to `{}`.
* `custom_serializer` is called first for each node and may return a dict to override the rendering, or the `UNSUPPORTED` sentinel to defer to the default, the same hook [`to_openapi`](/guides/openapi/) takes.
```python
from probatio import Schema, to_json_schema
from probatio.codecs import UNSUPPORTED
def as_password(node):
if node is str.strip:
return {"type": "string", "writeOnly": True}
return UNSUPPORTED
to_json_schema(Schema({"token": str.strip}), custom_serializer=as_password)
# {'type': 'object', 'properties': {'token': {'type': 'string', 'writeOnly': True}}, 'additionalProperties': False}
```
## Supported keywords
[Section titled “Supported keywords”](#supported-keywords)
`from_json_schema` understands the keywords below. A purely descriptive keyword it does not read (`title`, `examples`) is ignored, so a partial schema still yields a usable validator. A *restrictive* keyword it cannot honor (`if`/`then`/`else`, `patternProperties`, `dependentSchemas`, `dependencies`, `unevaluatedProperties`, `unevaluatedItems`, `$dynamicRef`, `$recursiveRef`) is refused with a `SchemaError` rather than silently dropped, so an untrusted schema is never quietly widened to accept what its author meant to forbid. `dependentRequired` is honored only in the symmetric all-or-none form that maps to an `Inclusive` group (see below); its asymmetric form is refused the same way. `propertyNames` is honored: a mapping key is itself validated by a schema, so it becomes the key validator of the mapping (see below). The object and array keywords apply even on a node without a `type` (scoped to instances of their type, as the spec says). `to_json_schema` emits the same constructs in the other direction.
| Area | Keywords |
| ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Objects | `properties`, `required`, `additionalProperties`, `propertyNames`, `minProperties`, `maxProperties`, `dependentRequired` (symmetric all-or-none only) |
| Arrays | `items`, `minItems`, `maxItems`, `prefixItems`, `uniqueItems`, `contains`, `minContains`, `maxContains` |
| Strings | `minLength`, `maxLength`, `pattern`, `format` (`date`, `date-time`, `time`, `email`, `uri`, `ipv4`, `ipv6`, `uuid`, `hostname`, `byte`), `writeOnly`, `contentEncoding` (`Base64`) |
| Numbers | `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `multipleOf` |
| Values | `enum`, `const`, `not`, `type` (including a type array like `["string", "null"]`) |
| Compose | `anyOf`, `oneOf`, `allOf`, `$ref` (resolved against `$defs`/`definitions`) |
The named validators map to JSON Schema like this, and decode back into the matching validator (a `writeOnly` property decodes to a `Secret` key), so they round-trip:
| Probatio construct | JSON Schema output |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `ExactSequence` | `prefixItems` (with `items: false`, matching `minItems`/`maxItems`) |
| `Unique` | `uniqueItems` |
| `Contains` | `contains` |
| `Equal` / `Literal` | `const` (`type: null` for a `None` target) |
| `NotIn` | `not` over an `enum` |
| `Email` / `Email()` | `format: email` |
| `Url` / `Url()` / `FqdnUrl()` | `format: uri` |
| `Datetime` / `Date` / `Time` | `format: date-time` / `date` / `time` (a custom `strptime` format has no JSON Schema equivalent, so it exports as a plain string) |
| `IPv4Address` / `IPv6Address` | `format: ipv4` / `ipv6` |
| `UUID` | `format: uuid` |
| `Hostname` / `Fqdn` | `format: hostname` |
| `Port` | a bounded integer |
| `MultipleOf` | `multipleOf` |
| `Secret` key | its property with `writeOnly: true` |
| `Base64` | `contentEncoding: base64` |
| `Inclusive` group | `dependentRequired` (all-or-none); the symmetric map decodes back to an `Inclusive` group |
| Variable key validator | `propertyNames`, for a mapping that declares no properties (a plain `str` key emits nothing: it accepts every JSON key) |
Combinators and a few more constructs also render, though most have no inverse so they do not round-trip:
| Probatio construct | JSON Schema output |
| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `Any` / `Or` | `anyOf` (one branch renders as that branch alone, none as `not: {}`) |
| `Union` / `Switch` | `anyOf` (the discriminant is an optimization, so any branch is allowed) |
| `All` / `And` | one merged object, or `allOf` when two validators disagree on a keyword |
| `Maybe` | `anyOf` with `{"type": "null"}` |
| `SomeOf` | `oneOf` (exactly one), `anyOf` (at least one), or `allOf` (all) |
| A leading `EnsureList` | `anyOf` of whichever unwrapped forms it reaches, beside the list. The scalar branch needs room for one item and an item a lone value can be, and takes on a `contains` as well; the `null` branch needs room for an empty list, which `contains` never leaves. Anything the branches cannot be read off, a keyword asserting about the list as a whole among it, widens instead, which `strict=True` reports |
| `Msg` | the wrapped validator’s shape (the message has no JSON Schema equivalent) |
| An `enum.Enum` class | `enum` of the member values |
| `Self` | `{"$ref": "#"}` (a recursive reference to the document root) |
| `Alias` | one property per accepted name (plus `anyOf` of `required` when required) |
| `Any` key of literal names | one property per name (plus `anyOf` of `required` when the key is required) |
| A group keyed on an `Any` | one member satisfied by any of its names, so the group renders under `allOf` rather than `dependentRequired`; a name another key can also match carries no rule at all |
| `Exclusive` group | at-most-one (`not` over the pairs), or `oneOf` when the group is required |
| `AtLeastOne` / `ExactlyOne` | `anyOf` / `oneOf` over the per-key `required` |
| `AtMostOne` / `AllOrNone` | at-most-one (`not` over the pairs) / `dependentRequired`; a key that is not a plain string has no spelling, so the rule widens |
| `RequiredWith` | `dependentRequired`, or an `anyOf` implication when every trigger has to fire |
| `RequiredWithout` / `RequiredIf` | `anyOf: [{not: trigger}, {required: ...}]`; `RequiredIf` compares with `const`, so a value with no JSON form widens |
| `Duration` / `AsTimedelta` | `format: duration`, which has no decoder, so it decodes to a plain string |
The known widener: JSON Schema has a single `hostname` format, so both `Hostname` and `Fqdn` export to it and decode back as `Hostname`, meaning a round-tripped `Fqdn` accepts a dotless host the original would reject. Pin it with a `pattern` or an explicit check if the distinction matters.
An `Inclusive` group round-trips through `dependentRequired`. `to_json_schema` emits the symmetric all-or-none map (every member requires every other), and `from_json_schema` reads that shape back into an `Inclusive` group per connected set of mutually dependent properties. The general `dependentRequired` is broader: an asymmetric dependency (`a` requires `b` but not the reverse) has no `Inclusive` equivalent, so it is refused rather than silently widened.
`propertyNames` maps onto the key side of a mapping, where probatio already validates keys with a schema. `{"propertyNames": {"enum": ["a", "b"]}}` decodes to `{In(["a", "b"]): ...}`, so a key outside the set is rejected instead of quietly accepted, and `to_json_schema` emits the keyword back for a restrictive key validator. The two directions treat a permissive key schema differently, on purpose. Decoding keeps `{"type": "string"}` as the mapping’s `str` key, because a decoded schema also runs against Python mappings, where a non-string key is reachable and should be rejected; only `{}` and `true`, which accept anything at all, are dropped. Encoding drops a plain `str` key, since every JSON property name is a string and emitting it would add noise to every record schema.
The encoder emits the keyword only for a mapping that declares no properties: JSON Schema applies `propertyNames` to declared names too, while a probatio literal key is matched ahead of the variable keys and never sees them, so emitting it beside a declared property would reject input the schema accepts. Dropping it widens instead, the direction the encoder is allowed to be wrong in.
This matters in practice: Zod emits `propertyNames` for every `z.record(...)` and Pydantic emits it for a `dict` keyed by a `Literal`, which makes the keyword common in machine-generated schemas such as MCP tool definitions.
`oneOf` decodes with its exact semantics (a value must match exactly one branch, so one matching two or more is rejected), unlike the looser `anyOf`.
`from_openapi` adds the OpenAPI `nullable` keyword, covered in [OpenAPI](/guides/openapi/).
The JSON data model
The decoder follows JSON’s types, not Python’s. A decoded `{"type": "integer"}` rejects `True` (a boolean is not a JSON number) and accepts `1.0` (a number with no fractional part), and a decoded `enum` or `const` keeps booleans distinct from `1` and `0`. On the encode side, `Datetime`, `Date`, and `Time` render ISO strings, and a decoded `format: date-time` accepts RFC 3339 timestamps (a `Z` or a numeric offset, with or without fractional seconds).
## Untrusted input is the default assumption
[Section titled “Untrusted input is the default assumption”](#untrusted-input-is-the-default-assumption)
`from_json_schema` and `from_openapi` treat their input as untrusted. A JSON Schema can arrive from anywhere, and two of its constructs can wreck a naive decoder: a `pattern` that backtracks catastrophically, and a document nested deep enough to overflow the Python stack. Both are refused with `SchemaError` rather than hanging or crashing.
A nested unbounded quantifier like `(a+)+` is the classic catastrophic regular expression. Probatio refuses to compile it:
```python
from probatio import from_json_schema
from_json_schema({"type": "string", "pattern": "(a+)+$"})
```
A pathologically deep document hits the same wall: past a generous depth limit, the decoder raises `SchemaError` instead of recursing into a `RecursionError`.
The guards are conservative
Both safeguards err toward refusing. An unusual but legitimate schema, a deeply nested one or a pattern that only looks dangerous, can trip them. That is the trade: a clean `SchemaError` you can catch beats a hang you cannot. If you control the source and trust it, validate the result, not the schema.
# Lazy building
> Defer a schema's compilation to first use, so schemas that are built but never validated cost nothing.
A `Schema` normally compiles its declaration into a validator the moment you build it. That is the right default: a malformed schema fails where it is written, the same as voluptuous. But some programs build far more schemas than they ever validate against. Home Assistant registers on the order of a thousand service schemas and a thousand websocket-command schemas at startup, and a given install exercises a small fraction of them. Every unused one still pays to compile, and holds its compiled validator tree in memory for the life of the process.
Lazy building defers that work. Under the `LAZY` build policy a schema stores its declaration and compiles on the first validation instead of at construction, so a schema that is built but never validated never compiles and never holds a validator tree. It is opt-in: the default stays eager, so nothing changes unless you ask.
## The default: eager
[Section titled “The default: eager”](#the-default-eager)
The process starts on the `EAGER` policy. A schema compiles when you build it, and a definition error is raised right there, at construction, exactly as voluptuous does. This is the drop-in behavior, and for most programs it is what you want: schemas are a fixed set built once at import, and building them is cheap.
```python
from probatio import Schema, Required
schema = Schema({Required("name"): str}) # compiled here, on this line
schema({"name": "app"}) # {'name': 'app'}
```
## Opting into lazy
[Section titled “Opting into lazy”](#opting-into-lazy)
Set the policy once, early, from deliberate startup code, before the schemas you want deferred are built. There is no environment variable on purpose: this is an architectural choice, not a deployment toggle.
```python
from probatio import BuildPolicy, set_build_policy
set_build_policy(BuildPolicy.LAZY) # defer every eligible schema to first use
set_build_policy(BuildPolicy.EAGER) # the default: compile at construction
```
Applications set this, not libraries
`set_build_policy` is a single process-wide switch, so only an application (the process that owns the run) should call it. A library that sets it changes how every schema in the process builds, the application’s own and those of unrelated dependencies included, which is not a library’s call to make. And unlike the compile policy, the build policy has no per-schema form, so a library cannot scope it to its own schemas either. Leave it to the application and rely on the `EAGER` default.
After `set_build_policy(BuildPolicy.LAZY)`, a schema you build does not compile until its first validation. A schema you build and never call never compiles at all, and never allocates its validator tree.
```python
from probatio import Schema, Required, BuildPolicy, set_build_policy
set_build_policy(BuildPolicy.LAZY)
# Registered at startup; the walk is deferred.
schema = Schema({Required("resource"): str})
# ... much later, only if this command is ever invoked:
schema({"resource": "sensor.sun"}) # compiles now, then validates
```
## What defers, and what does not
[Section titled “What defers, and what does not”](#what-defers-and-what-does-not)
Only a plain top-level `Schema` defers. Anything whose compiled form is needed at construction builds eagerly, even under `LAZY`, so laziness never leaves a half-built schema anywhere it would be read:
* A **combinator branch** (`Any(...)`, `All(...)`) compiles its branches when the combinator is built.
* A **`DataclassSchema` or `TypedDictSchema`** builds at construction, since it wires field validation (and, for a dataclass, instance construction) together then.
* A nested schema reused as a value builds when its parent is first validated.
* An explicit `compile=` request, or `Schema.compile()`, builds now.
Reading a lazy schema’s declaration does not build it: `.schema`, `str()`, `.extend()`, and the codecs all work on the declaration, so introspection stays free.
## The one trade: error timing
[Section titled “The one trade: error timing”](#the-one-trade-error-timing)
Deferring the compile walk defers the errors it raises. Under `EAGER`, a malformed schema (a contradiction like two presence markers on one key, an alias colliding with another key) raises `SchemaError` at construction. Under `LAZY`, that same error raises on the **first validation** instead, because that is when the walk runs. The schema is just as wrong; you learn about it a moment later.
For a program whose schemas are static and covered by tests, this is a non-issue, the tests validate, so a broken schema still fails in CI. But it is a real difference from voluptuous, which is why `LAZY` is opt-in and `EAGER` is the default. If you rely on construction-time schema errors, stay eager.
## When to use it
[Section titled “When to use it”](#when-to-use-it)
Reach for `LAZY` when you build many schemas at import but validate against only some of them in a given run: a large plugin surface, a command or service registry, per-feature validation that most installs never touch. The win is both time (the deferred walks never run) and memory (their validator trees never materialize), which matters most on a small device. If your schemas are a fixed set you validate against on every run, eager is simpler and there is nothing to gain.
# Loading and dumping
> Parse JSON, YAML, or TOML with your own library, then validate the result with Probatio.
Validation works on Python values, but data arrives as text: a JSON request body, a YAML config file, a TOML manifest. Probatio does not parse any of it. It validates the parsed object, and parsing (and serializing back out) stays with you, using whatever library you already trust for the format. That keeps a whole class of parser bugs and dependencies out of the library, and it means you are never surprised by which parser happens to be installed.
This is the same split voluptuous, pydantic (`model_validate`), marshmallow, and cattrs draw: the validator takes an object, you own the bytes. The examples here use [orjson](https://github.com/ijl/orjson) and [YAMLRocks](https://pypi.org/project/yamlrocks/) as fast, concrete choices, but nothing about Probatio ties you to them.
## Loading
[Section titled “Loading”](#loading)
Parse the text yourself, then hand the result to a schema. JSON parses with the standard library, so the two-liner works on a bare install:
```python
import json
from probatio import Schema, Required
schema = Schema({Required("port"): int})
schema(json.loads('{"port": 8080}')) # {'port': 8080}
```
Swapping in a faster parser is a one-import change; the schema call is identical. With orjson:
```python
import orjson
from probatio import Schema, Required
schema = Schema({Required("port"): int})
schema(orjson.loads('{"port": 8080}')) # {'port': 8080}
```
YAML has no standard-library parser. Reach for YAMLRocks (fast, YAML 1.2, and bomb-resistant) or PyYAML, and always use a safe load for untrusted input:
```python
import yamlrocks
from probatio import Schema, Required
schema = Schema({Required("port"): int})
schema(yamlrocks.loads("port: 8080")) # {'port': 8080}
```
TOML reads with the standard library’s `tomllib` (Python 3.11 and newer):
```python
import tomllib
from probatio import Schema, Required
schema = Schema({Required("port"): int})
schema(tomllib.loads("port = 8080")) # {'port': 8080}
```
Untrusted YAML
Use a safe loader for YAML you did not write. PyYAML’s `yaml.load` builds arbitrary Python objects from tags in the document; `yaml.safe_load` does not, so a hostile payload cannot instantiate classes or run constructors. A safe load still expands aliases, so a small document full of anchors that reference each other (the billion-laughs pattern) can blow up to gigabytes of logical nodes. YAMLRocks counts the expanded nodes and refuses a document that blows up; PyYAML does not, so bound the input size when you rely on it.
YAML spec versions differ
YAMLRocks implements YAML 1.2; PyYAML implements YAML 1.1. They parse a few inputs differently. Under 1.1, the bare words `yes`, `no`, `on`, and `off` are booleans, a leading-zero number like `0755` is octal, and `12:30` is a sexagesimal number; under 1.2 all of these are plain strings or decimals. Quote these values in your YAML (`"yes"`, `"0755"`) when the distinction matters, and validate with an explicit type so the schema, not the parser, decides.
## Dumping
[Section titled “Dumping”](#dumping)
Serialization is the mirror image: validate first, then serialize the validated value with your own library. There is a catch worth stating plainly. A validated value often carries types the text format has no native form for, like a `datetime`, a `Decimal`, or a `set`. Turning those back into text is a decision only you can make (an ISO string? a Unix timestamp? which format?), so it is your `default` hook, not Probatio’s job.
The standard library’s `json.dumps` takes a `default` callable for exactly this:
```python
import json
from datetime import date
from decimal import Decimal
def to_jsonable(value):
if isinstance(value, (set, frozenset)):
return sorted(value)
if isinstance(value, Decimal):
return float(value)
if isinstance(value, date):
return value.isoformat()
raise TypeError(type(value).__name__)
json.dumps({"when": date(2020, 1, 1)}, default=to_jsonable) # '{"when": "2020-01-01"}'
```
orjson takes the same `default` argument and handles `datetime` natively, so the hook only covers what it does not know:
```python
import orjson
from decimal import Decimal
def to_jsonable(value):
if isinstance(value, (set, frozenset)):
return sorted(value)
if isinstance(value, Decimal):
return float(value)
raise TypeError(type(value).__name__)
orjson.dumps({"scale": Decimal("1.5")}, default=to_jsonable) # b'{"scale":1.5}'
```
This is deliberately not a framework. Probatio’s transforms run one way: `AsDate` turns a string into a `date` on the way in, and it does not turn a `date` back into the string it came from. If you need a faithful round-trip (a value dumped in the same shape it was loaded), that is a bidirectional serializer’s job, like marshmallow, cattrs, or pydantic. Probatio validates; it does not serialize.
## Source locations
[Section titled “Source locations”](#source-locations)
When a config fails validation, the useful question is *where in the file*. Probatio’s `humanize_error` accepts a `locator`: a callable you supply that maps an error’s `path` to a `Location` (`line`, `column`, and `file`), so each failure line gains the place it points at, like `Got 70000 (at 2:9)`.
Building the locator needs a parser that records source positions. YAMLRocks (0.5.0 and newer) carries them, so you can walk the located tree it returns and resolve an error `path` to a position:
```python
from probatio import Schema, Required, Range, MultipleInvalid
from probatio.error import Location
from probatio.humanize import humanize_error
schema = Schema({Required("server"): {Required("port"): Range(min=1, max=65535)}})
data, positions = load_yaml_with_positions("server:\n port: 70000\n") # your loader
def locator(path):
node = positions_at(positions, path) # your lookup into the located tree
return Location(node.line, node.column) if node else None
try:
schema(data)
except MultipleInvalid as err:
print(humanize_error(data, err, locator=locator))
# value must be at most 65535 at 'server.port'. Got 70000 (at 2:9)
```
`Location` is exported from `probatio` (it lives in `probatio.error`). It renders as `file:line:column`, or programs can read its fields directly. Return `None` from the locator for a path you cannot place.
# OpenAPI
> Render a Probatio schema as an OpenAPI Schema object, build one back, and handle nullable.
OpenAPI Schema objects are JSON Schema with a few of their own rules. `to_openapi` and `from_openapi` are the OpenAPI pair, exported from the top level. `from_openapi` is the JSON Schema decoder plus the OpenAPI extras, so its supported keywords, round-trip caveats, and untrusted-input guards match [JSON Schema](/guides/json-schema/); read that page first. The `to_openapi` encoder is a separate implementation (it targets OpenAPI 3.0 or 3.1 and takes a `custom_serializer`), so its construct coverage differs from `to_json_schema` in places. This page covers what OpenAPI adds.
## Both directions
[Section titled “Both directions”](#both-directions)
`to_openapi(schema)` renders a schema as an OpenAPI Schema object. `from_openapi(dict)` is the inverse: it builds a `Schema` back. Use the pair to publish request and response schemas in the spec of an OpenAPI-described service, straight from the validators you already run, or to build a validator from a spec you consume. For LLM tool calling and MCP, which take JSON Schema rather than OpenAPI, see the [LLM tool recipe](/recipes/llm-tools/).
```python
from probatio import Schema, Required, Optional, to_openapi
schema = Schema({Required("name"): str, Optional("port", default=8080): int})
to_openapi(schema)
# {'type': 'object', 'properties': {'name': {'type': 'string'}, 'port': {'type': 'integer', 'default': 8080}}, 'required': ['name'], 'additionalProperties': False}
```
A closed object emits `additionalProperties: False`, the same as `to_json_schema`. An `ALLOW_EXTRA` object emits `additionalProperties: True`, and a `REMOVE_EXTRA` object omits the keyword (it accepts extra keys but strips them, so the wire shape is open). This is one of the places `to_openapi` diverges from voluptuous-openapi, which omits the keyword on a closed object: `to_openapi` emits correct OpenAPI even where the reference implementation does not.
A value that accepts anything (`object`, a callable without a type hint) renders as the empty schema `{}`, in every position, the same as `to_json_schema`. voluptuous-openapi renders `object` as a JSON object and gives any other untyped property `type: string`; both reject values the validator accepts, so `to_openapi` does neither. A bare bound (`Range`) is still typed `number`, as voluptuous-openapi does. A mixed enum stays untyped, since a `string` stamp would reject the `1` in `In([1, "x"])`. A bounded `Length` on its own renders one branch per sized type (`string`, `array`, `object`) with the bounds on that type’s keyword, because `Length` accepts all three and `minLength` alone constrains only strings. Beside a sized type, as in `All(str, Length(min=1))`, the bounds land on that type’s keyword directly; beside anything else the branches stay and the rest is intersected with them. An unbounded `Length()` accepts every value and renders as `{}`.
Going the other way, an OpenAPI Schema object becomes a working validator. `nullable` is read back too, so a nullable field accepts both `None` and a real value:
```python
from probatio import from_openapi
document = {
"type": "object",
"properties": {
"name": {"type": "string"},
"age": {"type": "integer", "nullable": True},
},
"required": ["name"],
}
schema = from_openapi(document)
schema({"name": "Ada", "age": None}) # {'name': 'Ada', 'age': None}
schema({"name": "Ada", "age": 37}) # {'name': 'Ada', 'age': 37}
```
## The nullable keyword
[Section titled “The nullable keyword”](#the-nullable-keyword)
`nullable` is the visible difference from JSON Schema. A value that also accepts `None` renders as two branches in JSON Schema, but as a single `nullable: True` in OpenAPI:
```python
from probatio import Schema, Maybe, to_json_schema, to_openapi
schema = Schema({"nickname": Maybe(str)})
to_json_schema(schema)["properties"]["nickname"]
# {'anyOf': [{'type': 'null'}, {'type': 'string'}]}
to_openapi(schema)["properties"]["nickname"]
# {'type': 'string', 'nullable': True}
```
`to_openapi` defaults to OpenAPI 3.0 (the `nullable` keyword above). Pass `openapi_version="3.1.0"` for 3.1, which drops `nullable` and expresses nullability the JSON Schema way instead, as an `anyOf` with a `{"type": "null"}` branch. Those two spellings, `"3.0"` and `"3.1.0"`, are the only ones accepted (they match voluptuous-openapi’s `OpenApiVersion` values); anything else, `"3.1"` included, raises `ValueError` rather than quietly rendering as one of them.
The split reaches further than the keyword. A leading `EnsureList` offers the forms it wraps beside the list, and its null form follows the version: a branch on 3.1, the flag on 3.0. Where the scalar form would have to say “not null”, which 3.0 has no way to spell, the whole rendering widens rather than writing a branch that admits one value too many, which is the direction the contract allows, where leaving it out would reject input the schema accepts. So 3.0 gets the branch with the array exclusion it can spell and without the null one it cannot, a shade wider than the schema, and `strict=True` refuses it on the grounds that it is not exact. A nullable item with a named type needs neither: the lone value is never the null half, so dropping the flag says the rest exactly.
Beside other branches on 3.0 the flag becomes a branch of its own, and the only shape 3.0 has for one is a nullable object, which lets every object through as well. The document is wider than the schema from there on, so `strict=True` reports it. Alone on the array the flag is exact, an array or null and nothing else, and strict has nothing to say.
## Group constraints across versions
[Section titled “Group constraints across versions”](#group-constraints-across-versions)
The `Inclusive` (all-or-none) and `Exclusive` (at most one, or exactly one when required) dict-group markers render as object-level constraints, and one of them splits on version. `Exclusive` uses `oneOf`/`not`, which both OpenAPI versions have. `Inclusive` maps to `dependentRequired`, which only OpenAPI 3.1 has:
```python
from probatio import Schema, Inclusive, to_openapi
schema = Schema({Inclusive("lat", "coords"): float, Inclusive("lon", "coords"): float})
to_openapi(schema, openapi_version="3.1.0")["dependentRequired"]
# {'lat': ['lon'], 'lon': ['lat']}
```
OpenAPI 3.0 has no `dependentRequired` (and silently ignores it), so there the same all-or-none is spelled with the keywords 3.0 does have, an `allOf` entry that accepts every member present or none present and rejects any partial combination. The 3.1 `dependentRequired` decodes back to an `Inclusive` group through `from_openapi`; the 3.0 form round-trips by behavior, not back to the marker.
The same object-level constraints come from the key-presence validators, which sit beside the mapping rather than on one of its keys. `AtLeastOne` renders as `anyOf` over the per-key `required`, `ExactlyOne` as `oneOf`, `AtMostOne` as the `not` over the pairs, and `AllOrNone` splits on version exactly as `Inclusive` does:
```python
from probatio import AllOrNone, AtLeastOne, All, Optional, Schema, to_openapi
base = Schema({Optional("a"): str, Optional("b"): str})
to_openapi(Schema(All(AtLeastOne("a", "b"), base)))["anyOf"]
# [{'required': ['a']}, {'required': ['b']}]
all_or_none = to_openapi(
Schema(All(AllOrNone("a", "b"), base)), openapi_version="3.1.0"
)
all_or_none["dependentRequired"]
# {'a': ['b'], 'b': ['a']}
```
The rules that make one key depend on another render too. `RequiredWith` is `dependentRequired` where that exists, and the rest are written as an implication, `anyOf: [{not: trigger}, {required: ...}]`, rather than `if`/`then`: OpenAPI 3.0 has the first keyword and not the second, so one shape serves both versions.
```python
from probatio import All, Optional, RequiredWithout, Schema, to_openapi
base = Schema({Optional("cert"): str, Optional("cert_path"): str})
to_openapi(Schema(All(RequiredWithout("cert", "cert_path"), base)))["anyOf"]
# [{'required': ['cert']}, {'required': ['cert_path']}]
```
Two things leave a rule out. A key that is not a plain string has no spelling in either document. So does a `RequiredIf` whose compared value JSON cannot hold, or holds differently: `(1, 2)` would render as `[1, 2]`, which the tuple never equals, so emitting it would make the document fire where Probatio does not. Either way the rule widens, which `strict=True` reports.
A group member does not have to be a literal key. `Inclusive(Any("hours", "minutes"), "d")` is one member that either name satisfies, and the rule is the usual all-or-none: that member and every other member of the group are either all present or all absent. `dependentRequired` cannot express it, having no way to say “if this name is present then at least one of those”, so on 3.1 a group holding such a member renders under `allOf` instead, at the cost of not decoding back to the marker.
A rule is only written for a key that certainly receives the names it lists. Several things can take a name first: a literal key, an `Alias` under any of its accepted names, another `Any` listing the same one, or a variable key such as `{str: ...}`, which matches anything. Rather than model that precedence, the codecs leave the rule out whenever a name is not provably theirs, so the document stays a superset of what Probatio validates rather than contradicting it. The properties are emitted either way, and `Extra` never competes, being the catch-all for names nothing else matched. Under `strict=True` the dropped rule raises instead, like any other widening.
## Strict mode
[Section titled “Strict mode”](#strict-mode)
By default a construct with no OpenAPI form widens to an open schema (`{}`), so the emitted document stays a superset of what Probatio validates. Pass `strict=True` to raise `SchemaError` instead, so a lossy conversion is caught rather than silently accepted, the same as [`to_json_schema`](/guides/json-schema/#widening-the-encoder-and-overriding-it):
```python
from probatio import Schema, In, to_openapi
to_openapi(Schema(In([b"raw"])), strict=True)
```
A faithfully open construct is not a loss, so it passes `strict=True`: `object` is the empty schema, and a bare `dict` is an open object.
A version-specific drop is a loss like any other. OpenAPI 3.0 has neither `contains` nor `prefixItems` and misreads both, so a `Contains` renders as a plain array there and an `ExactSequence` keeps only the length it pins. The document accepts what the schema rejects either way, so `strict=True` reports it rather than handing back the looser array without a word. An empty `ExactSequence` is the exception: it has no position to lose, and the length keywords say all of it.
## Customizing the output
[Section titled “Customizing the output”](#customizing-the-output)
`to_openapi` takes a `custom_serializer` hook, the same one `to_field_list` uses, to override how individual nodes render. It is documented with the field-list codec in [Field lists](/guides/field-lists/#a-custom-serializer-hook), since both codecs share it.
## Shared with JSON Schema
[Section titled “Shared with JSON Schema”](#shared-with-json-schema)
Everything else is the JSON Schema codec:
* The [supported keywords](/guides/json-schema/#supported-keywords) and how each validator maps to them.
* The round trip is [not lossless](/guides/json-schema/#both-directions); treat it as “the validatable shape survives”.
* `from_openapi` treats its input as [untrusted](/guides/json-schema/#untrusted-input-is-the-default-assumption): a catastrophic `pattern` or a pathologically deep document is refused with `SchemaError`, not a hang or a crash.
# The probatio decorator
> Validate a function's arguments straight from its type annotations, with coercion, per-parameter rules, optional return checking, your own error, and async support.
`@probatio` validates a callable’s arguments against its type annotations before the body runs. It reads the signature the same way `DataclassSchema` reads a dataclass: each annotated parameter becomes a validator, and the body receives the validated (and coerced) value.
It is the annotation-driven companion to [`validate`](/guides/custom-validators/#validating-function-arguments), the voluptuous drop-in where you name a schema per argument by hand. Reach for `@probatio` when your parameters are already typed and you want that typing enforced at the boundary.
## The basics
[Section titled “The basics”](#the-basics)
Annotate the parameters and decorate. Nothing else is required.
```python
from probatio import probatio
@probatio
def area(width: int, height: int) -> int:
return width * height
area(3, 4) # 12
```
A parameter that fails its type raises `MultipleInvalid` before the body runs, naming the parameter. It is the same exception a `Schema` raises, so one `except` covers both.
```python
from probatio import probatio, MultipleInvalid
@probatio
def area(width: int, height: int) -> int:
return width * height
try:
area("wide", 4)
except MultipleInvalid as err:
print(err) # expected int at 'width'
```
An unannotated parameter is left alone, so `self` and `cls` need no special handling and the decorator drops straight onto a method. A default the caller omits is also left alone: it is not validated or coerced, the body receives it exactly as declared, so write an already-valid default.
```python
from typing import Annotated
from probatio import probatio, Range
class Canvas:
@probatio
def scale(self, factor: Annotated[int, Range(min=1)]) -> int:
return factor * 2
Canvas().scale(3) # 6
```
## Coercion reaches the body
[Section titled “Coercion reaches the body”](#coercion-reaches-the-body)
An annotation drives the same engine the schema builders use, so a coercing annotation converts the value and the body receives the converted one. The annotation says what the parameter *is*, and the type confirms what the coercer produced.
```python
from typing import Annotated
from probatio import probatio, Coerce
@probatio
def repeat(text: str, times: Annotated[int, Coerce(int)]) -> str:
return text * times
repeat("ab", "3") # 'ababab'
```
The full annotation mapping is the one [dataclasses use](/guides/dataclasses/#annotations-drive-the-validators): unions become `Any`, `X | None` becomes `Maybe`, and a parameter typed as a dataclass validates a mapping into an instance.
```python
from dataclasses import dataclass
from probatio import probatio
@dataclass
class Point:
x: int
y: int
@probatio
def distance(point: Point) -> int:
return point.x + point.y
distance({"x": 3, "y": 4}) # 7
```
## Extra rules per parameter
[Section titled “Extra rules per parameter”](#extra-rules-per-parameter)
The annotation gives you the type check. For anything more (a length, a range, a pattern), pass a `{parameter: validator}` map as the first argument. It runs after the inferred type. (If you know `DataclassSchema`, this is its `additional_constraints` in decorator form.)
```python
from probatio import probatio, Length
@probatio({"name": Length(min=2)})
def greet(name: str) -> str:
return f"hi {name}"
greet("ada") # 'hi ada'
```
A constraint that names no parameter, or names a `*args`/`**kwargs`, is refused when the function is decorated, so a typo fails loudly instead of silently doing nothing. The packed parameters themselves are skipped entirely: an annotation on `*args` or `**kwargs` names no single value to validate, so it is ignored and the values pass through unchecked.
## Validating the return value
[Section titled “Validating the return value”](#validating-the-return-value)
Return validation is off by default. Opt in with `returns`: pass `True` to check the result against the `-> R` annotation, or pass a schema to check against it directly.
```python
from typing import Annotated
from probatio import probatio, Range
@probatio(returns=True)
def clamp(value: int) -> Annotated[int, Range(min=0)]:
return value
clamp(5) # 5
```
A map and a return schema combine, so one decorator can cover both ends:
```python
from probatio import probatio, Length
@probatio({"name": Length(min=2)}, returns=str)
def shout(name: str) -> str:
return name.upper()
shout("ada") # 'ADA'
```
## Raising your own error
[Section titled “Raising your own error”](#raising-your-own-error)
A library usually has an exception hierarchy of its own, and its callers expect one `except` to cover everything it raises. Pass `error` to raise that exception for an invalid argument instead of `MultipleInvalid`. It is called with the `Invalid`, and the result is raised from it, so the original failure (with its `path`) stays reachable as `__cause__`.
```python
from typing import Annotated
from probatio import probatio, Range
class ClientError(Exception):
"""Base for everything the client raises."""
class ClientValueError(ClientError, ValueError):
"""The caller passed an invalid argument."""
@probatio(error=ClientValueError)
def fetch(limit: Annotated[int, Range(min=0)]) -> int:
return limit
try:
fetch(-1)
except ClientError as err:
print(err) # value must be at least 0 at 'limit'
```
Any callable taking the `Invalid` works, when the exception wants more than the message:
```python
from probatio import Invalid, probatio
class ClientValueError(ValueError):
"""The caller passed an invalid argument."""
def bad_argument(invalid: Invalid) -> ClientValueError:
return ClientValueError(f"invalid {invalid.path[0]}: {invalid.msg}")
@probatio(error=bad_argument)
def fetch(limit: int) -> int:
return limit
try:
fetch("ten")
except ClientValueError as err:
print(err) # invalid limit: expected int
```
Only argument validation is translated. A result failing `returns` still raises `MultipleInvalid`, since that is a mistake in the function rather than the caller. A malformed call still raises the `TypeError` Python gives it, and whatever the body raises is left alone, so a rule that spans several arguments and is checked in the body raises the library’s exception itself.
## Async functions
[Section titled “Async functions”](#async-functions)
A coroutine function is validated the same way. The arguments are checked before the call, and the result schema (if any) runs on the awaited value.
```python
import asyncio
from typing import Annotated
from probatio import probatio, Coerce
@probatio
async def fetch(user_id: Annotated[int, Coerce(int)]) -> int:
return user_id + 1
asyncio.run(fetch("41")) # 42
```
## Skipping validation
[Section titled “Skipping validation”](#skipping-validation)
`@probatio` builds on `functools.wraps`, so the undecorated callable stays reachable through the standard `__wrapped__` attribute. Call it when the input is already trusted and you want to skip the check.
```python
from typing import Annotated
from probatio import probatio, Coerce
@probatio
def widen(value: Annotated[int, Coerce(int)]) -> int:
return value
widen("5") # 5, validated and coerced
widen.__wrapped__("5") # '5', straight through, no validation
```
That escape hatch frames when not to use the decorator at all. Every decorated call pays for validation, so decorate boundaries (a request handler, a service entry point, the public surface of a library), not hot inner functions that run in tight loops on values already validated one frame up. When a single hot call site sits behind an otherwise useful boundary check, `__wrapped__` skips the cost for just that site. See [Performance](/reference/performance/) for the cost model.
## Where to next
[Section titled “Where to next”](#where-to-next)
* [`validate`](/guides/custom-validators/#validating-function-arguments): the voluptuous-compatible decorator with hand-named schemas.
* [Schemas from dataclasses](/guides/dataclasses/): the same annotation engine, building an instance from a mapping.
* [Built-ins by role](/reference/builtins-by-role/): the validators and coercers to reach for in an annotation.
# Recursive schemas
> Use Self to validate tree-shaped and nested-recursive data, with a depth guard that turns runaway nesting into a clean error.
Some data refers to its own shape. A comment has replies, and each reply is a comment. A tree node has children, and each child is a node. You cannot write that with a schema that names itself, because the schema is not finished being defined yet. `Self` is the way out: inside a schema it means “validate against the whole enclosing schema.”
## Self refers to the enclosing schema
[Section titled “Self refers to the enclosing schema”](#self-refers-to-the-enclosing-schema)
Import `Self` and use it as a value, the same way you would use any validator. While the schema compiles, `Self` resolves to the schema being built, so it can appear inside its own definition.
```python
from probatio import Schema, Self, Required, Optional
comment = Schema(
{
Required("text"): str,
Optional("replies", default=list): [Self],
}
)
comment({"text": "hi"}) # {'text': 'hi', 'replies': []}
```
`replies` is a list of `Self`, so every reply is validated against the comment schema again, all the way down:
```python
from probatio import Schema, Self, Required, Optional
comment = Schema(
{
Required("text"): str,
Optional("replies", default=list): [Self],
}
)
data = {
"text": "top",
"replies": [
{"text": "first"},
{"text": "second", "replies": [{"text": "nested"}]},
],
}
comment(
data
) # {'text': 'top', 'replies': [{'text': 'first', 'replies': []}, {'text': 'second', 'replies': [{'text': 'nested', 'replies': []}]}]}
```
Recursion follows the data. A finite structure validates fine, because each level is just another call into the same schema, up to [the depth guard](#the-depth-guard) described below.
## Self in a combinator
[Section titled “Self in a combinator”](#self-in-a-combinator)
`Self` works as a direct mapping value or sequence element, like `[Self]` above or `{"child": Self}`, and also as a branch of a combinator, so a key can recurse or take an alternative. `Any(Self, "stop")` means “another level, or the literal `stop`”:
```python
from probatio import Schema, Self, Any
tree = Schema({"value": int, "next": Any(Self, "stop")})
tree({"value": 1, "next": {"value": 2, "next": "stop"}})
# {'value': 1, 'next': {'value': 2, 'next': 'stop'}}
```
`All(Self, ...)` works the same way, chaining the recursive check with another schema. `Self` has to be a *direct* branch of the combinator, as above. Buried inside a structure that is a branch (`Any({"inner": Self}, ...)`), it binds to that structure rather than the whole schema, so keep it at the top of the branch.
The one place `Self` does not reach is a plain wrapping validator like `Maybe`, which compiles its inner schema outside any combinator, so `Self` has nothing to resolve to and it raises `SchemaError` at build time:
```python
from probatio import Schema, Self, Maybe
Schema({"next": Maybe(Self)}) # SchemaError
```
For “this key or nothing,” make the key `Optional` rather than wrapping `Self` in `Maybe`:
```python
from probatio import Schema, Self, Required, Optional
linked = Schema(
{
Required("value"): int,
Optional("next"): Self,
}
)
linked({"value": 1, "next": {"value": 2}}) # {'value': 1, 'next': {'value': 2}}
```
## The depth guard
[Section titled “The depth guard”](#the-depth-guard)
Recursive schemas meet a classic problem: cyclic data, or data nested so deep that validating it blows the call stack. voluptuous lets that surface as a `RecursionError`, which is an interpreter-level crash, not a validation result. Probatio does not. It counts the `Self` recursion depth and, past a limit, raises a clean `Invalid` that you catch alongside every other validation error.
The limit is a fraction of the interpreter’s recursion limit, read live. So if you genuinely have deep data, raising the limit with `sys.setrecursionlimit()` scales the guard with it; you are not boxed in by a hard-coded number. Keep that within reason, though: past a point the operating system’s own stack, not the recursion limit, is the ceiling, so an extreme limit can still overflow.
Here is a node schema fed 5000 levels of nesting, built in a loop. It comes back as a `MultipleInvalid`, the same type you already handle:
```python
from probatio import Schema, Self, Required, Optional, MultipleInvalid
node = Schema(
{
Required("value"): int,
Optional("children", default=list): [Self],
}
)
data = {"value": 0, "children": []}
cursor = data
for depth in range(1, 5000):
child = {"value": depth, "children": []}
cursor["children"] = [child]
cursor = child
try:
node(data)
except MultipleInvalid as err:
print(err.errors[0].msg) # data is nested too deeply for this recursive schema
```
The same guard catches cyclic data, where a structure points back at itself and would otherwise recurse forever. Either way you get an `Invalid`, never a `RecursionError`, so untrusted input cannot crash validation.
One performance note: the [compiled engine](/guides/compiled-schemas/) skips recursive schemas, so a schema using `Self` always validates interpreted. It behaves exactly the same; it just does not get the compiled speedup.
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Dict schemas and markers](/guides/dict-schemas-and-markers/): `Required` and `Optional`, used heavily above.
* [Custom validators](/guides/custom-validators/): the callable contract `Self` builds on.
* [The validation model](/guides/validation-model/): how a schema compiles once and validates many times.
# Sequence schemas
> How list, tuple, and set schemas validate elements, and how to pin a sequence's shape and bounds.
A `list`, `tuple`, or `set` schema validates a sequence: the container type must match, and every element must match one of the element schemas. This page covers that rule and its one classic subtlety, error paths into sequences, fixed-shape sequences, and nesting. It builds on [the validation model](/guides/validation-model/); the mapping side lives in [dict schemas and markers](/guides/dict-schemas-and-markers/).
## Every element, any alternative
[Section titled “Every element, any alternative”](#every-element-any-alternative)
The schemas inside the brackets are alternatives, not positions. `Schema([int, str])` does not mean “an int, then a str”; it means “a list where every element is an int *or* a str”, in any order and any number:
```python
from probatio import Schema
schema = Schema([int, str])
schema([1, "a", 2]) # [1, 'a', 2]
schema([]) # []
```
Each element tries the alternatives in order and the first match wins, like `Any`. Because a match can transform, order matters here too: with `[Coerce(int), str]` a numeric string becomes an int before `str` gets a look.
```python
from probatio import Schema, Coerce
Schema([Coerce(int), str])(["5", "a"]) # [5, 'a']
```
An element that matches no alternative fails, and the message names the alternative tried last:
```python
from probatio import Schema, Invalid
try:
Schema([int, str])([1, 2.5])
except Invalid as err:
print(err) # expected str at '[1]'
```
Validation returns a new list; the input is not mutated. And the container is checked strictly: a tuple, a set, or a string (even though Python treats a string as a sequence) is not a list.
```python
from probatio import Schema, Invalid
try:
Schema([str])("abc")
except Invalid as err:
print(err) # expected a list
```
## The empty sequence schema
[Section titled “The empty sequence schema”](#the-empty-sequence-schema)
With no alternatives there is nothing an element could match, so `Schema([])` accepts only the empty list:
```python
from probatio import Schema, Invalid
schema = Schema([])
schema([]) # []
try:
schema([1])
except Invalid as err:
print(err) # invalid value at '[0]'
```
## Tuples and sets
[Section titled “Tuples and sets”](#tuples-and-sets)
A `tuple` or `set` schema follows the same rule with its own container type: the input must be a tuple (or a set), every element must match one of the alternatives, and the result is a new container of that type.
```python
from probatio import Schema
Schema((int,))((1, 2, 3)) # (1, 2, 3)
Schema({int})({1, 2}) # {1, 2}
```
The container types do not mix:
```python
from probatio import Schema, Invalid
try:
Schema((int,))([1, 2])
except Invalid as err:
print(err) # expected a tuple
```
One caveat for sets: a failing element still gets an index in the error path, but a set has no order, so that index reflects iteration order, not a position you can count on.
## Error paths into sequences
[Section titled “Error paths into sequences”](#error-paths-into-sequences)
A failing element is reported by its index. In `error.path` the index is an integer, and the dotted rendering shows it as `[n]`:
```python
from probatio import Schema, Invalid
schema = Schema({"ports": [int]})
try:
schema({"ports": [80, "nope"]})
except Invalid as err:
print(err) # expected int at 'ports[1]'
print(err.path) # ['ports', 1]
```
Like a dict schema, a list schema does not stop at the first failing element; every failure lands in the `MultipleInvalid`:
```python
from probatio import Schema, MultipleInvalid
try:
Schema([int])(["a", "b"])
except MultipleInvalid as err:
for sub in err.errors:
print(sub)
# expected int at '[0]'
# expected int at '[1]'
```
See [error handling](/guides/error-handling/) for paths and rendering in depth.
## Fixed shape and bounds
[Section titled “Fixed shape and bounds”](#fixed-shape-and-bounds)
When the positions *do* matter, reach for `ExactSequence`: it validates a fixed-length sequence position by position.
```python
from probatio import Schema, ExactSequence, Invalid
pair = Schema(ExactSequence([str, int]))
pair(["a", 1]) # ['a', 1]
try:
pair(["a"])
except Invalid as err:
print(err) # expected a sequence of 2 items
```
For bounds rather than positions, compose with `All`: the list schema validates the elements, then `Length` (or `NonEmpty`) checks the size of the result.
```python
from probatio import Schema, All, Length
tags = Schema(All([str], Length(min=1, max=3)))
tags(["a", "b"]) # ['a', 'b']
```
An empty list then fails the bound, not the element rule:
```python
from probatio import Schema, All, NonEmpty
Schema(All([str], NonEmpty()))([]) # value must not be empty
```
## Nesting
[Section titled “Nesting”](#nesting)
A sequence schema is a value like any other, so lists nest inside dicts and dicts inside lists, and error paths follow the nesting all the way down:
```python
from probatio import Schema, Required, Invalid
schema = Schema(
{
Required("servers"): [
{Required("host"): str, "ports": [int]},
],
}
)
schema({"servers": [{"host": "a", "ports": [80, 443]}]})
# {'servers': [{'host': 'a', 'ports': [80, 443]}]}
try:
schema({"servers": [{"host": "a", "ports": [80, "x"]}]})
except Invalid as err:
print(err) # expected int at 'servers[0].ports[1]'
print(err.path) # ['servers', 0, 'ports', 1]
```
## Where to next
[Section titled “Where to next”](#where-to-next)
* [The validation model](/guides/validation-model/): the mental model behind every schema shape.
* [Dict schemas and markers](/guides/dict-schemas-and-markers/): the mapping side, with `Required`, defaults, and the extra-key policy.
* [Built-in validators](/guides/validators/): `Unique`, `Sorted`, `Unordered`, `EnsureList`, and the rest of the collection toolbox.
# Testing with pytest-probatio
> Use a probatio schema as a pytest assertion matcher, with path-precise failures.
`pytest-probatio` is a small companion package that lets a probatio schema stand in as a pytest assertion matcher. A schema reads as the expected shape, and a mismatch is explained by probatio’s path-precise errors instead of a bare `assert`. It is handy for asserting the shape of API responses and other structured data in tests.
It ships as a separate distribution, so the core library stays dependency-free, and it is released lock-step with probatio at the same version.
## Installation
[Section titled “Installation”](#installation)
```bash
pip install pytest-probatio
```
The plugin registers itself with pytest; no configuration is needed.
## Matchers
[Section titled “Matchers”](#matchers)
Two matchers are exposed. The schema goes on one side of the comparison and the data on the other:
```python
from pytest_probatio import Exact, Partial
from probatio import Port
def test_response(response):
# Exact: an extra key makes it unequal.
assert response == Exact({"name": str, "port": Port()})
# Partial: extra keys are allowed under ==.
assert response == Partial({"name": str})
# The <= operator relaxes Exact to a partial match too.
assert Exact({"name": str}) <= response
```
The right-hand schema may be any probatio schema: a type, a validator, a nested dict, markers, and so on. With `Exact`, `==` requires no extra keys and `<=` allows them; `Partial` allows extra keys under `==`.
## Readable failures
[Section titled “Readable failures”](#readable-failures)
When the data does not match, pytest’s assertion rewriting prints each error by its path through the data, using probatio’s errors:
```text
data does not match the probatio schema (==):
port: expected a port number between 1 and 65535
```
So a failing `assert response == Exact(...)` points at the exact offending value rather than just reporting that the two are not equal.
GitHub annotations
The explanation is plain pytest assertion output, so it flows straight into [`pytest-github-actions-annotate-failures`](https://github.com/pytest-dev/pytest-github-actions-annotate-failures). Install that plugin in CI and a failed schema match becomes a GitHub annotation on the test, listing each offending field by its path, with no extra setup here.
## Reuse a schema across tests
[Section titled “Reuse a schema across tests”](#reuse-a-schema-across-tests)
A schema is data, so define the expected shape once and assert it across as many tests as you like. This is where the matcher earns its keep: one schema, many tests, and each failure still points at the offending field. A `Schema` you already use elsewhere (a production config or response schema) works as the matcher just as well as a fresh one.
```python
from pytest_probatio import Exact, Partial
from probatio import Schema, All, Email, Range
# The shape of a user, defined once and shared by every test below.
USER = Schema(
{
"id": All(int, Range(min=1)),
"name": str,
"email": Email(),
}
)
def test_create_user_returns_the_created_user(api):
response = api.post("/users", json={"name": "Ada", "email": "ada@example.com"})
assert response.status_code == 201
assert response.json() == Exact(USER)
def test_get_user_returns_the_user(api):
assert api.get("/users/1").json() == Exact(USER)
def test_user_list_wraps_users_in_a_page(api):
# The same USER schema, composed into a larger shape.
page = Exact({"users": [USER], "total": Range(min=0)})
assert api.get("/users").json() == page
def test_user_detail_may_carry_extra_fields(api):
# Partial: the response must contain a valid user; extra fields are allowed.
assert api.get("/users/1?expand=true").json() == Partial(USER)
```
The `id` field is `All(int, Range(min=1))` rather than a bare `Range`: `Range` alone accepts any comparable value, like `1.5`, so pin the type first. Here `api` is your application’s test client, an ordinary pytest fixture you provide. The schema can be injected the same way: a fixture that returns `USER` works just as well as the module-level constant. When a response is wrong, the failure names the exact field, even inside the composed list:
```text
data does not match the probatio schema (==):
users[0].email: expected an email address
```
## Asserting a rejection with `raises`
[Section titled “Asserting a rejection with raises”](#asserting-a-rejection-with-raises)
The matchers assert that data *fits* a schema. To assert that a schema *rejects* something, the core library ships a `raises` context manager, kept for drop-in compatibility with voluptuous. It comes from `probatio` itself, not from the plugin, so it works with or without pytest:
```python
from probatio import Schema, MultipleInvalid, raises
schema = Schema({"port": int})
with raises(MultipleInvalid):
schema({"port": "nope"})
```
The optional second argument asserts the exact `str()` of the error, and `regex=` matches it with `re.search` instead:
```python
with raises(MultipleInvalid, "expected int at 'port'"):
schema({"port": "nope"})
with raises(MultipleInvalid, regex=r"expected int"):
schema({"port": "nope"})
```
`pytest.raises` works too, of course. Reach for probatio’s `raises` when porting voluptuous tests that already use it, or when you want the exact-match message check without the pytest import.
## Why a separate package
[Section titled “Why a separate package”](#why-a-separate-package)
The core `probatio` library has no required dependencies. A pytest plugin needs pytest and registers a plugin entry point, which is a test-framework concern, so it lives in its own distribution. It shares probatio’s repository, so the two are developed and released together.
# Troubleshooting
> The common surprises when using Probatio, and how to deal with each.
A short list of the things that trip people up, with the symptom first and the fix second. Most of them are points where Probatio matches voluptuous exactly, so the fix is the same one you would reach for there.
## My test asserting the voluptuous error string broke
[Section titled “My test asserting the voluptuous error string broke”](#my-test-asserting-the-voluptuous-error-string-broke)
The number-one migration symptom. Probatio renders errors with a dotted path (`expected int at 'server.port'`) where voluptuous rendered `expected int for dictionary value @ data['server']['port']`. That is a deliberate deviation, listed on the [compatibility matrix](/reference/compatibility-matrix/), not a bug to report.
Do not assert on `str(err)`. Assert on the structured attributes, which match voluptuous exactly: `path`, `error_message` (the bare message without the path), and the exception class.
```python
from probatio import Schema, MultipleInvalid
schema = Schema({"server": {"port": int}})
try:
schema({"server": {"port": "nope"}})
except MultipleInvalid as err:
print(err) # expected int at 'server.port'
print(err.errors[0].path) # ['server', 'port']
print(err.errors[0].error_message) # expected int
```
## My extra keys were rejected
[Section titled “My extra keys were rejected”](#my-extra-keys-were-rejected)
A key the schema does not name is rejected by default, matching voluptuous. The first hour with a real payload usually surfaces this. Allow extras schema-wide with `extra=ALLOW_EXTRA`, or per key with the `Extra` catch-all (`{Extra: object}`), which the [dict schemas guide](/guides/dict-schemas-and-markers/) covers in full.
```python
from probatio import Schema, Invalid, ALLOW_EXTRA
strict = Schema({"name": str})
try:
strict({"name": "app", "debug": True})
except Invalid as err:
print(err) # not a valid option at 'debug'
relaxed = Schema({"name": str}, extra=ALLOW_EXTRA)
relaxed({"name": "app", "debug": True}) # {'name': 'app', 'debug': True}
```
## I caught `Invalid` but got a `MultipleInvalid`
[Section titled “I caught Invalid but got a MultipleInvalid”](#i-caught-invalid-but-got-a-multipleinvalid)
A `Schema` call always wraps its failures in `MultipleInvalid`, even when there is only one. `MultipleInvalid` is itself an `Invalid`, so `except Invalid` catches it; the individual failures are in `.errors`.
```python
from probatio import Schema, Invalid, MultipleInvalid
schema = Schema(int)
try:
schema("nope")
except Invalid as err: # catches MultipleInvalid too
print(type(err).__name__) # MultipleInvalid
print(err.errors[0].msg) # expected int
```
To branch on a specific kind of failure (like `RangeInvalid`), inspect `err.errors[0]`, not the `MultipleInvalid` itself.
## `SchemaError` from a wrapped `Self`
[Section titled “SchemaError from a wrapped Self”](#schemaerror-from-a-wrapped-self)
`Self` works as a direct mapping value or list element, and as a branch of a combinator (`Any(Self, "stop")`, `All(Self, ...)`). It does not work inside a plain wrapping validator like `Maybe`, which compiles it outside the recursive context and raises:
```python
from probatio import Schema, Maybe, Self
Schema({"next": Maybe(Self)}) # SchemaError
```
For an optional recursive key, mark the key `Optional`, do not wrap the value:
```python
from probatio import Schema, Optional, Required, Self
schema = Schema({Required("value"): int, Optional("next"): Self})
schema({"value": 1, "next": {"value": 2}}) # {'value': 1, 'next': {'value': 2}}
```
## My custom validator’s `ValueError` message has a `not a valid value:` prefix
[Section titled “My custom validator’s ValueError message has a not a valid value: prefix”](#my-custom-validators-valueerror-message-has-a-not-a-valid-value-prefix)
A plain `ValueError` from a callable is kept, but reported as `not a valid value: `. The reason is preserved; the prefix is not. Raise `Invalid` with your own message to control it exactly, with no prefix:
```python
from probatio import Schema, Invalid
def even(value):
if value % 2:
raise Invalid("must be even")
return value
try:
Schema(even)(3)
except Invalid as err:
print(err.errors[0].msg) # must be even
```
## `Remove` dropped my key, but a bad value still errored
[Section titled “Remove dropped my key, but a bad value still errored”](#remove-dropped-my-key-but-a-bad-value-still-errored)
`Remove` validates the value before dropping it. Only a value that validates is removed; a value that fails its schema is reported, the same as any other key. That is voluptuous behavior, not a Probatio quirk.
```python
from probatio import Schema, Remove
schema = Schema({"keep": int, Remove("drop"): str})
schema({"keep": 1, "drop": "x"}) # {'keep': 1}
```
## `data is nested too deeply for this recursive schema`
[Section titled “data is nested too deeply for this recursive schema”](#data-is-nested-too-deeply-for-this-recursive-schema)
A recursive `Self` schema has a depth guard, so cyclic or pathologically deep data raises a clean `Invalid` instead of crashing the interpreter with a `RecursionError`. The limit scales with the interpreter’s recursion limit. Real data is nowhere near it; if you knowingly need deeper, raise the interpreter’s recursion limit with `sys.setrecursionlimit()` before validating.
## A difference from voluptuous that is not documented
[Section titled “A difference from voluptuous that is not documented”](#a-difference-from-voluptuous-that-is-not-documented)
Probatio targets behavioral compatibility, and the handful of intentional differences are listed on the [compatibility page](/getting-started/compatibility/) and the [compatibility matrix](/reference/compatibility-matrix/). Anything else is a bug; please [open an issue](https://github.com/frenck/probatio/issues) with a small reproduction.
# Schemas from TypedDicts
> Generate a validating schema straight from a TypedDict, with the field annotations driving the validators and the typed dict coming back out.
A `TypedDict` describes the shape of a plain dict: which keys it has and what each value is typed as. `TypedDictSchema` reads that shape and builds a schema from it, so you validate against the keys you already declared instead of writing the schema twice.
The annotation engine is shared with [dataclasses](/guides/dataclasses/), but the two are different tools for different jobs. A dataclass schema validates a mapping and hands you back a constructed instance. A TypedDict is a plain dict at runtime, so a TypedDict schema validates the mapping and returns it unchanged, just typed. Dict in, dict out, at no construction cost.
## The basics
[Section titled “The basics”](#the-basics)
Pass a `TypedDict` type to `TypedDictSchema`. The validated mapping comes back as is, and your type checker sees it as the `TypedDict`, so `result["port"]` is an `int` to the type checker with nothing extra at runtime.
```python
from typing import TypedDict
from probatio import TypedDictSchema
class Config(TypedDict):
port: int
host: str
schema = TypedDictSchema(Config)
schema({"port": 8080, "host": "nas"}) # {'port': 8080, 'host': 'nas'}
```
By default every key is required, and a missing one is reported like any other validation error:
```python
from typing import TypedDict
from probatio import TypedDictSchema
class Config(TypedDict):
port: int
host: str
TypedDictSchema(Config)({"port": 8080}) # required key not provided at 'host'
```
The keys are closed: an unknown key is rejected, the same as the default dict schema. Pass `extra=ALLOW_EXTRA` (keyword-only) to keep unknown keys instead.
## Required and optional keys
[Section titled “Required and optional keys”](#required-and-optional-keys)
A `TypedDict` carries its own notion of which keys are required, and the schema honors it. A `total=False` class makes every key optional; `Required` and `NotRequired` set it per key. An optional key that is absent is simply left out of the result, no default is invented. Because requiredness comes from the `TypedDict` itself, there is no `required` argument to override it, unlike `DataclassSchema`.
```python
from typing import TypedDict, NotRequired
from probatio import TypedDictSchema
class Server(TypedDict):
name: str
port: NotRequired[int]
schema = TypedDictSchema(Server)
schema({"name": "nas", "port": 22}) # {'name': 'nas', 'port': 22}
schema({"name": "nas"}) # {'name': 'nas'}
```
A `total=False` class flips the default, so nothing is required:
```python
from typing import TypedDict
from probatio import TypedDictSchema
class Partial(TypedDict, total=False):
a: int
b: str
TypedDictSchema(Partial)({}) # {}
```
## Annotations drive the validators
[Section titled “Annotations drive the validators”](#annotations-drive-the-validators)
Each field’s annotation becomes a validator. The mapping is deep, not just the container type: a parameterized generic keeps its element types, a union with `None` becomes `Maybe`, and a nested `TypedDict` (or dataclass) recurses into its own schema. The full table is the same one [dataclasses use](/guides/dataclasses/#annotations-drive-the-validators).
```python
from typing import TypedDict
from probatio import TypedDictSchema
class Config(TypedDict):
port: int
host: str
class Service(TypedDict):
name: str
config: Config
TypedDictSchema(Service)({"name": "web", "config": {"port": 80, "host": "nas"}})
# {'name': 'web', 'config': {'port': 80, 'host': 'nas'}}
```
For anything beyond the type check (a length, a range, a pattern), the same two options as dataclasses apply: pass `additional_constraints`, a map from key to a validator that runs after the type check, or put the rule on the field with `Annotated`.
```python
from typing import TypedDict
from probatio import TypedDictSchema, Length
class Config(TypedDict):
port: int
host: str
schema = TypedDictSchema(Config, {"host": Length(min=2)})
schema({"port": 8080, "host": "nas"}) # {'port': 8080, 'host': 'nas'}
```
With `Annotated`, the rule lives on the field itself, next to the type:
```python
from typing import Annotated, TypedDict
from probatio import TypedDictSchema, Range
class Net(TypedDict):
port: Annotated[int, Range(min=1, max=65535)]
TypedDictSchema(Net)({"port": 8080}) # {'port': 8080}
```
## The functional form
[Section titled “The functional form”](#the-functional-form)
`create_typeddict_schema(typeddict_type, additional_constraints=None)` builds the same schema without the class wrapper. It returns a plain `Schema`, so it does not carry the `TypedDict` type for the type checker the way `TypedDictSchema` does.
```python
from typing import TypedDict
from probatio import create_typeddict_schema
class Config(TypedDict):
port: int
host: str
create_typeddict_schema(Config)({"port": 8080, "host": "nas"})
# {'port': 8080, 'host': 'nas'}
```
## What carries over from dataclasses
[Section titled “What carries over from dataclasses”](#what-carries-over-from-dataclasses)
The two share the annotation engine, so the [dataclasses guide](/guides/dataclasses/) covers the rest, and it all behaves the same with a `TypedDict`:
* The [annotation mapping table](/guides/dataclasses/#annotations-drive-the-validators).
* [Coercing a field’s type](/guides/dataclasses/#coercing-a-fields-type) with an `Annotated` hint.
* [Recursive](/guides/dataclasses/#recursive-dataclasses) and mutually recursive types, with the same depth guard.
* [Discriminated unions](/guides/dataclasses/#discriminated-unions) over a shared literal tag.
* [Key facets on fields](/guides/dataclasses/#key-facets-on-fields): `Key(...)` in the `Annotated` metadata to redact, alias, group, forbid, or override the presence of a field.
* [Extra keys propagating into nested schemas](/guides/dataclasses/#extra-keys-all-the-way-down): `extra` applies at every level, including a nested `TypedDict`, and `Key(extra=...)` pins a different policy for one field’s subtree.
The one difference is the result: a dataclass schema constructs an instance, a TypedDict schema returns the validated dict. Because nothing is constructed, a `TypedDict` also accepts `Key(forbidden=True)`/`Key(remove=True)` fields with no default (they simply shape the validated dict), where a dataclass would need one.
## Speed
[Section titled “Speed”](#speed)
The [compiled engine](/guides/compiled-schemas/) applies here too: `TypedDictSchema` takes the same `compile=True` flag and `.compile()` method, and under the default policy a hot schema compiles itself. The trusted escape hatch also carries over: `TypedDictSchema.construct(data)` skips validation and, since nothing is constructed, returns the dict unchanged (see [trusted construction](/guides/dataclasses/#trusted-construction-without-validation)).
```python
from typing import TypedDict
from probatio import TypedDictSchema
class Point(TypedDict):
x: int
y: int
POINT = TypedDictSchema(Point).compile()
POINT({"x": 1, "y": 2}) # {'x': 1, 'y': 2}
```
## Limits
[Section titled “Limits”](#limits)
The value is not coerced between container types: a list stays a list even where the annotation says `tuple`.
# The validation model
> How Probatio turns a schema into a validator, and what can go inside one.
A schema is data that describes data. You write it with plain Python, Probatio compiles it once into a fast validator, and then you call that validator with a value. This page is the mental model the rest of the guides build on.
## A schema is a value
[Section titled “A schema is a value”](#a-schema-is-a-value)
You do not call a builder API to construct a schema. You hand `Schema` an ordinary Python object, and its shape *is* the rule:
```python
from probatio import Schema
Schema(int)(42) # 42
Schema("on")("on") # 'on'
Schema([int])([1, 2, 3]) # [1, 2, 3]
Schema({"a": int})({"a": 1}) # {'a': 1}
```
Each kind of object means something:
| You write | It means |
| ---------------------- | ------------------------------------------------------------------------------------------------------------- |
| a type (`int`) | the value must be an instance of that type |
| a literal (`"on"`) | the value must equal that literal |
| a `dict` | a mapping, each key and value validated by its own schema |
| a `list`/`tuple`/`set` | a sequence, each item matching one of the element schemas (see [sequence schemas](/guides/sequence-schemas/)) |
| a callable | called with the value; it returns the result or raises |
| a nested `Schema` | validated as its own schema |
Because a schema is just data, schemas compose: a dict can hold lists of nested dicts, and any value position can be a validator like `All` or `Coerce`.
One type is not a bare `isinstance`: `float` honors the PEP 484 numeric tower, so `Schema(float)(5)` accepts the `int` and returns `5.0` (`bool` excluded). This is a deliberate deviation from voluptuous, which rejects it. See [ADR-017](https://github.com/frenck/probatio/blob/main/adr/017-numeric-tower-for-float.md).
## Compile once, validate many
[Section titled “Compile once, validate many”](#compile-once-validate-many)
`Schema(...)` does its analysis up front. Building the schema is the expensive step; calling it is cheap, so build a schema once and reuse it:
```python
from probatio import Schema, Required
schema = Schema({Required("name"): str}) # compiled here
for record in [{"name": "a"}, {"name": "b"}]:
schema(record) # cheap, repeated calls
```
Keep schemas at module scope, or build them once in a constructor, rather than rebuilding inside a hot loop. The numbers, and the opt-in compiled mode that goes further, live in the [performance reference](/reference/performance/) and the [compiled schemas guide](/guides/compiled-schemas/).
## Validation returns a new value
[Section titled “Validation returns a new value”](#validation-returns-a-new-value)
A validator returns the validated result; it does not mutate the input. The result can differ from the input, because validators may normalize: `Coerce` converts types, defaults fill in absent keys, and string transforms rewrite the value.
```python
from probatio import Schema, Coerce
schema = Schema({"port": Coerce(int)})
data = {"port": "443"}
schema(data) # {'port': 443}
data # {'port': '443'} (unchanged)
```
## Failure is an exception, not a return value
[Section titled “Failure is an exception, not a return value”](#failure-is-an-exception-not-a-return-value)
A valid value comes back from the call. An invalid one raises, so there is no “is it valid” flag to check; you validate inside a `try`, or let the error propagate.
```python
from probatio import Schema, Invalid
schema = Schema(int)
try:
schema("not a number")
except Invalid as err:
print(err) # expected int
```
Everything Probatio raises for bad *data* is an `Invalid` (or a `MultipleInvalid` collecting several). A broken *schema definition*, by contrast, raises `SchemaError`, because that is a programming mistake, not bad input. See [error handling](/guides/error-handling/) for the full picture.
## Order matters
[Section titled “Order matters”](#order-matters)
Within `All`, validators form a pipeline: each receives the previous result, left to right. `All(Strip, Length(min=1))` trims whitespace before it measures, so a string of only spaces fails; swapped, the untrimmed string would slip through. Marker defaults and group rules also have defined timing. Where order is significant, the guides call it out, and it matches voluptuous.
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Dict schemas and markers](/guides/dict-schemas-and-markers/): the mapping rules in depth.
* [Combinators](/guides/combinators/): `All`, `Any`, `Union`, and friends.
* [Built-in validators](/guides/validators/): the toolbox.
# Built-in validators
> A tour of the validators Probatio ships with, grouped by what they check.
A validator is a callable that takes a value and returns it (possibly normalized), or raises `Invalid`. Probatio ships a stack of them so you rarely have to write your own. This page walks the toolbox by category, with a short runnable example for nearly all of them.
These are the leaf validators. The combinators that compose them (`All`, `Any`, `Union`, `SomeOf`) live in the [combinators guide](/guides/combinators/), and the dictionary markers (`Required`, `Optional`, and friends) in the [dict schemas and markers guide](/guides/dict-schemas-and-markers/). You will see those names in passing here; this page is about the leaves. For the whole toolbox at a glance, including the transformers that change a value rather than check it, see [Built-ins by role](/reference/builtins-by-role/).
Note
Every validator is importable straight from `probatio`, for example `from probatio import Range, Coerce`.
Two naming conventions run through the toolbox, so a name tells you what it does with the value. A validator named `As` (`AsDatetime`, `AsTimedelta`) validates the input and returns a real object of that type; its plain sibling (`Datetime`, `Duration`) checks the same input but returns it unchanged, string-in string-out, for voluptuous compatibility. A validator named `From` (`FromEpoch`, `FromPercentage`, `FromJSONString`) builds a value from that source form. So `As` names the type you get out, `From` names the shape that goes in.
## Type and value
[Section titled “Type and value”](#type-and-value)
These check what a value is, or what it equals.
`Coerce` converts with `type(value)` and reports a clean `CoerceInvalid` when the conversion fails. `Boolean()` reads common truthy and falsy strings (`"yes"`, `"on"`, `"off"`, and such) as a `bool`; it is a factory, so call it. `Literal` and `Equal` pin a value to a constant. `In` and `NotIn` test membership, `Contains` tests the other direction, and `Match` runs a regular expression.
```python
from probatio import Schema, Coerce, Boolean, Literal, Equal
Schema(Coerce(int))("42") # 42
Schema(Boolean())("on") # True
Schema(Boolean())("off") # False
Schema(Literal("on"))("on") # 'on'
Schema(Equal(3))(3) # 3
```
```python
from probatio import Schema, In, NotIn, Contains, Match
Schema(In(["red", "green", "blue"]))("green") # 'green'
Schema(NotIn(["root", "admin"]))("frenck") # 'frenck'
Schema(Contains(2))([1, 2, 3]) # [1, 2, 3]
Schema(Match(r"^[a-z]+$"))("probatio") # 'probatio'
```
`Coerce` raises when the value cannot be converted:
```python
from probatio import Schema, Coerce
Schema(Coerce(int))("not a number")
```
## Numbers
[Section titled “Numbers”](#numbers)
Numeric bounds and conversions.
`Range` checks that a value falls within `min` and `max`. Both endpoints are inclusive by default; set `min_included=False` or `max_included=False` for an open bound. `Clamp` pins a value into the range instead of failing. `Number` validates a numeric string, optionally checking its precision (significant digits) and scale (decimal places).
```python
from probatio import Schema, Range, Clamp, Number
Schema(Range(min=1, max=10))(5) # 5
Schema(Range(min=0, max=1, max_included=False))(0.5) # 0.5
Schema(Clamp(min=0, max=100))(150) # 100
Schema(Number(precision=4, scale=2))("12.34") # '12.34'
```
`Positive`, `Negative`, and `NonNegative` are sign conveniences over `Range`. `Byte` (0 to 255) and `SmallFloat` (0 to 1) are bounded ranges, and `Latitude` (-90 to 90) and `Longitude` (-180 to 180) bound geographic coordinates. `MultipleOf` requires an integer multiple. `Percentage` validates a number, a bare numeric string, or a `"NN%"` string in 0 to 100 and returns it unchanged; `FromPercentage` parses it to a float. None of these coerce the type otherwise; wrap with `Coerce` for that (`All(Coerce(int), NonNegative())`).
```python
from probatio import (
Schema,
Positive,
MultipleOf,
Percentage,
FromPercentage,
Latitude,
Longitude,
)
Schema(Positive())(5) # 5
Schema(MultipleOf(15))(45) # 45
Schema(Percentage())("80%") # '80%'
Schema(FromPercentage())("80%") # 80.0
Schema(Latitude())(52.37) # 52.37
Schema(Longitude())(4.9) # 4.9
```
A small family of arithmetic mutators rescales a number, the readable stand-in for the `Coerce(lambda v: v / 1000)` calculations that pile up on device and sensor fields. `Multiply` and `Divide` apply a gain or a unit conversion, `Offset` shifts a value, and `Round` tidies the result (to `ndigits` decimals, or to the nearest integer by default). An integer operand on an `int` keeps an `int`; a `Divide` or a fractional operand yields a `float`. Chain them in an `All` when a calculation takes more than one step.
```python
from probatio import Schema, All, Multiply, Divide, Offset, Round
Schema(Divide(1000))(5000) # 5.0 (milliunits to units)
Schema(Multiply(0.1))(50) # 5.0 (a gain)
Schema(Offset(50))(100) # 150 (a shift, stays an int)
Schema(Round(2))(5.126) # 5.13
Schema(All(Offset(-273.15), Round(2)))(300) # 26.85 (Kelvin to Celsius, chained)
```
`Remap` linearly maps a number from one range onto another, the Arduino `map()`. It is the workhorse for a raw reading: a 0 to 1023 ADC or a 0 to 255 byte to a percentage, or an RSSI in dBm to a signal percentage. It does not clamp, so wrap it in `Clamp` to bound the result, and add `Round` to tidy it.
```python
from probatio import Schema, All, Remap, Clamp, Round
Schema(All(Remap(0, 255, 0, 100), Round(1)))(128) # 50.2 (a byte as a percentage)
Schema(All(Remap(-100, -50, 0, 100), Clamp(0, 100)))(-70) # 60.0 (RSSI dBm to percent)
```
`Scale` folds the whole affine transform into one call, `value * factor / divisor + offset`, optionally rounded, when a single field does several steps at once.
```python
from probatio import Schema, Scale
Schema(Scale(divisor=1000))(5000) # 5.0 (milliunits to units)
Schema(Scale(10))(5) # 50 (integer gain, stays an int)
Schema(Scale(100, divisor=255, round=1))(128) # 50.2 (a byte as a percentage)
Schema(Scale(offset=-273.15, round=2))(300) # 26.85 (Kelvin to Celsius)
```
A few finishers round out the number work. `Round` takes the nearest value; `RoundUp` and `RoundDown` take the ceiling and floor (to a whole integer). `Snap` rounds to the nearest multiple of a step, for a setpoint. `Abs` is the magnitude, and `Modulo` wraps a value, so `Modulo(360)` keeps a heading in 0 to 359 (`MultipleOf`, by contrast, validates divisibility rather than transforming).
```python
from probatio import Schema, RoundUp, RoundDown, Snap, Abs, Modulo
Schema(RoundUp())(4.1) # 5
Schema(RoundDown())(4.9) # 4
Schema(Snap(0.5))(1.2) # 1.0 (nearest half)
Schema(Abs())(-3) # 3
Schema(Modulo(360))(370) # 10 (wrap a heading)
```
## Collections and structure
[Section titled “Collections and structure”](#collections-and-structure)
These shape sequences, sets, and objects.
`Length` bounds the length of a sized value. `Unique` requires distinct items. `Set` turns an iterable into a `set` (handy because JSON has no set type, so sets arrive as lists). `ExactSequence` validates a fixed-length sequence position by position, `Unordered` does the same but lets the items appear in any order. `Object` validates an object’s attributes like a mapping. `Maybe` allows `None`, otherwise it defers to the wrapped validator. `EnsureList` normalizes the common “one value or a list of them” shape: a scalar becomes a single-item list, a list passes through, and `None` becomes an empty list. `NonEmpty` requires a sized value (string, list, mapping) to not be empty.
```python
from probatio import Schema, Length, Unique, ExactSequence, Unordered, Maybe, EnsureList
Schema(Length(min=1, max=3))([1, 2]) # [1, 2]
Schema(Unique())([1, 2, 3]) # [1, 2, 3]
Schema(ExactSequence([str, int]))(["a", 1]) # ['a', 1]
Schema(Unordered([str, int]))([1, "a"]) # [1, 'a']
Schema(Maybe(int))(None) # None
Schema(Maybe(int))(5) # 5
Schema(EnsureList())("one") # ['one']
```
`Maybe` composes with a coercer for the “optional, coerce if present” field: `Maybe(Coerce(float))` leaves `None` alone and coerces anything else, so there is no null check to hand-write:
```python
from probatio import Schema, Maybe, Coerce
Schema(Maybe(Coerce(float)))(None) # None
Schema(Maybe(Coerce(float)))("0.5") # 0.5
```
`Sorted` requires a collection to already be in order (it does not reorder):
```python
from probatio import Schema, Sorted
Schema(Sorted())([1, 2, 3]) # [1, 2, 3]
```
A handful of shapers rewrite a list instead of validating it. `Split` cuts a delimited string into a list (stripping pieces and dropping empties by default), and `Join` is its inverse. `Sort` orders a list and `Dedupe` drops repeats, keeping first-seen order, the transformer siblings of `Sorted` and `Unique`. `First` and `Last` pick an element, and `Without` prunes listed values, so `Without(None)` clears the holes.
```python
from probatio import Schema, Split, Join, Sort, Dedupe, First, Last, Without
Schema(Split(","))("a, b ,c") # ['a', 'b', 'c']
Schema(Join(","))([1, 2, 3]) # '1,2,3'
Schema(Sort())([3, 1, 2]) # [1, 2, 3]
Schema(Dedupe())([1, 2, 1, 3]) # [1, 2, 3]
Schema(First())([1, 2, 3]) # 1
Schema(Last())([1, 2, 3]) # 3
Schema(Without(None, 0))([1, None, 0, 2]) # [1, 2]
```
`Set` returns a real set, so its repr ordering is not stable:
```python
from probatio import Schema, Set
result = Schema(Set())([1, 2, 2, 3])
sorted(result) # [1, 2, 3]
```
`Object` rebuilds an object of the same type after validating its attributes:
```python
from probatio import Schema, Object
class Point:
def __init__(self, x, y):
self.x, self.y = x, y
result = Schema(Object({"x": int, "y": int}))(Point(1, 2))
result.x # 1
result.y # 2
```
## Strings
[Section titled “Strings”](#strings)
Two kinds live here: transforms that rewrite the value, and format validators that check it.
The transforms (`Lower`, `Upper`, `Capitalize`, `Title`, `Strip`) are plain functions, not classes. Use them bare, `Lower`, not `Lower()`. `Replace` substitutes a pattern. The format validators (`Email()`, `Url()`, `FqdnUrl()`) are factories, so you call them to build the validator, matching voluptuous. They avoid backtracking regular expressions, so a crafted input cannot hang them.
```python
from probatio import Schema, Lower, Upper, Capitalize, Title, Strip, Replace
Schema(Lower)("HELLO") # 'hello'
Schema(Upper)("hello") # 'HELLO'
Schema(Capitalize)("hello world") # 'Hello world'
Schema(Title)("hello world") # 'Hello World'
Schema(Strip)(" hi ") # 'hi'
Schema(Replace("-", "_"))("a-b-c") # 'a_b_c'
```
A few more transforms take an argument, so they are classes you call. `CollapseWhitespace` squeezes each run of whitespace to one space (`Strip` only trims the ends). `RemovePrefix` and `RemoveSuffix` strip a known affix when present, and `Truncate` caps the length. Unlike the plain transforms above, these reject a non-string rather than coercing it.
```python
from probatio import Schema, CollapseWhitespace, RemovePrefix, RemoveSuffix, Truncate
Schema(CollapseWhitespace())(" a b c ") # 'a b c'
Schema(RemovePrefix("sensor."))("sensor.temp") # 'temp'
Schema(RemoveSuffix("_raw"))("value_raw") # 'value'
Schema(Truncate(5))("hello world") # 'hello'
```
```python
from probatio import Schema, Email, Url, FqdnUrl
Schema(Email())("me@example.com") # 'me@example.com'
Schema(Url())("https://example.com/path") # 'https://example.com/path'
Schema(FqdnUrl())("https://example.com") # 'https://example.com'
```
`Slug` validates a slug (lowercase alphanumerics with hyphen or underscore separators), returning it unchanged. It checks the shape; it does not slugify arbitrary text (transliteration belongs to a dedicated package, reached with `Coerce`). `IsRegex` checks that the value is itself a compilable regular expression (it validates the pattern, it does not run it).
```python
from probatio import Schema, Slug
Schema(Slug())("my-config_key-2") # 'my-config_key-2'
```
More string checks: the character classes `Alpha`, `Alphanumeric`, `ASCII`, `PrintableASCII`, and `NoWhitespace`; the affix checks `StartsWith` and `EndsWith`; `ByteLength` (UTF-8 bytes, not code points); and `HexColor`.
```python
from probatio import Schema, Alphanumeric, StartsWith, HexColor
Schema(Alphanumeric())("abc123") # 'abc123'
Schema(StartsWith("https://"))("https://example.com") # 'https://example.com'
Schema(HexColor())(
"#FF8800"
) # '#FF8800' (validated, unchanged; use Lower to fold case)
```
Tip
The transforms coerce to a string first, so `Lower(123)` returns `"123"`. If you want to reject non-strings, put a `str` check ahead of the transform with `All`.
## Format and checksum
[Section titled “Format and checksum”](#format-and-checksum)
A handful of validators check a structured format or a checksum, all pure (no network, no extra dependency). Like the network validators below, these are Probatio additions with no voluptuous equivalent: `CreditCard` (the Luhn check), `IBAN` (the ISO 13616 mod-97 check), `DataURI` (an RFC 2397 `data:` URI), and `E164` (a phone number in international format). By default `CreditCard`, `IBAN`, and `E164` canonicalize the input (strip grouping, upper-case the IBAN); pass `normalize=False` to validate and return the value unchanged.
```python
from probatio import Schema, CreditCard, IBAN, E164
Schema(CreditCard())("4242 4242 4242 4242") # '4242424242424242'
Schema(IBAN())("de89 3704 0044 0532 0130 00") # 'DE89370400440532013000'
Schema(E164())("+1 (415) 555-2671") # '+14155552671'
Schema(E164(normalize=False))("+14155552671") # '+14155552671' (unchanged)
```
These check shape, not existence: `CreditCard` confirms the Luhn checksum, not that the card is real, and `E164` confirms the international format, not that the number is dialable (which needs a phone-number database).
## Date and time
[Section titled “Date and time”](#date-and-time)
`Datetime` and `Date` validate a string against a `strptime` format, returning the string unchanged when it parses. `Datetime` defaults to ISO 8601 (`%Y-%m-%dT%H:%M:%S.%fZ`), `Date` defaults to `%Y-%m-%d`. Pass `format=` for anything else.
```python
from probatio import Schema, Datetime, Date
Schema(Datetime())("2026-06-25T10:30:00.000000Z") # '2026-06-25T10:30:00.000000Z'
Schema(Date())("2026-06-25") # '2026-06-25'
Schema(Date(format="%d/%m/%Y"))("25/06/2026") # '25/06/2026'
```
`Time` is the time-of-day sibling, defaulting to `%H:%M:%S`. `Duration` validates a duration (a `timedelta`, a number of seconds as an `int`/`float`/numeric string, a `H:MM[:SS]` string, an ISO 8601 duration like `P1DT2H30M`, or a `timedelta` keyword mapping) and returns the value unchanged; `AsTimedelta` accepts the same forms and returns the parsed `datetime.timedelta`, the same `Datetime`/`AsDatetime` split. An ISO 8601 duration is limited to the fields a `timedelta` can represent (weeks, days, hours, minutes, seconds); a year or a month is rejected, since neither is a fixed length.
For time zones, `TimeZoneInfo` validates an IANA name (for the `ZoneInfo` object itself, wrap the type: `Coerce(zoneinfo.ZoneInfo)`; its constructor takes the name). `TimeZone` validates a fixed UTC offset (`+01:00`, `Z`, `UTC`), with `AsTimezone` as the object-returning sibling that parses the offset into a `datetime.timezone`.
```python
import zoneinfo
from probatio import (
Schema,
Time,
Duration,
AsTimedelta,
TimeZone,
TimeZoneInfo,
AsTimezone,
Coerce,
)
Schema(Time())("14:30:00") # '14:30:00'
Schema(Duration())("1:30:00") # '1:30:00'
Schema(AsTimedelta())("1:30:00") # datetime.timedelta(seconds=5400)
Schema(AsTimedelta())("P1DT2H30M") # datetime.timedelta(days=1, seconds=9000)
Schema(TimeZoneInfo())("Europe/Amsterdam") # 'Europe/Amsterdam'
Schema(Coerce(zoneinfo.ZoneInfo))(
"Europe/Amsterdam"
) # zoneinfo.ZoneInfo(key='Europe/Amsterdam')
Schema(TimeZone())("+01:00") # '+01:00'
Schema(AsTimezone())("+01:00") # datetime.timezone(datetime.timedelta(seconds=3600))
```
`AsDatetime`, `AsDate`, and `AsTime` are the object-returning siblings of `Datetime`, `Date`, and `Time`. They return the parsed `datetime`, `date`, or `time` instead of the original string, and they parse ISO 8601 out of the box, so no `format=` is needed for the common case. Pass `format=` to parse a specific `strptime` layout instead. `AsDatetime` takes `require_timezone=True` to reject a naive result; the ISO default reads the offset, so that needs no extra format. A native object passes straight through, so a `datetime`, `date`, or `time` that a YAML or TOML loader already produced validates without a round-trip (and `AsDate` rejects a `datetime`, since it carries a time a pure date would drop).
```python
from probatio import Schema, AsDatetime, AsDate, AsTime
Schema(AsDate())("2026-06-25") # datetime.date(2026, 6, 25)
Schema(AsTime())("14:30:00") # datetime.time(14, 30)
Schema(AsDatetime())("2026-06-25T10:30:00+02:00")
# datetime.datetime(2026, 6, 25, 10, 30, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200)))
Schema(AsDatetime(format="%d/%m/%Y %H:%M"))("25/06/2026 10:30")
# datetime.datetime(2026, 6, 25, 10, 30)
```
Reach for these when the next step wants a real object instead of a string; `Datetime`/`Date`/`Time` stay string-in, string-out for voluptuous compatibility. Parsing uses the standard library on purpose: a faster backend like ciso8601 accepts a different set of strings and returns a different `tzinfo` type, which would make validation depend on what happens to be installed.
`FromEpoch` reads a Unix timestamp (an `int` or `float`) into a timezone-aware UTC `datetime`. It takes `unit="seconds"` by default or `unit="milliseconds"`. The result is always UTC: a naive, local datetime would depend on the host’s time zone, so the same input would validate to a different moment on a different machine.
```python
from probatio import Schema, FromEpoch
Schema(FromEpoch())(1719571800)
# datetime.datetime(2024, 6, 28, 10, 50, tzinfo=datetime.timezone.utc)
Schema(FromEpoch(unit="milliseconds"))(1719571800000)
# datetime.datetime(2024, 6, 28, 10, 50, tzinfo=datetime.timezone.utc)
```
## Network and identifiers
[Section titled “Network and identifiers”](#network-and-identifiers)
These have no voluptuous equivalent; they are Probatio additions. They validate and return the value unchanged: `IPv4Address`/`IPv6Address`/`IPAddress`/`IPNetwork` check the address or network, `UUID` checks a UUID, `MacAddress` a MAC address, `Hostname` and `Fqdn` a name, and `Port` returns an `int`. When you want the parsed Python object rather than the validated string, wrap the type with `Coerce` (its constructor takes the string), and use `NormalizeMacAddress` for a canonical MAC.
```python
from probatio import Schema, IPAddress, UUID, MacAddress, Port, ULID
Schema(IPAddress())("192.0.2.1") # '192.0.2.1'
Schema(UUID())("12345678-1234-5678-1234-567812345678")
# '12345678-1234-5678-1234-567812345678'
Schema(MacAddress())("AA-BB-CC-DD-EE-FF") # 'AA-BB-CC-DD-EE-FF'
Schema(Port())("8080") # 8080
Schema(ULID())("01ARZ3NDEKTSV4RRFFQ69G5FAV") # '01ARZ3NDEKTSV4RRFFQ69G5FAV'
```
Reach for `Coerce` when you want the object instead of the validated string:
```python
import ipaddress
from uuid import UUID as UUIDType
from probatio import Schema, Coerce
Schema(Coerce(ipaddress.IPv4Address))("192.0.2.1") # IPv4Address('192.0.2.1')
Schema(Coerce(UUIDType))("12345678-1234-5678-1234-567812345678")
# UUID('12345678-1234-5678-1234-567812345678')
```
`NormalizeMacAddress` rewrites a MAC to a canonical form, lowercase and colon-separated by default. Pass `upper=True` or a different `separator` (`""` for bare hex); `MacAddress` itself only validates and returns the input unchanged:
```python
from probatio import Schema, NormalizeMacAddress
Schema(NormalizeMacAddress())("AA-BB-CC-DD-EE-FF") # 'aa:bb:cc:dd:ee:ff'
Schema(NormalizeMacAddress(upper=True))("aa-bb-cc-dd-ee-ff") # 'AA:BB:CC:DD:EE:FF'
Schema(NormalizeMacAddress(separator="-"))("aabbccddeeff") # 'aa-bb-cc-dd-ee-ff'
```
`IPNetwork` accepts host bits (it validates, it does not normalize to the network); `Hostname` takes a bare label, `Fqdn` requires a dotted name:
```python
from probatio import Schema, IPNetwork, Hostname, Fqdn
Schema(IPNetwork())("192.0.2.5/24") # '192.0.2.5/24'
Schema(Hostname())("localhost") # 'localhost'
Schema(Fqdn())("host.example.com") # 'host.example.com'
```
## Encoding
[Section titled “Encoding”](#encoding)
`JSONString` validates that a string is valid JSON (and, with an inner schema, that the decoded value matches it) and returns the string unchanged. `FromJSONString` is the decoding sibling: it parses the string with the standard library’s `json` and returns the decoded value.
```python
from probatio import Schema, JSONString, FromJSONString
Schema(JSONString())('{"a": 1, "b": [2, 3]}') # '{"a": 1, "b": [2, 3]}'
Schema(FromJSONString())('{"a": 1, "b": [2, 3]}') # {'a': 1, 'b': [2, 3]}
Schema(FromJSONString({"port": int}))('{"port": 8080}') # {'port': 8080}
```
`Base64` and `Hex` validate that a string is a valid encoding (they return it unchanged; decode it yourself with `Coerce` if you want the bytes).
```python
from probatio import Schema, Base64, Hex
Schema(Base64())("aGVsbG8=") # 'aGVsbG8='
Schema(Hex())("deadbeef") # 'deadbeef'
```
`HexInt` is the parsing sibling of `Hex`: it reads a hexadecimal string (a leading `0x` is optional) and returns the `int`.
```python
from probatio import Schema, HexInt
Schema(HexInt())("ff") # 255
Schema(HexInt())("0x1A") # 26
```
## Filesystem
[Section titled “Filesystem”](#filesystem)
`IsDir`, `IsFile`, and `PathExists` check a path on disk, with `IsSymlink`, `IsSocket`, `IsFifo`, and `IsBlockDevice` for the special file types. Because they touch the real filesystem, the example below creates a temporary directory and file first.
```python
import os
import tempfile
from probatio import Schema, IsDir, IsFile, PathExists
with tempfile.TemporaryDirectory() as path:
file_path = os.path.join(path, "config.yaml")
open(file_path, "w").close()
Schema(IsDir())(path) # an existing directory
Schema(IsFile())(file_path) # an existing file
Schema(PathExists())(path) # any existing path
```
## Truthiness
[Section titled “Truthiness”](#truthiness)
`IsTrue` requires a truthy value, `IsFalse` a falsy one. Both return the value unchanged.
```python
from probatio import Schema, IsTrue, IsFalse
Schema(IsTrue())(1) # 1
Schema(IsFalse())(0) # 0
```
## Cross-field rules
[Section titled “Cross-field rules”](#cross-field-rules)
A marker annotates one key. These three look at the whole mapping, so you place them after a dict schema with `All`: the dict validates first, then the rule sees the validated mapping.
`RequiredWith` makes keys conditional on a trigger key being present: `RequiredWith("tls", "cert", "key")` requires `cert` and `key` whenever `tls` is in the mapping. `RequiredWithout` is the mirror, requiring keys when a trigger is *absent*: `RequiredWithout("cert", "cert_path")` requires `cert_path` when `cert` is not given. `RequiredIf` keys on a value: `RequiredIf({"auth": "token"}, "token")` requires the `token` key only when `auth` equals `"token"`. When the rule does not fire, nothing is required and the mapping passes through unchanged.
Each takes several triggers (or conditions) with a `mode`. For the presence rules, pass a list of trigger keys with `mode="any"` (the default, any one fires) or `mode="all"` (every trigger must hold). For `RequiredIf`, several conditions combine with `mode="all"` (the default, every condition must match) or `mode="any"`.
```python
from probatio import Schema, All, Optional, RequiredWith
schema = Schema(
All(
{"tls": bool, Optional("cert"): str, Optional("key"): str},
RequiredWith("tls", "cert", "key"),
),
)
schema({"tls": True, "cert": "c", "key": "k"}) # unchanged
schema({}) # unchanged, no trigger
```
`Check` runs an arbitrary predicate over the value with a paired message. A falsy result, or a predicate that raises (a missing key, a wrong type), is reported with that message, so a cross-field rule never leaks a raw exception.
```python
from probatio import Schema, All, Check
schema = Schema(
All(
{"start": int, "end": int},
Check(lambda d: d["start"] < d["end"], "start must be before end"),
),
)
schema({"start": 1, "end": 2}) # unchanged
```
For a plain predicate on a single value, rather than a whole mapping, the `truth` decorator (a voluptuous-compat helper) turns it into a validator; the [custom validators guide](/guides/custom-validators/) covers it.
`AtLeastOne`, `AtMostOne`, `ExactlyOne`, and `AllOrNone` are the key-group presence rules: how many of a set of keys may or must appear. `AtLeastOne("host", "url")` requires one of the keys (or both); `AtMostOne("include", "exclude")` allows one at most; `ExactlyOne("token", "password")` requires one and only one; `AllOrNone("lat", "lon")` requires either all of the keys or none of them. They are the dict-level form of `Inclusive`/`Exclusive`, handy when the constraint reads more naturally as a standalone check than as a marker on each key.
```python
from probatio import Schema, AtLeastOne, AtMostOne, ExactlyOne, AllOrNone
Schema(AtLeastOne("host", "url"))({"host": "nas"}) # unchanged
Schema(AtMostOne("include", "exclude"))({"include": 1}) # unchanged
Schema(ExactlyOne("token", "password"))({"token": "t"}) # unchanged
Schema(AllOrNone("lat", "lon"))({"lat": 1, "lon": 2}) # unchanged
```
A non-mapping is rejected by default, so the rule stands on its own without an enclosing dict schema. Pass `require_mapping=False` to leave a non-mapping untouched when the rule sits inside a larger pipeline. When more than one key conflicts (or, for `AllOrNone`, when a key is missing its partners), each offending key is reported with its own path.
## Transition validators
[Section titled “Transition validators”](#transition-validators)
When you validate an update, some fields must not change. `Immutable` and `WriteOnce` compare the new data against its previous value, which you pass as the call’s `context` (see [call-time context](/guides/custom-validators/)). The same compiled schema then checks every update. `Immutable` rejects any change to a field; `WriteOnce` lets a field be set once (from absent or `None`) and freezes it after. With no context (a first validation) they do nothing.
```python
from probatio import Schema, All, Required, Optional, Immutable, MultipleInvalid
schema = Schema(All({Required("id"): int, Optional("name"): str}, Immutable("id")))
old = {"id": 1, "name": "ada"}
schema({"id": 1, "name": "bob"}, context=old) # name may change, unchanged otherwise
try:
schema({"id": 2}, context=old)
except MultipleInvalid as err:
print(err.errors[0].path) # ['id'], with the message "'id' cannot be changed"
```
## Defaults and messages
[Section titled “Defaults and messages”](#defaults-and-messages)
These do not reject values; they patch them.
`DefaultTo` replaces `None` with a default and passes everything else through. `EmptyToNone` is the near-reverse: an empty string or container (`""`, `[]`, `{}`) becomes `None`, the “empty means unset” shape from config and forms, while a falsy scalar like `0` stays put. `SetTo` ignores the input and always produces a fixed value. `Msg` wraps another validator and swaps its failure message for one you choose, which is the simplest way to make an error read in your own words.
```python
from probatio import Schema, DefaultTo, EmptyToNone, SetTo
Schema(DefaultTo("fallback"))(None) # 'fallback'
Schema(DefaultTo("fallback"))("value") # 'value'
Schema(EmptyToNone())("") # None
Schema(EmptyToNone())(0) # 0
Schema(SetTo(42))("anything") # 42
```
A callable given to `DefaultTo` or `SetTo` is read as a factory, not as the value itself, so `DefaultTo(list)` substitutes a new empty list and two validated values never share one mutable default. This is the same normalization a marker default gets (`Optional("tags", default=list)`).
```python
Schema(DefaultTo(list))(None) # [] (a new list each time, not the list type)
```
`Map` translates a value through a table you supply, like a device status code to a name. It is the bring-your-own-mapping answer to a domain conversion: probatio does not pick the table, so a disputed or app-specific lookup stays yours. A value not in the table is rejected, unless you pass a `default`.
```python
from probatio import Schema, Map
Schema(Map({0: "off", 1: "on", 2: "auto"}))(1) # 'on'
Schema(Map({0: "off"}, default="unknown"))(9) # 'unknown'
```
A fixed `default` folds every miss to one value. To rewrite only the keys you list and leave everything else untouched, pass `default=PASSTHROUGH`. That is the difference between “unknown means this value” and “only touch what I named”, which is what you want when a table just cleans up a few sentinel strings ahead of a stricter validator.
```python
from probatio import Schema, Map, PASSTHROUGH
sentinels = Map({"N.v.t.": None, "n/a": None}, default=PASSTHROUGH)
Schema(sentinels)("N.v.t.") # None
Schema(sentinels)("Personenauto") # 'Personenauto' (unmapped, left as-is)
```
`Msg` rewrites the message on failure:
```python
from probatio import Schema, Msg, Range
Schema(Msg(Range(min=10), "too small"))(5) # raises with: too small
```
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Combinators](/guides/combinators/): compose these with `All`, `Any`, `Union`, and `SomeOf`.
* [Dict schemas and markers](/guides/dict-schemas-and-markers/): `Required`, `Optional`, defaults, and extra-key policy.
* [API reference](/reference/): every name and signature in one place.
# About Probatio
> Why Probatio exists, the state of voluptuous, and who builds it.
Probatio exists because a huge amount of Python leans on [voluptuous](https://github.com/alecthomas/voluptuous) for data validation, and voluptuous has gone quiet. The idea is excellent and the API is loved; what it needs is a maintained home. Probatio is that home: the same public API, written fresh, MIT licensed, and actively developed.
## voluptuous runs a lot of Python
[Section titled “voluptuous runs a lot of Python”](#voluptuous-runs-a-lot-of-python)
Home Assistant validates every integration’s configuration with voluptuous. So does a long tail of libraries and tools across the ecosystem. The premise that made it popular is genuinely good: a schema is just data describing data, built from plain Python types, dicts, lists, and small helpers, then called with a value to validate it. You already know the language; the schema is just more of it.
That reach is also the problem. When the library underneath millions of installations stops moving, every consumer inherits the bugs that never get fixed, the sharp edges that never get filed down, and the Python versions that never get supported.
## The state of validation in Python
[Section titled “The state of validation in Python”](#the-state-of-validation-in-python)
Reaching for validation in Python tends to mean one of a few trades:
* **voluptuous** is the lightweight, schema-is-data choice that Home Assistant and many others standardized on. The model is compact and easy to compose, but the project has seen little movement for years, and several rough edges (a `RecursionError` on deep data, a leaked `TypeError` from a built-in validator, errors reported twice) sit unfixed.
* **mashumaro** is a fast, dataclass-first take focused on serialization and deserialization. It is excellent at what it does, and several ideas in Probatio are borrowed from it with credit (see the architecture decision records). It is a codec library more than a free-form validator, though, so the schema-is-data model is not the problem it set out to solve.
* **pydantic** is the heavyweight, framework choice: a `BaseModel` class that is your schema, type, validator, and serializer at once. It is fast and capable, but it asks you to describe data as pydantic model classes, a different mental model and a migration, not a drop-in. (Probatio also returns typed objects, from your own dataclasses and TypedDicts, without a base class; the [comparison page](/project/comparison/) draws the line.) It has also broken backward compatibility and reshaped its API substantially in the past (the v1 to v2 rewrite), which is not a foundation this project wants to stand on.
* **marshmallow** validates and (de)serializes through declarative schema classes with explicit field objects, built around the load and dump cycle. A solid model, but a class you declare, not data you compose.
* **cerberus** keeps the schema as data, like voluptuous, but as dicts of string rule names rather than live Python objects, so composing and reusing schema pieces means manipulating nested dicts.
* **jsonschema** validates against the JSON Schema specification. The right tool when the contract is a JSON Schema document, less so as a general Python-side validator.
There simply was not a maintained library that kept voluptuous’s schema-is-data feel while fixing the edges and keeping pace with Python. The [comparison page](/project/comparison/) walks through each of these in more depth, including when the other library is the better choice.
## What Probatio is
[Section titled “What Probatio is”](#what-probatio-is)
Probatio is a clean-room reimplementation of voluptuous: the same public API, matched by behavior rather than by copied source ([ADR-001](https://github.com/frenck/probatio/blob/main/adr/001-clean-room-reimplementation-of-voluptuous.md)), so changing `import voluptuous` to `import probatio` keeps existing schemas working. That clean-room rule is what lets it be MIT licensed ([ADR-002](https://github.com/frenck/probatio/blob/main/adr/002-mit-license.md)) and fit anywhere voluptuous did.
On top of the drop-in promise, Probatio fixes the rough edges (clean errors on deep or cyclic data, no leaked exceptions from built-ins, a richer error model), adds tooling voluptuous never had (JSON Schema, OpenAPI, dataclass and field-list codecs), and is pure Python with zero required runtime dependencies. It is also fast: on a representative config schema, roughly two to three times faster than voluptuous interpreted, and more once a hot schema compiles itself. The [performance page](/reference/performance/) has the honest numbers.
The compatibility is measured, not asserted: Home Assistant’s own `config_validation` test suite runs against Probatio (the full suite passing), and so does voluptuous’s own 0.16.0 test suite, with every divergence documented. The [voluptuous compatibility](/reference/compatibility-matrix/) reference lists those documented deviations name by name.
## Who builds it
[Section titled “Who builds it”](#who-builds-it)
Probatio is created and written by **Franck Nijhof**, better known as **Frenck**, a [GitHub Star](https://stars.github.com/profiles/frenck/) and the Home Assistant lead, where he has spent years working with voluptuous-validated configuration at a scale and on a hot path that most projects never see. That experience is where Probatio comes from.
Find more of his work at [frenck.dev](https://frenck.dev) and on GitHub at [@frenck](https://github.com/frenck).
# Architecture
> How Probatio is put together, from errors and markers up to the voluptuous compatibility shim.
This page is a map, not an essay. It walks the design in layers, bottom to top, so you know where a given piece of behavior lives. For the reasoning behind the big choices, read the architecture decision records in [`adr/`](https://github.com/frenck/probatio/tree/main/adr), starting at [`adr/README.md`](https://github.com/frenck/probatio/blob/main/adr/README.md).
Probatio is pure Python. No native extension, nothing to build at install time. That is a deliberate choice, not a limitation I have not gotten around to yet. The target workload (config validation, such as Home Assistant) runs at load time, not in a hot inner loop, and the interpreted engine already beats voluptuous there. On top of that engine sits an optional compiled fast path: a hot schema generates specialized Python for itself, purely as an accelerator. [ADR-004](https://github.com/frenck/probatio/blob/main/adr/004-single-validation-engine.md) and [ADR-011](https://github.com/frenck/probatio/blob/main/adr/011-opt-in-compiled-schema.md) together tell that story: one set of validation semantics, with speed layered on top of it.
## The layers
[Section titled “The layers”](#the-layers)
### Errors and markers
[Section titled “Errors and markers”](#errors-and-markers)
At the bottom sit the things everything else speaks in.
Errors live in `error.py`. `Invalid` is a single validation failure that carries a `path` to the offending value. `MultipleInvalid` collects several. The semantic subclasses (`TypeInvalid`, `RangeInvalid`, `RequiredFieldInvalid`, and the rest) let callers catch by kind. Keeping the path precise is the whole game: the engine calls back into arbitrary user code constantly, so every layer is careful to report where a value went wrong, not just that it did.
Markers live in `markers.py`. These are dictionary keys that carry intent: `Required`, `Optional`, `Remove`, `Inclusive`, `Exclusive`, and the `Extra` catch-all. Each compares and hashes by its underlying key, so a marker can stand in for the bare key it wraps.
### Errors for humans
[Section titled “Errors for humans”](#errors-for-humans)
On top of the raw errors sits the rendering layer. Every built-in error carries a stable `translation_key` and `placeholders`, drawn from the message catalog in `_messages.py`, so a consumer can present a failure in its own words or its own language instead of parsing a finished string. `humanize.py` renders the default human-facing output: dotted paths, “did you mean …?” suggestions, and `Secret` redaction. [ADR-015](https://github.com/frenck/probatio/blob/main/adr/015-structured-errors-and-localization.md) covers why the structured data is the API and the English text is just one rendering of it.
### The engine
[Section titled “The engine”](#the-engine)
`_engine.py` is the interpreted core. A schema is compiled once into a callable, then run per validation. Think of it like `re.compile`: you pay the build cost once, then validate many values cheaply.
The engine owns the mapping and sequence validators, where the awkward voluptuous semantics live: marker precedence, literal keys before type keys, the extra-key policy, and the exact error paths. There is one set of validation semantics, and this engine is it. [ADR-004](https://github.com/frenck/probatio/blob/main/adr/004-single-validation-engine.md) explains why an earlier second engine was removed: two full implementations must stay behaviorally identical on every corner case, and that equivalence tax rots.
[ADR-011](https://github.com/frenck/probatio/blob/main/adr/011-opt-in-compiled-schema.md) revisits that decision with a design that avoids the tax. `_codegen.py` generates a flat, success-path-only function for a schema’s exact shape, and `_compile_policy.py` decides when: under the default `AUTO` policy a schema compiles itself only once it has proven hot. The generated code handles only the happy path. On any failure (a missing key, a type mismatch, a validator raising) it bails to the interpreted engine, which produces every error, path, and ordering. So compiling changes speed, never behavior, and the semantics still live in one place. The knobs are in the [compiled schemas guide](/guides/compiled-schemas/).
One small optimization stays: validators can declare `__probatio_safe__` to promise they only raise `Invalid`, and the engine then calls them without the extra guard that wraps stray `ValueError`. It is a contract, not a second engine.
### The Schema compiler
[Section titled “The Schema compiler”](#the-schema-compiler)
`schema.py` is the front door. `Schema(...)` takes a declarative definition (a type, a literal, a callable, a dict, a list, a nested schema, or any validator) and compiles it into the engine callable. Calling the schema validates a value and returns the normalized result, or raises.
This is the compile-once-then-validate model in one object. Build the schema when your program starts, call it on every value after that.
### Dataclass and TypedDict schemas
[Section titled “Dataclass and TypedDict schemas”](#dataclass-and-typeddict-schemas)
`dataclass_schema.py` builds schemas from type annotations instead of dict literals. `create_dataclass_schema` and `create_typeddict_schema` read a class’s fields, infer `Required` and `Optional` from defaults, and turn each annotation (including `Annotated[int, Range(...)]`) into a schema fragment through a shared annotation engine. `DataclassSchema` validates a dict and constructs the instance in one step. [ADR-013](https://github.com/frenck/probatio/blob/main/adr/013-markers-on-annotated-fields.md) covers the field-metadata spec that lets an `Annotated` field carry marker facets like `Secret` and `Alias`.
### The probatio decorator
[Section titled “The probatio decorator”](#the-probatio-decorator)
`decorator.py` is the same annotation engine pointed at a function signature. `@probatio` reads each parameter’s annotation, validates the arguments before the body runs, and works on coroutine functions too ([ADR-014](https://github.com/frenck/probatio/blob/main/adr/014-annotation-driven-argument-decorator.md)). The older `validate` decorator (schemas named by hand, sync-only) stays for the voluptuous drop-in.
### The validators
[Section titled “The validators”](#the-validators)
`validators/` holds the building blocks you compose schemas from, one module per group: combinators (`All`, `Any`, `Union`, `SomeOf`), comparison and membership (`Range`, `In`, `Length`, `Equal`), coercion (`Coerce`, `Boolean`), conditional presence (`RequiredWith`, `ExactlyOne`), encodings (`Base64`, `JSONString`), checksummed formats (`CreditCard`, `IBAN`), identifiers (`UUID`, `ULID`, `MacAddress`), network (`IPAddress`, `Hostname`, `Port`), strings (`Email`, `Url`, the case transforms), structural (`ExactSequence`, `Unique`, `Maybe`), temporal (`Datetime`, `Date`, `Duration`), transitions between an old and a new payload (`Immutable`, `WriteOnce`), predicates, and the `validate` decorator. They share a small base in `validators/_base.py`.
### The codecs
[Section titled “The codecs”](#the-codecs)
`codecs/` converts schemas to and from other schema languages, for the constructs that map cleanly: JSON Schema (`jsonschema.py`), OpenAPI (`openapi.py`), and a plain field-list serialization (`fields.py`). A custom serializer returns the `UNSUPPORTED` sentinel to defer a construct it does not handle to the built-in logic.
`from_json_schema` treats its input as untrusted. It refuses a catastrophically backtracking `pattern` and a pathologically deep document, so a hostile schema cannot hang the process or overflow the stack. That guard lives in `codecs/_regex_safety.py`.
### The voluptuous compatibility shim
[Section titled “The voluptuous compatibility shim”](#the-voluptuous-compatibility-shim)
`compat/` is the thin top layer that backs the drop-in promise. It exposes the voluptuous names and signatures so changing the import from `voluptuous` to `probatio` keeps existing schemas working. The plumbing behind it lives in `_vol_shim/`: the voluptuous-shaped modules (`error`, `validators`, `schema_builder`, `util`) that `compat.install_as_voluptuous` registers under the `voluptuous` name, so even a dependency that imports voluptuous internals resolves to Probatio. Behavior, not source, is what is matched here. No code was copied ([ADR-001](https://github.com/frenck/probatio/blob/main/adr/001-clean-room-reimplementation-of-voluptuous.md)).
## Where to read next
[Section titled “Where to read next”](#where-to-read-next)
* [`adr/README.md`](https://github.com/frenck/probatio/blob/main/adr/README.md): the index of architecture decision records, with the why behind each major decision.
* The [API reference](/reference/): the public surface, grouped by what each name does.
# Comparison to alternatives
> Where Probatio fits among Python data validation tools, and when another library is the better choice.
Probatio does runtime validation of arbitrary Python data. The schema is data: plain types, dicts, lists, and callables. You build a schema from those pieces, call it with a value, and get back the normalized value or an `Invalid` error with a path to what went wrong. There are no model classes to declare. It is a drop-in reimplementation of [voluptuous](https://github.com/alecthomas/voluptuous), and it is pure Python with no native extension.
That shapes where it fits. Probatio is at home when your data is free-form dicts and lists (config files, request bodies, payloads from another system) and you want the schema to stay data you can build, pass around, and compose at runtime. It can also hand back a typed object: point `DataclassSchema` or `TypedDictSchema` at a dataclass or `TypedDict` you already own, and you get a validated, statically typed result. What it is not is a model *framework*: no base class your types inherit, and validation is all it does, with no bundled serialization or settings. For that, other tools do a better job, and this page says where.
## The short answer
[Section titled “The short answer”](#the-short-answer)
| Pick this when | Reach for |
| ----------------------------------------------------------------------------- | ----------------------------- |
| You use voluptuous today and want a maintained replacement | Probatio |
| Your data is plain dicts and lists, schema as data | Probatio |
| You want validated, typed results from your own dataclasses or TypedDicts | Probatio |
| You want a model framework: one class for validation, serialization, settings | pydantic |
| You want declarative schema classes with load and dump | marshmallow |
| Your schemas are cerberus rule dicts | cerberus, or port to Probatio |
| Your contract is a JSON Schema document | jsonschema |
| You are (de)serializing your own classes | attrs / cattrs |
| You (de)serialize dataclasses fast, in a known format | mashumaro |
| You just need a dict turned into a dataclass, no validation | dacite |
## voluptuous
[Section titled “voluptuous”](#voluptuous)
This is the library Probatio replaces. Same model, same public API: a schema is data, and you validate by calling it. Probatio is the clean-room reimplementation that is actively maintained and MIT licensed. No code was copied from voluptuous; behavior is matched, not source.
If you already run voluptuous, the switch is usually a one-line import change. Read [migrating from voluptuous](/getting-started/migrating-from-voluptuous/) for what carries over and the few intentional differences.
```python
from probatio import Schema, Required, Optional
schema = Schema({Required("name"): str, Optional("age"): int})
schema({"name": "Frenck", "age": 40})
# {'name': 'Frenck', 'age': 40}
```
## pydantic
[Section titled “pydantic”](#pydantic)
Pydantic models your data as classes built on Python type hints, with a native (Rust) core and strong editor and type-checker support. You declare a model class that inherits from pydantic’s `BaseModel`, and that one class is your schema, your type, your validator, and your serializer: typed objects back, with autocompletion, static checking, coercion, and `.model_dump()` all following from the hints.
Probatio hands back typed objects too, which is newer than its reputation suggests: `DataclassSchema` and `TypedDictSchema` validate into a dataclass or `TypedDict` you already own, and both are generic, so the result is statically typed (a `Config`, not `Any`), editor and checker included. The difference is the framework. Pydantic asks your types to be pydantic types, the `BaseModel` base class, and bundles serialization and settings on top; Probatio validates free-form data, or validates into the plain types you already have, and does validation only, you own the serialization. So it is the better fit when your data is plain dicts and lists, when the schema should be data you build and compose at runtime rather than a class you declare, when you want to validate into your own dataclasses without adopting a model framework, or when you are replacing voluptuous.
On raw speed the Rust core is genuinely fast, but the advantage is not uniform, and it is worth being precise about where it comes from. It is largest on heavy coercion and decoding: parsing a stack of ISO datetimes and enums out of JSON, where the native code does real work Probatio does in Python. The cross-library benchmark on the [performance page](/reference/performance/) is exactly that kind of workload, and pydantic v2 sits near the top of it. The advantage shrinks on a schema dominated by your own **custom validators**, because a native core cannot run a Python validator natively; it has to call it across the Rust boundary (pydantic does so efficiently, but it is still a Python call), so the speedup is muted exactly where the work is your code rather than the library’s. “Faster because Rust” holds in the parsing-heavy regime, not everywhere, and Probatio stays pure Python with no native extension to build or install.
Note
The two can live side by side. Reach for pydantic when you want its model framework, serialization, and settings on a `BaseModel`; reach for Probatio to validate free-form data, or to validate into your own plain dataclasses and TypedDicts. Picking per use case is fine.
## marshmallow
[Section titled “marshmallow”](#marshmallow)
`marshmallow` validates and (de)serializes through declarative schema classes: you declare a `Schema` subclass with typed `fields`, and `load`/`dump` turn dicts into validated data (or objects, via `@post_load`) and back. It validates, like Probatio and pydantic, but the schema is a class you declare with explicit field objects, and it is built around the load and dump cycle, with a long-standing ecosystem (Flask, webargs, and the like).
Probatio is the better fit when the schema is data you build and compose at runtime rather than a class you declare, when you validate free-form dicts, or when you are replacing voluptuous. marshmallow fits when you want explicit schema classes and its serialization story. Probatio is also a good deal faster on the cross-library [benchmark](/reference/performance/) (marshmallow does more per field and is pure Python), but speed is not the main axis here; the model is.
## cerberus
[Section titled “cerberus”](#cerberus)
`cerberus` is the other schema-is-data validator. You describe rules in a dict of string rule names (`{"name": {"type": "string", "required": True}}`), hand it to a `Validator`, and check the result. The shared instinct with Probatio is real: the schema is data, not a class hierarchy.
The difference is what that data is made of. A cerberus schema is a small rule language interpreted by the library, so extending it means registering a rule or subclassing the validator. A Probatio schema is live Python objects: plain types, callables, and validators you compose directly, so a custom rule is just a function, and the schema participates in the rest of your code (imports, refactoring, composition) like any other Python value. cerberus is maintained and still ships releases; if its rule dicts fit how you think, it is a fine choice. If you want the schema itself to be Python you compose, Probatio fits better.
## jsonschema
[Section titled “jsonschema”](#jsonschema)
The `jsonschema` library validates data against the [JSON Schema](https://json-schema.org) specification. If your contract already *is* a JSON Schema document, perhaps shared across services or languages, then a JSON Schema validator is the honest fit. That is its job, and it does it.
Probatio is not a JSON Schema validator. It does interoperate, though: it can read a JSON Schema into a Probatio schema and write one back out, for the constructs that map cleanly between the two.
Tip
See [JSON Schema](/guides/json-schema/) for `from_json_schema` and `to_json_schema`, and for which constructs round-trip.
## attrs and cattrs
[Section titled “attrs and cattrs”](#attrs-and-cattrs)
`attrs` defines classes with less boilerplate, and `cattrs` structures and unstructures those classes to and from plain data. That is class (de)serialization: turning a dict into a typed instance and back.
That is a different job from schema-style validation of free-form data. If your goal is to (de)serialize your own classes, attrs and cattrs fit. If your goal is to validate arbitrary incoming data against a schema you describe as data, Probatio fits. They can also pair up: validate the raw data with Probatio, then structure the result into your classes with cattrs.
## mashumaro
[Section titled “mashumaro”](#mashumaro)
`mashumaro` (de)serializes dataclasses to and from dicts and formats like JSON, YAML, and msgpack. It code-generates the conversion per class, so it is fast, and it reads the field types from the dataclass itself. Home Assistant leans on it for exactly that: turning stored or transmitted payloads into typed dataclass instances at speed.
Like attrs and cattrs, that is class (de)serialization driven by your declared types, not schema validation of free-form data. Reach for mashumaro when you own the dataclasses and want them filled from a known format quickly. Reach for Probatio when the data is arbitrary and the schema is the thing you describe and compose at runtime. They pair the same way: validate the incoming data with Probatio, then hand the result to mashumaro to build your dataclasses.
The two are close on the dict-to-dataclass path, which is worth seeing precisely because they do different amounts of work. mashumaro deserializes and largely trusts the declared types; Probatio’s `DataclassSchema` validates every field against its type and then constructs. On a small dataclass, mashumaro builds it in about 0.5 µs and Probatio in about 0.7 µs compiled (about 1.5 µs interpreted, before the schema compiles itself), so compiled Probatio is within roughly 1.4x while actually validating, and the gap all but closes as the field count grows. That is the trade: mashumaro is faster because it trusts the input, Probatio costs a little more because it does not. If the input is wrong (a string where an int belongs), mashumaro may hand you a dataclass whose values do not match their annotations, and Probatio rejects it. Numbers, and how to run them yourself, are on the [Performance](/reference/performance/) page (`just bench-dataclass`).
When the input *is* trusted (your own data round-tripping back, or already validated upstream), Probatio’s opt-in [`DataclassSchema.construct`](/guides/dataclasses/#trusted-construction-without-validation) builds without validating, and on that path it is the fastest in the cross-library benchmark, ahead of mashumaro and cattrs, because it generates a flat constructor per dataclass. The one thing it does not do is decode: a `datetime` field keeps whatever the dict held, where mashumaro converts a string to a `datetime`. So for data that needs decoding, mashumaro is still the right reach; for already-typed trusted data, `construct` wins.
## dacite and dataclasses-json
[Section titled “dacite and dataclasses-json”](#dacite-and-dataclasses-json)
Both turn a dict into a dataclass. `dacite`’s `from_dict` builds an instance from a dict with light type checking and no schema language; `dataclasses-json` adds `from_dict`/`to_dict` (and JSON) to a dataclass through a decorator or mixin. They are the same family as cattrs and mashumaro: dataclass (de)serialization driven by the declared types, not schema validation of free-form data.
Reach for them when you own the dataclasses and want them filled from a dict and do not need full validation. Probatio’s `DataclassSchema` covers the same dict-to-dataclass path with a richer type mapping and real validation, or its opt-in `construct` for trusted input. Among the pure (de)serializers, mashumaro and cattrs are the fast ones; on the cross-library [benchmark](/reference/performance/) `dacite` and `dataclasses-json` are a good deal slower, so the choice between them is about features and ergonomics, not speed.
# Credits
> The work Probatio is built on top of, and who maintains it.
## Standing on voluptuous
[Section titled “Standing on voluptuous”](#standing-on-voluptuous)
Probatio is a clean-room reimplementation of [voluptuous](https://github.com/alecthomas/voluptuous). The API and the behavior it mirrors are the work of voluptuous and its author, Alec Thomas, and that credit is owed plainly. The pleasant idea that a schema is just data describing data comes straight from there.
What Probatio does not do is copy code. It is written fresh and matches voluptuous by behavior, validated against how voluptuous behaves, not by lifting its source. That is what makes the clean MIT license possible. [ADR-001](https://github.com/frenck/probatio/blob/main/adr/001-clean-room-reimplementation-of-voluptuous.md) and [ADR-002](https://github.com/frenck/probatio/blob/main/adr/002-mit-license.md) cover the reasoning.
The [license](/project/license/) page covers the MIT terms and third-party notices in full.
## Ideas borrowed with credit
[Section titled “Ideas borrowed with credit”](#ideas-borrowed-with-credit)
Clean-room does not mean invented in a vacuum. Several ideas in Probatio are borrowed from [mashumaro](https://github.com/Fatal1ty/mashumaro), and the debt is recorded where each decision was made: [ADR-008](https://github.com/frenck/probatio/blob/main/adr/008-type-to-validator-registry.md) (the type-to-validator registry), [ADR-009](https://github.com/frenck/probatio/blob/main/adr/009-call-time-validation-context.md) (the call-time validation context), [ADR-012](https://github.com/frenck/probatio/blob/main/adr/012-trusted-construction-without-validation.md) (trusted construction without validation), and [ADR-013](https://github.com/frenck/probatio/blob/main/adr/013-markers-on-annotated-fields.md) (field metadata on `Annotated` types) all cite its prior art. [pydantic](https://docs.pydantic.dev) earns the same credit in ADR-013: its `Annotated` field spec is the precedent that settled how Probatio’s field metadata works.
## Author
[Section titled “Author”](#author)
Created and written by **Franck Nijhof**, better known as **Frenck**, a [GitHub Star](https://stars.github.com/profiles/frenck/) and the Home Assistant lead. Find more of his work at [frenck.dev](https://frenck.dev) and on GitHub at [@frenck](https://github.com/frenck).
# License
> Probatio is open source under the MIT License.
Probatio is free and open-source software, distributed under the **MIT License**. In short: you may use, copy, modify, and distribute it, including in commercial and closed-source projects, as long as the copyright notice and this permission notice are included. The software is provided “as is”, without warranty.
MIT was a deliberate choice: it lets Probatio fit anywhere voluptuous did, including Home Assistant, without trading a maintenance win for a license headache ([ADR-002](https://github.com/frenck/probatio/blob/main/adr/002-mit-license.md)). The clean-room reimplementation ([ADR-001](https://github.com/frenck/probatio/blob/main/adr/001-clean-room-reimplementation-of-voluptuous.md)) is what makes that license honest: no voluptuous source is copied, so nothing constrains it.
The authoritative copy is [`LICENSE`](https://github.com/frenck/probatio/blob/main/LICENSE) in the repository.
## MIT License
[Section titled “MIT License”](#mit-license)
```text
MIT License
Copyright (c) 2026 Franck Nijhof
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```
## Third-party notices
[Section titled “Third-party notices”](#third-party-notices)
Probatio has no third-party runtime dependencies. It is pure Python and uses only the standard library at runtime, so nothing third-party is bundled in the wheel. The details, including the optional integrations that ship under their own licenses when you install them yourself, are in [`THIRD_PARTY_LICENSES.md`](https://github.com/frenck/probatio/blob/main/THIRD_PARTY_LICENSES.md) in the repository.
# Projects using Probatio
> Where Probatio fits, and who is using it.
Probatio is young. This page tracks projects that use it and, just as usefully, the ecosystems it is built to serve. If you adopt Probatio, please [edit this page on GitHub](https://github.com/frenck/probatio/edit/main/docs/src/content/docs/project/projects.md) and open a pull request to add yourself here.
## Home Assistant
[Section titled “Home Assistant”](#home-assistant)
 
[Home Assistant](https://www.home-assistant.io) has validated every integration’s configuration with Probatio since Core 2026.9, on a hot path hit by millions of installations at startup and on every reload. voluptuous is no longer installed at all: the `voluptuous` name is aliased to Probatio in `sys.modules` at startup, so custom integrations keep working unchanged while the core itself imports Probatio directly.
Probatio tracks voluptuous behavior closely, with documented deviations; how that compatibility is measured, including against Home Assistant’s own test suite, is on the [about page](/project/about/). See the [Home Assistant recipe](/recipes/home-assistant/).
## Libraries
[Section titled “Libraries”](#libraries)
A library that Home Assistant installs and that declares voluptuous pulls in a package Home Assistant then shadows, since the name is aliased to Probatio at startup. Depending on Probatio directly drops the redundant install.
* [ramses-rf](https://github.com/ramses-rf/ramses_rf), an interface for the RAMSES-II RF protocol used by Honeywell-compatible HVAC and CH/DHW systems. It backs [ramses_cc](https://github.com/ramses-rf/ramses_cc).
* [evohome-async](https://github.com/zxdavb/evohome-async), an async client for the Resideo TCC web API, behind Home Assistant’s evohome integration. Switched on `main`, not yet released.
## Where Probatio fits
[Section titled “Where Probatio fits”](#where-probatio-fits)
Probatio was designed as a drop-in successor to voluptuous, so it fits anywhere voluptuous is used today: a maintained library, the same schema-is-data API, and fixes for the rough edges. Home Assistant, above, is the first of those. The following are the other ecosystems that motivated its design.
### ESPHome
[Section titled “ESPHome”](#esphome)
ESPHome validates device configurations with voluptuous, and (at the time of writing) reaches into its internals for friendly errors and speed. Probatio offers those as first-class features (close-match key suggestions, an engine that matches per key in linear time), so a consumer does not have to fork the validator to get them. See [migrating from voluptuous](/getting-started/migrating-from-voluptuous/) for what that move looks like.
### Anywhere voluptuous is used
[Section titled “Anywhere voluptuous is used”](#anywhere-voluptuous-is-used)
Beyond Home Assistant and ESPHome, voluptuous validates configuration and request data across a long tail of libraries, CLIs, and services. For any of them the move is the same: change the import, keep the schemas, and gain a maintained library with a richer error model and codecs for JSON Schema, OpenAPI, dataclasses, and field lists. Start with [migrating from voluptuous](/getting-started/migrating-from-voluptuous/).
# Stability and roadmap
> Where Probatio stands today, and what stable means before 1.0.
Probatio is pre-1.0. Honest status: it works, it is tested, and it is young. It is on the 0.x line; the [release notes on GitHub](https://github.com/frenck/probatio/releases) track the latest release and what each one changed.
## The goal
[Section titled “The goal”](#the-goal)
Be a faithful drop-in for voluptuous. Change the import, keep your schemas. That is the whole point, and it sets the bar for correctness: behavior is validated against how voluptuous behaves, and a divergence is treated as a bug unless it is documented as an intentional deviation ([ADR-001](https://github.com/frenck/probatio/blob/main/adr/001-clean-room-reimplementation-of-voluptuous.md)).
The order of work follows from that. Match voluptuous first, then iterate.
## What stable means here
[Section titled “What stable means here”](#what-stable-means-here)
Three things, and the distinction matters.
**The voluptuous-compatible surface** is the contract, and it tracks voluptuous. The names and signatures every voluptuous user already depends on (`Schema`, `Required`, `All`, `Any`, `Coerce`, `Invalid`, and the rest) do not move without a documented reason, and when they do it is to sit closer to voluptuous, not further from it.
**The Probatio-only surface** is public too, and there is a lot of it: the parts voluptuous never had. Schemas built from your own types (`DataclassSchema`, `TypedDictSchema`), the extra validators and markers (`AsDatetime`, `Slug`, `Secret`, `Alias`, the cross-field rules, and more), the codecs (`to_json_schema` and `to_openapi` with their `from_*` inverses, plus the output-only `to_field_list`), and the build and compile policies. Anything importable straight from `probatio` and listed in `__all__` is part of this surface. Before 1.0 it may still shift as it settles under real use; at 1.0 it is frozen under semantic versioning like the rest, and a public-surface snapshot test in the suite covers the whole of it, so a change can only be deliberate.
**The error model** gets its own line, because it is the largest thing Probatio adds over voluptuous. The error classes (`Invalid` and its subclasses) and the structured data they carry, the `path` to the offending value, the machine-readable `code`, the `context`, and the `translation_key`, are the contract: catch them by type, read those fields, and rely on them. The `translation_key` names the exact sentence of each message, and with its `placeholders` it is what a translation keys on, so renaming or removing one is a breaking change ([ADR-015](https://github.com/frenck/probatio/blob/main/adr/015-structured-errors-and-localization.md); the [translation keys](/reference/translation-keys/) reference lists them all). The one thing not frozen is the rendered English text: a custom `msg=` overrides it, and a default may be reworded to read better, but the key stays put, so a translation follows the key, not the words.
**The internals are not promised.** Before 1.0 they may change while the project gathers production feedback and the implementation settles. If you reach past the public API into private modules (anything with a leading underscore), that is on you.
Pre-1.0
The Probatio-only surface may still shift before 1.0 as production feedback comes in. The voluptuous-compatible surface is already the contract; the internals never are.
## Versioning
[Section titled “Versioning”](#versioning)
Probatio follows semantic versioning. While it is pre-1.0 (`0.x`), the practical reading is: a patch release (`0.x.y`) is for fixes and compatibility corrections, a minor release (`0.x`) may add to the public surface or change an internal, and any breaking change to the public API is called out in the [release notes](https://github.com/frenck/probatio/releases).
Because the public API tracks voluptuous, most changes are really compatibility fixes: Probatio moving closer to how voluptuous behaves. Those are bug fixes, even when they change Probatio’s old output, and they ship as patch releases. A deliberate deviation from voluptuous is documented on the [compatibility page](/getting-started/compatibility/) and is never silent. Release notes are drafted from the merged pull requests, so every change is traceable to its reason.
## No promises I cannot keep
[Section titled “No promises I cannot keep”](#no-promises-i-cannot-keep)
There are no dates here, and no feature list of things that do not yet exist. The roadmap is the goal above: faithful compatibility, then iteration from real use. When something ships, it ships in the docs and the [release notes](https://github.com/frenck/probatio/releases), not as a promise on this page.
# Security
> Probatio's safety model for validating untrusted data and schemas.
Probatio sits at the edge of a program: it validates data that came from somewhere you do not control. That data is often hostile, and through `from_json_schema` even the *schema* can be hostile. This page is honest about what Probatio defends against, how, and where the responsibility stays with you.
## Threat model
[Section titled “Threat model”](#threat-model)
The thing Probatio touches most is untrusted data. A developer writes a schema, Probatio validates incoming values against it. That direction is safe: a schema is plain Python the developer wrote, and validating data against it does not expose any code-execution surface. Probatio does not `eval`, `exec`, `pickle`, or `marshal` the values it validates. There is no path where validating data runs attacker code.
The harder case is `from_json_schema` (and `from_openapi`). There the schema itself is decoded from an untrusted document. That widens the attack surface, because now both the data and the rules describing it are attacker-controlled.
An attacker against this surface is not after code execution; there is none to get. The realistic goals are denial of service: burn CPU until the process is useless, or exhaust the stack and crash the interpreter. Probatio’s safeguards target exactly those two outcomes.
Note
“Untrusted schema” means a JSON Schema or OpenAPI document you decode with `from_json_schema` or `from_openapi`. A schema you write in Python is trusted code, the same as the rest of your program.
## Threats and mitigations
[Section titled “Threats and mitigations”](#threats-and-mitigations)
| Threat | Vector | Mitigation |
| ----------------------------------------- | ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------- |
| Catastrophic backtracking (ReDoS) | A `pattern` in an untrusted JSON Schema, compiled to a regex | `from_json_schema` refuses a nested unbounded quantifier with `SchemaError`, before it compiles |
| Stack exhaustion from a deep document | A pathologically nested untrusted JSON Schema | The decoder caps nesting depth and raises `SchemaError` instead of overflowing the stack |
| Stack exhaustion from deep or cyclic data | Crafted data run through a recursive `Self` schema | A recursion depth guard raises a clean `Invalid` instead of `RecursionError` |
| Arbitrary object construction from YAML | Tags in an untrusted YAML payload | Probatio never parses YAML (or anything else), so it adds no deserialization surface; parse with a safe loader before validating |
These guards cover the specific shapes in the table, and only those. What they do not do is bound the overall size of the input: a validator cannot know how much data is too much for your application, so capping the size and shape of what you hand to a schema stays the calling application’s job. The project’s [security policy](https://github.com/frenck/probatio/blob/main/.github/SECURITY.md) draws the same line in its out-of-scope list.
## Regex denial of service
[Section titled “Regex denial of service”](#regex-denial-of-service)
Python’s `re` engine backtracks, so a pattern like `(a+)+$` runs in exponential time on crafted input. There is no safe timeout for `re` in pure Python, so the only defense is to refuse the dangerous pattern before compiling it.
`from_json_schema` does that. When a `pattern` contains a nested unbounded quantifier (an unbounded repeat applied to a group that is itself unbounded), the decoder raises `SchemaError` rather than building a validator that could hang:
```python
from probatio import from_json_schema
from_json_schema({"type": "string", "pattern": "(a+)+$"})
```
A benign pattern compiles as you would expect:
```python
from probatio import from_json_schema
schema = from_json_schema({"type": "string", "pattern": "^[a-z]+$"})
print(schema("hello")) # hello
```
This is a heuristic, not a proof
The detector targets the dominant catastrophic shape, nested unbounded quantifiers. It does not prove a pattern is safe against every form of pathological backtracking. Treat it as a strong guard against the common case, not a guarantee.
The trust boundary is the `from_json_schema` path, and only that path. A `Match` pattern you write in Python is *not* checked. That matches voluptuous: a developer-written regex is the developer’s responsibility. If you compile a pattern from input you do not trust, screen it yourself before handing it to `Match`.
## Recursion and stack exhaustion
[Section titled “Recursion and stack exhaustion”](#recursion-and-stack-exhaustion)
Two recursive shapes can drive Probatio into the Python stack: a deeply nested schema document, and deeply nested data validated against a recursive schema. A naive recursive walk turns either into a `RecursionError`, which is an unhandled crash. Probatio guards both.
A JSON Schema document nested past a fixed depth is refused while decoding:
```python
from probatio import from_json_schema
def nest(levels):
root = {"type": "object", "properties": {}}
cursor = root
for _ in range(levels):
child = {"type": "object", "properties": {}}
cursor["properties"]["x"] = child
cursor = child
return root
from_json_schema(nest(500))
```
On the data side, `Self` lets a schema validate a recursive structure, like a tree. Feed it data nested deeper than the recursion guard allows, and it raises a clean `Invalid` with the path to where it gave up, not a `RecursionError`:
```python
from probatio import Schema, Self, Invalid
schema = Schema({"value": int, "children": [Self]})
# A normal tree validates fine.
ok = {"value": 1, "children": [{"value": 2, "children": []}]}
print(schema(ok)) # {'value': 1, 'children': [{'value': 2, 'children': []}]}
def deep(levels):
node = {"value": 0, "children": []}
for _ in range(levels):
node = {"value": 0, "children": [node]}
return node
try:
schema(deep(5000))
except Invalid as err:
print(err.error_message) # data is nested too deeply for this recursive schema
```
The result is the same in both directions: a depth that would crash the process becomes a normal, catchable error instead.
## Parsing stays with you
[Section titled “Parsing stays with you”](#parsing-stays-with-you)
Probatio validates parsed Python objects; it never parses a document itself. That keeps a whole class of parser vulnerabilities out of the library, but it also puts the choice of parser on you. Parse untrusted YAML with a safe loader (PyYAML’s `yaml.safe_load`, not `yaml.load`) so a hostile document cannot construct arbitrary Python objects, then hand the result to the schema.
```python
import yaml
from probatio import Schema, Required
schema = Schema({Required("name"): str, Required("port"): int})
schema(yaml.safe_load("name: app\nport: 8080")) # {'name': 'app', 'port': 8080}
```
## Keeping secrets out of error output
[Section titled “Keeping secrets out of error output”](#keeping-secrets-out-of-error-output)
Configuration often carries credentials: a password, an API token, a private key. Wrap the key in `Secret`, and a validation failure under it is redacted: the error still reports the path and the reason, but shows `` instead of the offending value, so a rejected credential does not leak into a rendered error, a log line, or a stack trace.
```python
from probatio import Schema, Required, Secret
from probatio.humanize import humanize_error
from probatio.error import MultipleInvalid
schema = Schema({Required(Secret("api_token")): str, Required("host"): str})
data = {"api_token": 12345, "host": "example.com"}
try:
schema(data)
except MultipleInvalid as err:
print(humanize_error(data, err))
# expected str at 'api_token'. Got
```
The redaction is targeted: only the secret key’s value is hidden, a sibling field’s value still renders. Each error also carries a `secret` flag (on the error and in `as_dict()`), so a consumer building its own output can redact the same values.
The threat model is deliberately narrow: `Secret` covers validation error output, the place a schema library controls. It does not reach values you log yourself elsewhere, and it does not encrypt or mask data at rest. The validated value passes through unchanged; if you also need to keep it out of your own log lines, wrap it in your own carrier after validation.
## Reporting a vulnerability
[Section titled “Reporting a vulnerability”](#reporting-a-vulnerability)
Found something that looks like a security issue? Please [report it privately](https://github.com/frenck/probatio/security/advisories/new) through GitHub’s private vulnerability reporting, not as a public issue. That gives a fix time to land before the details are out in the open.
The full policy lives in [`SECURITY.md`](https://github.com/frenck/probatio/blob/main/.github/SECURITY.md): the disclosure timeline, what to include in a report, and an email fallback if you cannot use GitHub’s reporting flow.
# Validating a config file
> Define a schema for an app config, then load and validate a file end to end.
A config file is just data on disk. You parse it, then check it against a schema. Probatio validates the parsed result: you own the parsing, with whatever library fits the format, and hand the object to the schema. This recipe walks the whole path, from schema to a friendly error.
## Define the schema
[Section titled “Define the schema”](#define-the-schema)
Start with the shape of a valid config. A required name, an optional log level limited to a known set, and a nested `server` section with a host and a port. The port is coerced to an int and range-checked, so `"9000"` from the file becomes `9000`:
```python
from probatio import Schema, Required, Optional, All, Coerce, Range, In
schema = Schema(
{
Required("name"): str,
Optional("log_level", default="info"): In(
["debug", "info", "warning", "error"]
),
Required("server"): {
Required("host"): str,
Optional("port", default=8080): All(Coerce(int), Range(min=1, max=65535)),
},
}
)
```
`Required` keys must be present. `Optional` keys may be missing, and their `default` fills in when they are. `In` limits a value to a fixed set of choices. `All(Coerce(int), Range(...))` runs left to right: it converts the value to an int, then checks the bounds.
## Write a config file
[Section titled “Write a config file”](#write-a-config-file)
For a self-contained example, write a small JSON file. JSON parses with the standard library, so this works on a stdlib-only install. Note the port is a string here, on purpose, to show the coercion:
```python
import json
from pathlib import Path
config = {
"name": "my-app",
"server": {"host": "localhost", "port": "9000"},
}
_ = Path("config.json").write_text(json.dumps(config, indent=2))
```
## Load and validate
[Section titled “Load and validate”](#load-and-validate)
Read the file, parse it, then validate the parsed object. JSON parses with the standard library’s `json`, so this works on a bare install:
```python
result = schema(json.loads(Path("config.json").read_text()))
print(result)
# {'name': 'my-app', 'server': {'host': 'localhost', 'port': 9000}, 'log_level': 'info'}
```
Two things normalized on the way through. The port string `"9000"` became the int `9000`, and the missing `log_level` was filled with its default `"info"`. The result is a fresh dict; your file is untouched.
A YAML file works the same way: parse it with a YAML library (use a safe loader), then validate the result.
```python
from pathlib import Path
import yaml # PyYAML
result = schema(yaml.safe_load(Path("config.yaml").read_text()))
```
## Handle a bad config
[Section titled “Handle a bad config”](#handle-a-bad-config)
When the file does not match, validation raises. `humanize_error` from `probatio.humanize` renders the failure against the data, naming the value that went wrong, which is exactly what you want to print for whoever wrote the config:
```python
import json
from pathlib import Path
from probatio import Invalid
from probatio.humanize import humanize_error
bad = {"name": "my-app", "server": {"host": "localhost", "port": "70000"}}
Path("bad.json").write_text(json.dumps(bad))
try:
schema(json.loads(Path("bad.json").read_text()))
except Invalid as err:
print(humanize_error(bad, err))
# value must be at most 65535 at 'server.port'. Got '70000'
```
The message points straight at `server.port` and shows the offending value. The [error handling guide](/guides/error-handling/) goes deeper into paths, collecting every error at once, and the structured layer underneath.
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Quick start](/getting-started/quick-start/): the basics of building a schema.
* [Dict schemas and markers](/guides/dict-schemas-and-markers/): required and optional keys, defaults, and the extra-key policy.
* [Home Assistant](/recipes/home-assistant/): using Probatio where voluptuous is already in place.
# Cookbook
> Worked schema patterns for tagged unions, coercion, open mappings, key groups, recursion, and friendly errors.
A page of patterns you reach for again and again. Each one is a small, runnable schema with a passing input and, where it helps, a rejected one. Copy a block, run it, adapt it.
## Tagged union
[Section titled “Tagged union”](#tagged-union)
When your data carries a field that decides its shape (a `type`, a `platform`, a `condition`), `TaggedUnion` routes on that field’s value to the one matching schema. The win is the error: a bad branch reports its own field errors, not “matched none of the alternatives”, and an unknown tag is rejected with the valid tags named.
```python
from probatio import Schema, TaggedUnion
schema = Schema(
TaggedUnion(
"type",
{
"point": {"type": "point", "x": int, "y": int},
"label": {"type": "label", "text": str},
},
)
)
schema({"type": "point", "x": 1, "y": 2}) # {'type': 'point', 'x': 1, 'y': 2}
schema({"type": "label", "text": "hi"}) # {'type': 'label', 'text': 'hi'}
```
A point with a non-int coordinate fails against the point branch, not the whole union:
```python
from probatio import Schema, TaggedUnion
schema = Schema(TaggedUnion("type", {"point": {"type": "point", "x": int, "y": int}}))
schema({"type": "point", "x": "nope", "y": 2}) # expected int at 'x'
```
Each case can be any schema, so define the branch schemas once and reference them, a plain `Schema`, a `DataclassSchema`, or any validator. When a branch already pins its tag as a literal (`Required("type"): "point"`), pass the branches as a list and the tag is read from each, so it is written once instead of repeated as the mapping key:
```python
from probatio import Schema, TaggedUnion, Required
POINT = Schema({Required("type"): "point", Required("x"): int, Required("y"): int})
LABEL = Schema({Required("type"): "label", Required("text"): str})
schema = Schema(TaggedUnion("type", [POINT, LABEL])) # tag read from each branch
schema({"type": "label", "text": "hi"}) # {'type': 'label', 'text': 'hi'}
```
The routing table is built once, so the list form validates just as fast as the mapping. Use the `{tag: schema}` mapping when a branch does not carry the tag itself (its schema never mentions the key, or one branch covers several tags); a list branch that pins no literal is a build-time error pointing you to the mapping form.
Pass `default=` for a fallback schema when the tag is not listed. For a discriminator that is not a simple key lookup (a computed tag, a subset of branches), drop to the lower-level `Union(..., discriminant=fn)`, where `discriminant(value, alternatives)` returns the branches to try. `TaggedUnion` is the common case of that.
Coming from Home Assistant, `TaggedUnion` is the direct equivalent of `cv.key_value_schemas` (the mapping form), and its cases take the same reusable `Schema` objects.
Deep dive: [Combinators](/guides/combinators/).
## Coercing config and environment strings
[Section titled “Coercing config and environment strings”](#coercing-config-and-environment-strings)
Config files and environment variables hand you strings. `Coerce(int)` turns a numeric string into an int, `Boolean` reads the common truthy and falsy spellings, and wrapping a coercion in `All` lets you range-check the result. Order matters: coerce first, then validate the typed value.
```python
from probatio import Schema, Coerce, Boolean, All, Range
port = Schema(All(Coerce(int), Range(min=1, max=65535)))
port("9000") # 9000
flag = Schema(Boolean())
flag("yes") # True
flag("off") # False
```
`Coerce(int)` raises cleanly when the string is not a number, so a typo in a port does not crash with a raw `ValueError`:
```python
from probatio import Schema, Coerce, All, Range
port = Schema(All(Coerce(int), Range(min=1, max=65535)))
port("eighty")
```
Tip
`Boolean` is a factory, so call it: `Boolean()`, matching voluptuous. Pass a message to customize the error (`Boolean("not a flag")`). The string transforms like `Lower` and `Strip` are plain functions instead, so use those bare.
Deep dive: [Validators](/guides/validators/).
## Device and sensor conversions
[Section titled “Device and sensor conversions”](#device-and-sensor-conversions)
Raw device and sensor readings usually need a small calculation before they are useful: a milliunit divided down, a byte scaled to a percentage, a temperature converted, a status code named. The arithmetic mutators compose into these without a hand-written `Coerce(lambda ...)`, and because you supply the numbers, probatio never has to bake in a disputed formula.
```python
from probatio import Schema, All, Divide, Scale, Remap, Clamp, Round, Map
# Milliunits to units (millivolts to volts, milliseconds to seconds).
Schema(Divide(1000))(21500) # 21.5
# A 0..255 byte to a 0..100 percentage.
Schema(All(Remap(0, 255, 0, 100), Round(0)))(128) # 50.0
# Celsius to Fahrenheit, the affine transform in one call (value * 9 / 5 + 32).
Schema(Scale(9, divisor=5, offset=32))(20) # 68.0
# A device status code to a name (you own the table).
Schema(Map({0: "off", 1: "on", 2: "auto"}))(2) # 'auto'
```
`Map` rejects a key it does not know, which is right for a closed set of codes. When a table exists only to rewrite a few sentinel values ahead of a stricter step, pass `default=PASSTHROUGH` so an unlisted value flows through untouched. A scraped source that sends `"N.v.t."` (not applicable) for a missing enum is the common case: fold the sentinel to `None`, let a real value reach the enum.
```python
from enum import StrEnum
from probatio import Schema, All, Map, Maybe, PASSTHROUGH
class VehicleType(StrEnum):
CAR = "Personenauto"
vehicle_type = Schema(
All(Map({"N.v.t.": None}, default=PASSTHROUGH), Maybe(VehicleType))
)
vehicle_type("N.v.t.") # None
vehicle_type("Personenauto") #
```
`Map` runs first here and rewrites only what it names, so the enum coerces whatever is left. Order matters: put the `Map` before the enum in the `All`, or the enum rejects the sentinel before the table ever sees it.
The same works inside a dataclass field, and the field stays typed as the enum. A coercer in the metadata (`Map` or `Coerce`) runs before a producing base like an enum, so the sentinel is folded ahead of the enum without dropping to a `str` field:
```python
from dataclasses import dataclass
from enum import StrEnum
from typing import Annotated
from probatio import DataclassSchema, Key, Map, PASSTHROUGH
class VehicleType(StrEnum):
CAR = "Personenauto"
@dataclass
class Vehicle:
vehicle_type: Annotated[
VehicleType | None,
Key(alias="voertuigsoort"),
Map({"N.v.t.": None}, default=PASSTHROUGH),
] = None
schema = DataclassSchema(Vehicle)
schema({"voertuigsoort": "N.v.t."}) # Vehicle(vehicle_type=None)
schema(
{"voertuigsoort": "Personenauto"}
) # Vehicle(vehicle_type=)
```
An asserting constraint keeps the opposite order. An `In([...])` in the metadata runs *after* the enum, so it checks the produced member, not the raw string. Only a coercer moves ahead of a producing base.
RSSI to a signal percentage is the classic case with no single agreed formula. `Remap` lets you pick the input range, and `Clamp` keeps the result in bounds when a reading runs past it:
```python
from probatio import Schema, All, Remap, Clamp, Round
# Linear dBm -100..-50 to 0..100%, your chosen range, clamped and rounded.
rssi = Schema(All(Remap(-100, -50, 0, 100), Clamp(0, 100), Round(0)))
rssi(-70) # 60.0
rssi(-40) # 100 (past the top of the range, clamped)
```
Probatio deliberately ships no `RSSIToPercentage` or `CelsiusToFahrenheit`: the formula is a policy choice, and unit conversion is bottomless. The primitives keep that choice in your schema, where it is readable and yours to change.
Deep dive: [Validators](/guides/validators/).
## Open mapping with typed keys and values
[Section titled “Open mapping with typed keys and values”](#open-mapping-with-typed-keys-and-values)
To describe “any string key, integer value”, use a type as the dict key. A type key validates every key of that type, which is how you accept an open mapping without listing each name.
```python
from probatio import Schema
schema = Schema({str: int})
schema({"a": 1, "b": 2}) # {'a': 1, 'b': 2}
```
For “these known keys, plus anything else”, combine literal keys with the `Extra` catch-all. `{Extra: validator}` validates every otherwise-unmatched key. Use `{Extra: object}` to wave anything through:
```python
from probatio import Schema, Extra
schema = Schema({"name": str, Extra: object})
schema({"name": "app", "debug": True, "retries": 3})
# {'name': 'app', 'debug': True, 'retries': 3}
```
Deep dive: [Dict schemas and markers](/guides/dict-schemas-and-markers/).
## Validating dynamic keys
[Section titled “Validating dynamic keys”](#validating-dynamic-keys)
When the keys themselves are data (device slugs, entity ids, translation keys, a status code), put a validator in the key position. It runs on every key: a format check rejects a bad key pointing at the key, and a coercer or normalizer rewrites the key into the result. This is the same idea as a type key, with any validator.
```python
from probatio import Schema, Slug, Match, Coerce, In
# Keys must be slugs (area or device maps keyed by name).
Schema({Slug(): int})({"living-room": 1, "kitchen": 2})
# {'living-room': 1, 'kitchen': 2}
# Keys must match a pattern (an entity id, a translation key: [a-z0-9_-]).
Schema({Match(r"^[a-z0-9_-]+$"): str})({"already_running": "In progress"})
# {'already_running': 'In progress'}
# Coerce the key: JSON object keys are always strings, take integer ids back.
Schema({Coerce(int): str})({"1": "on", "2": "off"})
# {1: 'on', 2: 'off'}
# Restrict the key to a known set.
Schema({In(["celsius", "fahrenheit"]): float})({"celsius": 21.5})
# {'celsius': 21.5}
```
A bad key is reported at that key, not swallowed as an unexpected extra:
```python
from probatio import Schema, Slug
Schema({Slug(): int})({"Not A Slug": 1}) # expected a slug at 'Not A Slug'
```
Coming from Home Assistant, this is the direct form of `cv.schema_with_slug_keys`: `{Slug(): value_schema}` (or your own key validator, `{key_validator: value_schema}`) validates each key and then the values, no wrapper needed. Normalizing keys works the same way, `{Lower: value_schema}` folds the case of every key.
Deep dive: [Dict schemas and markers](/guides/dict-schemas-and-markers/).
## Nested optional sections with defaults
[Section titled “Nested optional sections with defaults”](#nested-optional-sections-with-defaults)
A whole config section can be optional, holding a nested schema with its own defaults. Give the outer `Optional` a callable `default` (here `dict`) so an absent section becomes a fresh empty dict. The default runs through the nested schema like any value, so an absent section comes back fully populated with the inner defaults, the same as a section provided as an empty dict.
```python
from probatio import Schema, Optional
schema = Schema(
{
Optional("logging", default=dict): {
Optional("level", default="info"): str,
Optional("file", default="app.log"): str,
},
}
)
schema({}) # {'logging': {'level': 'info', 'file': 'app.log'}}
schema({"logging": {}}) # {'logging': {'level': 'info', 'file': 'app.log'}}
schema({"logging": {"level": "debug"}})
# {'logging': {'level': 'debug', 'file': 'app.log'}}
```
Note
Use a callable default (`dict`, `list`) rather than a literal `{}` or `[]`. The callable runs per validation, so each result gets its own fresh container instead of sharing one mutable object across calls.
Deep dive: [Dict schemas and markers](/guides/dict-schemas-and-markers/).
## Mutually exclusive and co-dependent keys
[Section titled “Mutually exclusive and co-dependent keys”](#mutually-exclusive-and-co-dependent-keys)
`Exclusive` ties keys into a group where at most one may appear. `Inclusive` ties keys into a group that must appear together, all or none. Both take the group name as their second argument.
```python
from probatio import Schema, Exclusive, Inclusive
schema = Schema(
{
Exclusive("token", "auth"): str,
Exclusive("password", "auth"): str,
Inclusive("host", "server"): str,
Inclusive("port", "server"): int,
}
)
schema({"token": "abc", "host": "localhost", "port": 8080})
# {'token': 'abc', 'host': 'localhost', 'port': 8080}
```
Two keys from the same exclusive group is an error:
```python
from probatio import Schema, Exclusive, Invalid
schema = Schema(
{
Exclusive("token", "auth"): str,
Exclusive("password", "auth"): str,
}
)
try:
schema({"token": "abc", "password": "hunter2"})
except Invalid as err:
print(err)
# two or more values in the same group of exclusion 'auth' at ''
```
Half of an inclusive group is also an error:
```python
from probatio import Schema, Inclusive, Invalid
schema = Schema(
{
Inclusive("host", "server"): str,
Inclusive("port", "server"): int,
}
)
try:
schema({"host": "localhost"})
except Invalid as err:
print(err)
# some but not all values in the same group of inclusion 'server' at ''
```
Deep dive: [Dict schemas and markers](/guides/dict-schemas-and-markers/).
## At least one of a group of keys
[Section titled “At least one of a group of keys”](#at-least-one-of-a-group-of-keys)
To require that *at least one* of several keys is present, while still allowing more than one, use a `Required(Any(...))` key. The `Any` lists the acceptable keys, and the mapped value validates each one that appears. This is the “one or more” counterpart to `Exclusive` (at most one) and `Inclusive` (all or none).
```python
from probatio import Schema, Required, Any
schema = Schema(
{
Required(Any("email", "phone")): str,
"name": str,
}
)
schema({"name": "ada", "email": "a@b.c"}) # {'name': 'ada', 'email': 'a@b.c'}
```
Providing none of them fails, with the error naming the whole group:
```python
from probatio import Schema, Required, Any, Invalid
schema = Schema({Required(Any("email", "phone")): str})
try:
schema({})
except Invalid as err:
print(
err
) # at least one of ['email', 'phone'] is required at '[Any('email', 'phone', msg=None)]'
```
That default group label is honest but ugly: the path segment renders the `Any(...)` marker’s repr, because the group has no natural key name. (The repr leak itself is a known library issue, tracked separately.) The production form is a custom `msg` on the marker, read back through `error_message`, which carries the message without the path:
```python
from probatio import Schema, Required, Any, Invalid
schema = Schema(
{Required(Any("email", "phone"), msg="provide an email or a phone number"): str}
)
try:
schema({})
except Invalid as err:
print(err.errors[0].error_message) # provide an email or a phone number
```
Deep dive: [Dict schemas and markers](/guides/dict-schemas-and-markers/).
## Dropping deprecated keys
[Section titled “Dropping deprecated keys”](#dropping-deprecated-keys)
`Remove` drops matching keys from the output. The value is still validated, so a type error is not hidden; only a value that passes is dropped. Handy for retiring a setting without breaking configs that still carry it.
```python
from probatio import Schema, Remove
schema = Schema({"name": str, Remove("legacy_mode"): bool})
schema({"name": "app", "legacy_mode": True}) # {'name': 'app'}
```
Deep dive: [Dict schemas and markers](/guides/dict-schemas-and-markers/).
## Extending a base schema with a cross-field rule
[Section titled “Extending a base schema with a cross-field rule”](#extending-a-base-schema-with-a-cross-field-rule)
To add keys to a shared base *and* check a rule across the whole mapping, compose the two: `extend` merges the new keys, and `All` layers a whole-mapping validator on top. There is no need for `extend` to grow special cases; the combinators already compose.
```python
from probatio import Schema, All, Invalid, Required
base = Schema({Required("min"): int})
def min_below_max(config):
if config["min"] >= config["max"]:
raise Invalid("min must be below max")
return config
schema = Schema(All(base.extend({Required("max"): int}), min_below_max))
schema({"min": 1, "max": 10}) # {'min': 1, 'max': 10}
```
The whole-mapping rule runs after the keys validate:
```python
from probatio import Schema, All, Invalid, Required
base = Schema({Required("min"): int})
def min_below_max(config):
if config["min"] >= config["max"]:
raise Invalid("min must be below max")
return config
schema = Schema(All(base.extend({Required("max"): int}), min_below_max))
try:
schema({"min": 10, "max": 5})
except Invalid as err:
print(err) # min must be below max
```
Deep dive: [Combinators](/guides/combinators/).
## Recursive tree
[Section titled “Recursive tree”](#recursive-tree)
`Self` references the schema being defined, which is how you validate tree-shaped data of unbounded depth. It must sit as a direct mapping value or list element.
```python
from probatio import Schema, Required, Optional, Self
node = Schema(
{
Required("name"): str,
Optional("children", default=list): [Self],
}
)
node({"name": "root", "children": [{"name": "leaf"}]})
# {'name': 'root', 'children': [{'name': 'leaf', 'children': []}]}
```
A bad value deep in the tree is rejected with a path that points right at it:
```python
from probatio import Schema, Required, Optional, Self
node = Schema(
{
Required("name"): str,
Optional("children", default=list): [Self],
}
)
node({"name": "root", "children": [{"name": 42}]})
```
Deep dive: [Recursive schemas](/guides/recursive-schemas/).
## Friendly error messages
[Section titled “Friendly error messages”](#friendly-error-messages)
The default `str(error)` is precise but terse. `humanize_error` from `probatio.humanize` renders the failure against the data, naming the path and the offending value, which is what you want to show whoever wrote the config.
```python
from probatio import Schema, All, Coerce, Range, Invalid
from probatio.humanize import humanize_error
schema = Schema({"port": All(Coerce(int), Range(min=1, max=65535))})
bad = {"port": "70000"}
try:
schema(bad)
except Invalid as err:
print(humanize_error(bad, err))
# value must be at most 65535 at 'port'. Got '70000'
```
`validate_with_humanized_errors` does the same in one call: it validates and, on failure, raises a plain `Error` carrying the humanized message.
```python
from probatio import Schema, Range
from probatio.humanize import validate_with_humanized_errors
schema = Schema({"port": Range(min=1, max=65535)})
validate_with_humanized_errors({"port": 70000}, schema)
```
To replace a single validator’s message with your own wording, wrap it in `Msg`:
```python
from probatio import Schema, Match, Msg, Invalid
schema = Schema(Msg(Match(r"^[a-z]+$"), "use lowercase letters only"))
try:
schema("Nope123")
except Invalid as err:
print(err) # use lowercase letters only
```
Deep dive: [Error handling](/guides/error-handling/) and [Custom error messages](/guides/custom-error-messages/).
# Home Assistant
> Use Probatio where Home Assistant uses voluptuous, with or without touching internals.
Home Assistant validates its configuration with voluptuous, and it does so heavily. The `config_validation` helper is one of the largest voluptuous consumers there is. Probatio mirrors the voluptuous API, so it can stand in.
## The simple case: change the import
[Section titled “The simple case: change the import”](#the-simple-case-change-the-import)
Home Assistant code does not import voluptuous names one by one. The idiom is `import voluptuous as vol`, with every schema written against the `vol.` namespace: `vol.Schema`, `vol.Required`, `vol.Optional`, `vol.All`. That idiom makes the migration a one-line diff, with every `vol.` reference untouched:
```diff
import voluptuous as vol
import probatio as vol
```
The rest of the file does not change. This runs on Probatio as-is:
```python
import probatio as vol
CONFIG_SCHEMA = vol.Schema(
{
vol.Required("name"): str,
vol.Optional("port", default=8123): int,
}
)
CONFIG_SCHEMA({"name": "living-room"}) # {'name': 'living-room', 'port': 8123}
```
The names, signatures, and behavior match. `Required`, `Optional`, `All`, `Any`, `Coerce`, `Range`, `In`, the markers, the error classes: same surface, fresh implementation.
## The harder case: code that imports voluptuous internals
[Section titled “The harder case: code that imports voluptuous internals”](#the-harder-case-code-that-imports-voluptuous-internals)
Not everything imports the public API. Some dependencies reach into voluptuous *internals*. Home Assistant’s `annotatedyaml`, for example, imports `voluptuous.schema_builder._compile_scalar` directly. You cannot change the import in code you do not own.
For that, Probatio ships `install_as_voluptuous`. Call it once at process startup, before anything imports voluptuous:
```python
from probatio.compat import install_as_voluptuous
install_as_voluptuous()
```
It registers a small voluptuous-shaped shim, backed by Probatio, into `sys.modules` under the `voluptuous` name, so every later `import voluptuous` resolves to Probatio for the rest of the process. It is process-wide, so call it from an application entry point, not from library code, and call it early: if a real voluptuous was already imported, it is shadowed and a `RuntimeWarning` is emitted, since references already taken to the real module will not update. The [Compatibility](/getting-started/compatibility/) page covers exactly what it registers and when to call it.
## It is tested against the real thing
[Section titled “It is tested against the real thing”](#it-is-tested-against-the-real-thing)
This is not a paraphrase of compatibility. Probatio is validated against Home Assistant’s own `config_validation` test suite, with voluptuous swapped out for Probatio through `install_as_voluptuous`. As of July 2026, 136 of the 142 tests pass; the harness in `compat/home_assistant/` in the repository is the source of truth for the current count. The 6 that fail all assert the exact voluptuous-rendered error string, which Probatio deliberately renders differently (a dotted path instead of `@ data[...]`, see the [compatibility matrix](/reference/compatibility-matrix/)). The error attributes those tests could assert on instead (`path`, `error_message`, the error class) all match.
Getting there surfaced real compatibility gaps that isolated unit tests had missed, like `Remove(key)` keeping a value that fails its schema, and the `error_type` tag on a failed mapping value. Those are fixed. The harness reproduces the run against a Home Assistant checkout.
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Migrating from voluptuous](/getting-started/migrating-from-voluptuous/): the general swap, beyond Home Assistant.
* [Validating a config file](/recipes/config-file/): a worked end-to-end example.
* [Error handling](/guides/error-handling/): paths, multiple errors, and readable messages.
# LLM tool calling and MCP
> One Probatio schema as both the tool's JSON Schema definition and the validator for the model's arguments.
LLM tool calling runs on JSON Schema: an Anthropic tool declares its parameters as an `input_schema`, an MCP tool as an `inputSchema`, both plain JSON Schema documents. And the arguments the model sends back are untrusted input that you should validate before executing anything. That is one schema doing two jobs, and Probatio covers both directions: `to_json_schema` renders the tool definition, and calling the schema validates the arguments.
## Define the tool once
[Section titled “Define the tool once”](#define-the-tool-once)
```python
from probatio import Schema, Required, Optional, All, In, Range, to_json_schema
get_weather = Schema(
{
Required("location", description="City name, like 'Amsterdam'"): str,
Optional("unit", default="celsius"): In(["celsius", "fahrenheit"]),
Optional("days", default=1): All(int, Range(min=1, max=14)),
}
)
to_json_schema(get_weather)
# {'type': 'object', 'properties': {'location': {'type': 'string', 'description': "City name, like 'Amsterdam'"}, 'unit': {'enum': ['celsius', 'fahrenheit'], 'default': 'celsius'}, 'days': {'type': 'integer', 'minimum': 1, 'maximum': 14, 'default': 1}}, 'additionalProperties': False, 'required': ['location']}
```
The `description` on a marker flows into the JSON Schema, and the model reads it, so write it for the model. The output also carries `additionalProperties: False`, which strict tool use requires.
## Use it as the tool definition
[Section titled “Use it as the tool definition”](#use-it-as-the-tool-definition)
The rendered dict goes straight into an Anthropic `tools` entry as `input_schema`:
```python
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-opus-4-8",
max_tokens=1024,
tools=[
{
"name": "get_weather",
"description": "Get the weather forecast for a city.",
"input_schema": to_json_schema(get_weather),
}
],
messages=[{"role": "user", "content": "What is the weather in Amsterdam?"}],
)
```
An MCP server is the same move with a different key: the dict becomes the tool’s `inputSchema` in its tool listing.
## Validate the model’s arguments
[Section titled “Validate the model’s arguments”](#validate-the-models-arguments)
When the model calls the tool, run its arguments through the same schema before executing. Valid arguments come back normalized, with the defaults filled in:
```python
get_weather({"location": "Amsterdam"})
# {'location': 'Amsterdam', 'unit': 'celsius', 'days': 1}
```
Bad arguments fail with a path-precise error. Feed that message back to the model as the tool result (with `is_error` set) and it will correct itself:
```python
from probatio import MultipleInvalid
try:
get_weather({"location": "Amsterdam", "days": 30})
except MultipleInvalid as err:
print(err) # value must be at most 14 at 'days'
```
One schema, both directions: the definition the model sees and the validation its output gets cannot drift apart, because they are the same object.
## When the schema comes from elsewhere
[Section titled “When the schema comes from elsewhere”](#when-the-schema-comes-from-elsewhere)
Sometimes the JSON Schema is not yours: a tool listed by an MCP server you consume, or a tool definition stored as JSON. `from_json_schema` turns it into a validator, treating the document as untrusted (a catastrophic `pattern` or a pathologically deep document is refused with `SchemaError`):
```python
from probatio import from_json_schema
document = {
"type": "object",
"properties": {
"query": {"type": "string"},
"limit": {"type": "integer", "minimum": 1},
},
"required": ["query"],
}
validate_args = from_json_schema(document)
validate_args({"query": "probatio", "limit": 5}) # {'query': 'probatio', 'limit': 5}
```
## Where to next
[Section titled “Where to next”](#where-to-next)
* [JSON Schema](/guides/json-schema/): the full keyword mapping, round-trip caveats, and the untrusted-input guards.
* [Validating API requests](/recipes/web-api/): the same validate-or-reject flow at an HTTP boundary.
* [Error handling](/guides/error-handling/): shaping the failure you feed back to the model.
# Validating API requests
> Validate a request body with a schema and turn failures into a JSON 400 with per-field errors.
A request body is untrusted input. Validate it at the boundary, and when it fails, answer with a 400 that names every offending field, not just the first one. Probatio collects all failures in one pass and exposes them as structured data, so the error response is a small dict comprehension away. This recipe builds that flow with plain Python first, then shows where it plugs into a framework.
## The request schema
[Section titled “The request schema”](#the-request-schema)
A realistic signup payload: a username with length bounds, an email, a minimum age, an optional newsletter flag with a default, a nested address, and a list of tags:
```python
from probatio import Schema, Required, Optional, All, Length, Range, Email
signup = Schema(
{
Required("username"): All(str, Length(min=3, max=30)),
Required("email"): Email(),
Required("age"): All(int, Range(min=13)),
Optional("newsletter", default=False): bool,
Required("address"): {
Required("city"): str,
Required("postal_code"): str,
},
Optional("tags", default=list): [str],
}
)
```
A valid body comes back normalized, with the defaults filled in:
```python
body = {
"username": "ada",
"email": "ada@example.com",
"age": 37,
"address": {"city": "Delft", "postal_code": "2611"},
}
signup(body)
# {'username': 'ada', 'email': 'ada@example.com', 'age': 37, 'address': {'city': 'Delft', 'postal_code': '2611'}, 'newsletter': False, 'tags': []}
```
## Validate or 400
[Section titled “Validate or 400”](#validate-or-400)
An invalid body raises `MultipleInvalid`, whose `errors` list holds one `Invalid` per problem. Each carries `path` (the keys to the offending value) and `error_message` (the bare message, no path). That is everything a JSON error response needs:
```python
from probatio import MultipleInvalid
def error_response(err):
"""Build the JSON body of a 400 from a MultipleInvalid."""
return {
"error": "validation_failed",
"details": [
{"field": ".".join(str(seg) for seg in e.path), "message": e.error_message}
for e in err.errors
],
}
bad = {
"username": "ab",
"email": "not-an-email",
"age": 37,
"address": {"city": "Delft"},
}
try:
signup(bad)
except MultipleInvalid as err:
print(error_response(err))
# {'error': 'validation_failed', 'details': [{'field': 'username', 'message': 'length of value must be at least 3'}, {'field': 'email', 'message': 'expected an email address'}, {'field': 'address.postal_code', 'message': 'required key not provided'}]}
```
Three problems, three entries, and the nested one reads as `address.postal_code`. The client can highlight each field.
For a machine-readable contract, each error also offers `as_dict()`, the whole structured layer in one serializable dict: a stable `code`, the `path` as a list, and localization slots:
```python
try:
signup(bad)
except MultipleInvalid as err:
print(err.errors[0].as_dict())
# {'code': 'length', 'message': 'length of value must be at least 3', 'path': ['username'], 'secret': False, 'context': {}, 'translation_key': 'length_min', 'placeholders': {'min': 3}}
```
Prefer `as_dict()` when clients branch on the failure kind or translate messages; the hand-built shape above is fine when they only display them.
## Plugging into a framework
[Section titled “Plugging into a framework”](#plugging-into-a-framework)
The flow is the same everywhere: parse JSON, call the schema, catch `MultipleInvalid`, return 400. In Flask:
```python
from flask import Flask, jsonify, request
from probatio import MultipleInvalid
app = Flask(__name__)
@app.post("/signup")
def create_signup():
try:
data = signup(request.get_json())
except MultipleInvalid as err:
return jsonify(error_response(err)), 400
user = create_user(data) # your application logic, on clean data
return jsonify(user), 201
```
FastAPI users can do the same inside a route taking a `dict` body, or raise `HTTPException(status_code=400, detail=error_response(err))`.
## Where to next
[Section titled “Where to next”](#where-to-next)
* [Error handling](/guides/error-handling/): paths, `MultipleInvalid`, and the structured layer behind `as_dict()`.
* [JSON Schema](/guides/json-schema/): publish the same schema to clients as JSON Schema with `to_json_schema`.
* [Dict schemas and markers](/guides/dict-schemas-and-markers/): required and optional keys, defaults, and the extra-key policy.
# API reference
> The public surface of Probatio, grouped by what each name does.
This is the public surface of `probatio`. It mirrors voluptuous, so the names and signatures match what you already know. Everything here is importable straight from `probatio` (for example `from probatio import Schema, All, Range`), except `humanize_error` and `validate_with_humanized_errors`, which live in `probatio.humanize`.
## Schema
[Section titled “Schema”](#schema)
```python
class Schema:
def __init__(self, schema, required=False, extra=PREVENT_EXTRA): ...
def __call__(self, data): ...
def extend(self, schema, required=None, extra=None) -> Schema: ...
@classmethod
def infer(cls, data, **kwargs) -> Schema: ...
```
`Schema` compiles a validation definition once, then validates values when called, returning the normalized result or raising `MultipleInvalid`. `required` and `extra` are positional, matching voluptuous. `extend` returns a new schema with another mapping’s keys merged in (nested mappings merge recursively). Two schemas compare equal when their definitions match, regardless of dict key order.
`Schema.infer(data)` builds a schema from concrete example data: each value becomes its type, recursing into mappings and lists, so an API response can seed a schema. Keyword arguments pass through to `Schema`.
```python
from probatio import Schema, Required
Schema.infer(
{"name": "app", "port": 80}
) # == Schema({Required("name"): str, Required("port"): int})
```
A schema is built from plain Python: a type (`int`), a literal (`"on"`), a callable, a `dict`, a `list`/`tuple`/`set`, a nested `Schema`, or any validator below.
```python
from probatio import Schema, Required, Optional
schema = Schema({Required("name"): str, Optional("port", default=8080): int})
schema({"name": "app"}) # {'name': 'app', 'port': 8080}
```
Probatio validates parsed Python objects; parsing stays with the caller. Parse your JSON, YAML, or TOML with the library of your choice, then hand the result to the schema (`schema(json.loads(source))`).
## Markers
[Section titled “Markers”](#markers)
Markers are dictionary keys that carry intent. Each compares and hashes by its underlying key, so it can stand in for the bare key.
* `Required(key, msg=None, default=UNDEFINED, description=None)`: the key must be present.
* `Optional(key, msg=None, default=UNDEFINED, description=None)`: the key may be present; a `default` fills it in when absent.
* `Remove(key)`: drop matching keys from the validated output.
* `Forbidden(key, msg=None, description=None)`: the key must not be present; if it appears, validation fails with “key not allowed”. The mapped value is ignored, so the idiom is `{Forbidden("password"): object}`.
* `Secret(key, msg=None, description=None)`: redact the key’s value from validation error output (`` instead of the value). Composes with the presence markers by nesting, so `Optional(Secret("password"))` is an optional, redacted key. A Probatio addition (see [dict schemas and markers](/guides/dict-schemas-and-markers/#redacting-secret-values)).
* `Inclusive(key, group_of_inclusion, msg=None, default=UNDEFINED, description=None)`: all keys sharing a group must appear together, or none. `key` is a key schema, so a member can be an `Any` of names or a type, and counts as present when some key matches it.
* `Exclusive(key, group_of_exclusion, msg=None, description=None, *, required=False, default=UNDEFINED)`: at most one key from a group may appear; `key` is a key schema, so one member can cover several names. `required=True` makes the group demand exactly one key; a `default` fills that member in when the group is empty (and satisfies the group, so it wins over `required`). Both are group-level.
* `Extra`: a catch-all key. `{Extra: validator}` validates every otherwise-unmatched key.
* `Alias(key, *aliases, accept_canonical=True, required=False, default=UNDEFINED, msg=None, description=None)`: accept the value under any of the alias names and emit it under the canonical `key`. The names are tried in order, first one present wins; `accept_canonical=False` makes it a strict rename that accepts only the aliases. A Probatio addition (see [dict schemas and markers](/guides/dict-schemas-and-markers/)).
* `Marker`: the base class the markers above derive from, exposed for `isinstance` checks and custom markers.
* `Self`: a sentinel that refers to the enclosing schema, for recursive structures. Works as a mapping value, a sequence element, or a direct combinator branch. See [recursive schemas](/guides/recursive-schemas/).
The extra-key policy is set with the `extra` argument to `Schema`:
* `PREVENT_EXTRA` (the default): reject keys not in the schema.
* `ALLOW_EXTRA`: keep them untouched.
* `REMOVE_EXTRA`: drop them from the output.
## Constants and sentinels
[Section titled “Constants and sentinels”](#constants-and-sentinels)
* `UNDEFINED`: the “no value” sentinel for marker defaults.
* `Undefined`: the type of `UNDEFINED`, for `isinstance` checks.
* `default_factory(value)`: normalize a marker default into a zero-argument factory (the voluptuous helper).
* `Schemable`: the type alias for anything `Schema` accepts, for annotations.
* `__version__`: the installed Probatio version string.
## Combinators
[Section titled “Combinators”](#combinators)
* `All(*validators, msg=None, required=False)`: every validator must pass, the output of each feeding the next. Aliased as `And`.
* `Any(*validators, msg=None, required=False)`: the first validator that accepts the value wins. Aliased as `Or`.
* `Union(*validators, msg=None, required=False, discriminant=None)`: like `Any`, but a `discriminant(value, validators)` can narrow which validators to try, for a tagged union. Aliased as `Switch`.
* `SomeOf(validators, min_valid=None, max_valid=None, msg=None, required=False)`: the value must pass between `min_valid` and `max_valid` of the validators. Raises `NotEnoughValid` or `TooManyValid`.
```python
from probatio import All, Any, SomeOf, Range, Schema
Schema(All(str, str.strip))(" hi ") # 'hi'
Schema(Any(int, str))("a") # 'a'
Schema(SomeOf(min_valid=2, validators=[Range(1, 5), int, 3]))(3) # 3
```
A `Union` discriminant picks the branch instead of trying every alternative:
```python
from probatio import Schema, Union
def by_type(value, alternatives):
return [a for a in alternatives if a["type"] == value.get("type")]
schema = Schema(
Union({"type": "a", "v": int}, {"type": "b", "v": str}, discriminant=by_type)
)
schema({"type": "a", "v": 1}) # {'type': 'a', 'v': 1}
```
## Validators
[Section titled “Validators”](#validators)
The leaf validators, grouped by category. Every name is importable straight from `probatio`.
### Type and value
[Section titled “Type and value”](#type-and-value)
* `Coerce(type, msg=None)`: convert with `type(value)`, failing cleanly as `CoerceInvalid`.
* `Boolean(msg=None, clsoverride=None)`: a factory returning a validator that reads common truthy/falsy strings (`"yes"`, `"off"`) as a `bool`. Call it: `Boolean()`.
* `Literal(lit)`: require the value to equal a literal, returning the literal.
* `Equal(target, msg=None)`: require the value to equal `target`.
* `In(container, msg=None, *, fold_case=False, space=None)`: the value must be a member of `container`. `fold_case` matches case-insensitively; `space` collapses each whitespace run in a string to the given character; either returns the normalized value. A missed string value suggests the closest members (`did you mean ...?`) and records them on the error’s `candidates`.
* `NotIn(container, msg=None)`: the value must not be a member of `container`.
* `Contains(item, msg=None)`: the value (a collection) must contain `item`.
* `Match(pattern, msg=None)`: the value must match a regular expression.
### Numbers
[Section titled “Numbers”](#numbers)
* `Range(min=None, max=None, min_included=True, max_included=True, msg=None)`: numeric bounds, inclusive by default.
* `Clamp(min=None, max=None, msg=None)`: pin a value into a range instead of failing.
* `Number(precision=None, scale=None, msg=None, yield_decimal=False)`: validate a numeric string, checking precision/scale.
* `Positive(msg=None)`, `Negative(msg=None)`, `NonNegative(msg=None)`: sign conveniences (`> 0`, `< 0`, `>= 0`) over `Range`.
* `MultipleOf(factor, msg=None)`: require a number to be an integer multiple of `factor`.
* `Percentage(msg=None)`: validate a number or `"NN%"` string in 0 to 100, returned unchanged. `FromPercentage(msg=None)` parses it to a `float`.
* `Byte(msg=None)`, `SmallFloat(msg=None)`: a number in 0 to 255, or in 0 to 1.
* `Latitude(msg=None)`, `Longitude(msg=None)`: a coordinate in -90 to 90, or -180 to 180.
### Collections and structure
[Section titled “Collections and structure”](#collections-and-structure)
* `Length(min=None, max=None, msg=None)`: bound the length of a sized value.
* `Unique(msg=None)`: require all items to be distinct.
* `Set(msg=None)`: convert an iterable into a `set`.
* `ExactSequence(validators, msg=None)`: validate a fixed-length sequence, position by position.
* `Unordered(validators, msg=None)`: validate a sequence whose items may appear in any order.
* `Object(schema, cls=UNDEFINED)`: validate an object’s attributes like a mapping, rebuilding the same type. `cls` pins the class.
* `Maybe(validator, msg=None)`: allow `None`, otherwise validate against `validator`.
* `EnsureList()`: wrap a scalar in a list; a list passes through, `None` becomes `[]`.
* `NonEmpty(msg=None)`: require a sized value (string, list, mapping) to not be empty.
* `Sorted(msg=None)`: require a sequence to be in ascending order.
```python
from probatio import Object, Schema, Unordered
Schema(Unordered([str, int]))([1, "a"]) # [1, 'a']
class Point:
def __init__(self, x, y):
self.x, self.y = x, y
result = Schema(Object({"x": int, "y": int}))(Point(1, 2)) # validates the attributes
result.x # 1
```
### Strings
[Section titled “Strings”](#strings)
The transforms are plain functions; use them bare (`Lower`, not `Lower()`).
* `Lower`, `Upper`, `Capitalize`, `Title`, `Strip`: case and whitespace transforms.
* `Replace(pattern, substitution, msg=None)`: replace every match of a regular expression in a string.
* `Email()`, `Url()`, `FqdnUrl()`: format validators that avoid backtracking regular expressions.
* `Slug(msg=None)`: validate a slug (lowercase alphanumerics, hyphen/underscore separators).
* `IsRegex(msg=None)`: the value is itself a compilable regular expression.
* `Alpha`, `Alphanumeric`, `ASCII`, `PrintableASCII`, `NoWhitespace` (each `(msg=None)`): ASCII character-class checks (letters, letters and digits, ASCII, printable ASCII, no whitespace).
* `StartsWith(prefix, msg=None)`, `EndsWith(suffix, msg=None)`: the string starts with or ends with a fixed affix.
* `ByteLength(min=None, max=None, msg=None)`: bound the UTF-8 byte length (not the code-point count).
* `HexColor(msg=None)`: validate a hex color string (`#rgb` or `#rrggbb`), returned unchanged. Compose with `Lower` (or `Upper`) to fold case: `All(HexColor(), Lower)`.
### Date and time
[Section titled “Date and time”](#date-and-time)
* `Datetime(format=None, msg=None)`: validate a datetime string against a `strptime` format, default `%Y-%m-%dT%H:%M:%S.%fZ`, matching voluptuous. Use `AsDatetime` for ISO 8601 parsing.
* `Date(format=None, msg=None)`: validate a date string (default `%Y-%m-%d`).
* `Time(format=None, msg=None)`: validate a time-of-day string (default `%H:%M:%S`).
* `AsDatetime(format=None, require_timezone=False, msg=None)`: parse a string to a `datetime.datetime`. ISO 8601 by default, or a `strptime` `format=`. With `require_timezone`, a naive result is rejected.
* `AsDate(format=None, msg=None)`: parse a string to a `datetime.date` (ISO 8601 by default, or a `strptime` `format=`).
* `AsTime(format=None, msg=None)`: parse a string to a `datetime.time` (ISO 8601 by default, or a `strptime` `format=`).
* `Duration(msg=None)`: validate a duration (timedelta, seconds as an int/float/string, `H:MM[:SS]`, an ISO 8601 duration like `P1DT2H30M`, or a mapping), returning the value unchanged.
* `AsTimedelta(msg=None)`: parse the same duration forms to a `datetime.timedelta` (the object-returning sibling of `Duration`).
* `TimeZoneInfo(msg=None)`: validate an IANA zone name, returning it unchanged. Object via `Coerce(zoneinfo.ZoneInfo)`.
* `TimeZone(msg=None)`: validate a fixed UTC offset (`+01:00`, `Z`, `UTC`), returning it unchanged.
* `AsTimezone(msg=None)`: parse a fixed UTC offset to a `datetime.timezone` (the object-returning sibling of `TimeZone`).
* `FromEpoch(unit="seconds", msg=None)`: parse a Unix timestamp (`int` or `float`, `unit="seconds"` or `"milliseconds"`) into a timezone-aware UTC `datetime`.
### Filesystem
[Section titled “Filesystem”](#filesystem)
* `IsDir(msg=None)`: an existing directory path.
* `IsFile(msg=None)`: an existing file path.
* `PathExists(msg=None)`: a path that exists, of any kind.
* `IsSymlink(msg=None)`: an existing symbolic link.
* `IsSocket(msg=None)`: an existing socket.
* `IsFifo(msg=None)`: an existing named pipe (FIFO).
* `IsBlockDevice(msg=None)`: an existing block device.
### Network and identifiers
[Section titled “Network and identifiers”](#network-and-identifiers)
These are Probatio additions (voluptuous has no equivalent). They validate and return the value unchanged; wrap with `Coerce(the type)` when you want the parsed Python object.
* `IPv4Address(msg=None)`: an IPv4 address string. Object via `Coerce(ipaddress.IPv4Address)`.
* `IPv6Address(msg=None)`: an IPv6 address string. Object via `Coerce(ipaddress.IPv6Address)`.
* `IPAddress(msg=None)`: an IP address of either version. Object via `Coerce(ipaddress.ip_address)`.
* `IPNetwork(msg=None)`: a CIDR network (host bits allowed), returned unchanged (not normalized). Object via `Coerce(lambda v: ipaddress.ip_network(v, strict=False))` (plain `ip_network` defaults to `strict=True` and would reject host bits).
* `MacAddress(msg=None)`: validate a MAC address (common separators and bare hex), returned unchanged.
* `NormalizeMacAddress(upper=False, separator=":", msg=None)`: validate and return a canonical MAC (`aa:bb:cc:dd:ee:ff` by default; `upper=True` uppercases, `separator=` sets the separator, `""` for bare hex).
* `UUID(msg=None, version=None)`: a UUID string, returned unchanged; `version` pins the version. Object via `Coerce(uuid.UUID)`.
* `ULID(msg=None)`: validate a ULID (26 Crockford base32 characters, either case), returned unchanged. Compose with `Upper` for the canonical upper case.
* `Hostname(msg=None)`: a hostname (RFC 1123); a bare label like `localhost` is valid.
* `Fqdn(msg=None)`: a fully-qualified domain name (a dotted hostname).
* `Port(msg=None)`: a port number (1 to 65535), returned as an `int`.
### Encoding
[Section titled “Encoding”](#encoding)
* `JSONString(schema=None, msg=None)`: validate a JSON string (optionally the decoded value against `schema`), returning the string unchanged.
* `FromJSONString(schema=None, msg=None)`: parse a JSON string to the decoded value (the decoding sibling of `JSONString`), optionally validating it against `schema`.
* `Base64(msg=None)`, `Hex(msg=None)`: validate a Base64 or hexadecimal string, returning it unchanged.
* `HexInt(msg=None)`: parse a hexadecimal integer (a string like `"0x1A"` or `"1a"`, or an `int`) to an `int`.
* `CreditCard(normalize=True, msg=None)`: a credit card number that passes the Luhn checksum (12 to 19 digits, spaces or hyphens allowed). Normalized to bare digits by default; `normalize=False` returns the value unchanged.
* `IBAN(normalize=True, msg=None)`: an IBAN that passes the ISO 13616 mod-97 checksum (spaces allowed; the per-country length is not checked). Normalized to the compact, upper-cased form by default; `normalize=False` returns it unchanged.
* `DataURI(msg=None)`: an RFC 2397 data URI (`data:[][;base64],`), validating the Base64 payload when declared.
* `E164(normalize=True, msg=None)`: a phone number in international E.164 format (`+` then up to 15 digits). A format check, not a check that the number is dialable. Grouping characters are stripped by default; `normalize=False` rejects them and returns the value unchanged.
### Truthiness
[Section titled “Truthiness”](#truthiness)
* `IsTrue(msg=None)`: the value must be truthy.
* `IsFalse(msg=None)`: the value must be falsy.
### Cross-field rules
[Section titled “Cross-field rules”](#cross-field-rules)
Apply these after a dict schema with `All`; they inspect the whole mapping.
* `RequiredWith(trigger, *required, mode="any", msg=None)`: require keys when the trigger key is present. `trigger` may be a list of keys, combined by `mode` (`"any"` or `"all"`).
* `RequiredWithout(trigger, *required, mode="any", msg=None)`: require keys when the trigger key is absent (the mirror of `RequiredWith`).
* `RequiredIf(conditions, *required, mode="all", msg=None)`: require keys when the `{key: value}` conditions hold, combined by `mode` (`"all"` or `"any"`).
* `Check(predicate, msg)`: run a predicate over the value; falsy or raising reports `msg`.
* `AtLeastOne(*keys, msg=None, require_mapping=True)`: at least one of the keys must be present in the mapping.
* `AtMostOne(*keys, msg=None, require_mapping=True)`: at most one of the keys may be present.
* `ExactlyOne(*keys, msg=None, require_mapping=True)`: exactly one of the keys must be present.
* `AllOrNone(*keys, msg=None, require_mapping=True)`: all of the keys must be present together, or none of them.
* `Immutable(*fields, msg=None)`: reject a change to a field between the previous value (from `current_context()`) and the new one.
* `WriteOnce(*fields, msg=None)`: allow a field to be set once (from absent or `None`), then reject a later change.
### Defaults and messages
[Section titled “Defaults and messages”](#defaults-and-messages)
* `DefaultTo(default, msg=None)`: replace `None` with a default. A callable default is a factory, so `DefaultTo(list)` substitutes a new empty list.
* `SetTo(value)`: ignore the input and always produce a fixed value. A callable value is a factory, the same way.
* `Msg(validator, msg, cls=None)`: wrap a validator and replace its failure message.
## Decorators and helpers
[Section titled “Decorators and helpers”](#decorators-and-helpers)
* `probatio(constraints=None, returns=None, *, error=None)`: decorator validating a callable’s arguments from their annotations (sync or async). `constraints` is a `{parameter: validator}` map layered after the inferred type; `returns` opts into result validation (`True` uses the `-> R` annotation, or pass a schema); `error` raises an exception of your own for an invalid argument, called with the `Invalid` and raised from it. See [the probatio decorator](/guides/probatio-decorator/).
* `validate(*args, **kwargs)`: decorator validating a function’s arguments (and `__return__`) against hand-named schemas (the voluptuous drop-in).
* `raises(exc, msg=None, regex=None)`: context manager asserting a block raises `exc`, optionally matching it.
* `message(default=None, cls=None, *, translation_key=None)`: decorator turning a function that raises `ValueError` into a validator factory (the voluptuous helper). The decorated function becomes a factory: call it to get the validator (`Schema(isint())`, not `Schema(isint)`), optionally passing a per-use message and `Invalid` subclass.
* `truth(func)`: decorator turning a predicate into a validator (the voluptuous helper). A truthy result returns the value unchanged; a falsy one fails as “not a valid value”. Use the decorated function directly: `Schema(isdir)`.
* `current_context()`: the `context` value of the active `schema(data, context=...)` call, or `None` outside one. Lets a custom validator read call-time state, such as the previous value `Immutable` compares against. See the [custom validators guide](/guides/custom-validators/).
```python
from typing import Annotated
from probatio import probatio, Range
@probatio(returns=True)
def multiply(arg1: int, arg2: int) -> Annotated[int, Range(min=0)]:
return arg1 * arg2
multiply(3, 4) # 12
```
## Errors
[Section titled “Errors”](#errors)
Every failure is an `Invalid` (or a `MultipleInvalid` collecting several). The subclasses let callers catch by kind; all of them carry a `path` to the offending value and render it in `str(error)`.
* `Error`: base for everything Probatio raises.
* `SchemaError`: the schema definition itself is invalid (a programming error).
* `Invalid`: a single validation failure, with a `path`.
* `MultipleInvalid`: a collection of `Invalid` errors, proxying the first.
Semantic subclasses let callers catch by kind: `TypeInvalid`, `RangeInvalid`, `CoerceInvalid`, `RequiredFieldInvalid`, `ExtraKeysInvalid` (which carries `.candidates`, the close-match suggestions for an unknown key), and many more. The [errors reference](/reference/errors/) lists every subclass with its stable `code`.
## Humanizing errors
[Section titled “Humanizing errors”](#humanizing-errors)
In `probatio.humanize`:
* `humanize_error(data, validation_error, max_sub_error_length=..., *, locator=None)`: render an error against the data as a readable string, naming the offending value. With a `locator` (a callable you supply that maps an error `path` to a `Location`, typically backed by a location-aware loader), each line gains its source position.
* `validate_with_humanized_errors(data, schema, ...)`: validate, raising `Error` with a humanized message on failure.
Importable straight from `probatio` (it lives in `probatio.error`, not `probatio.humanize`):
* `Location`: a source position (`line`, `column`, `file`) a locator returns; renders as `file:line:column`.
## Schema interchange
[Section titled “Schema interchange”](#schema-interchange)
Probatio converts schemas to and from JSON Schema and OpenAPI, for the constructs that map cleanly (see the [JSON Schema guide](/guides/json-schema/) for the supported keywords, and [Field lists](/guides/field-lists/) for the `to_field_list` shape):
* `to_json_schema(schema)` / `from_json_schema(dict)`
* `to_openapi(schema, *, custom_serializer=None, openapi_version="3.0")` / `from_openapi(dict)`
* `to_field_list(schema, *, custom_serializer=None)`: render a schema as a plain field list (no inverse; the format is one-way). A `custom_serializer` is a callable that returns a node, or `UNSUPPORTED` to defer to the built-in handling.
* `UNSUPPORTED`: the sentinel a `custom_serializer` returns to defer.
`from_json_schema` treats its input as untrusted: it refuses a catastrophically backtracking `pattern` and a pathologically deep document rather than hanging or overflowing the stack.
## Dataclasses
[Section titled “Dataclasses”](#dataclasses)
Probatio builds a schema from a dataclass or a `TypedDict`, driven by its field annotations (see the [dataclasses guide](/guides/dataclasses/) and the [TypedDicts guide](/guides/typeddict/)):
* `DataclassSchema(dataclass_type, additional_constraints=None, *, required=False, extra=PREVENT_EXTRA)`: a `Schema` that validates a mapping and constructs an instance of `dataclass_type`.
* `create_dataclass_schema(dataclass_type, additional_constraints=None, *, required=False, extra=PREVENT_EXTRA)`: the functional form, returning the same `Schema`.
* `is_dataclass(obj)`: the standard-library check, re-exported.
* `TypedDictSchema(typeddict_type, additional_constraints=None, *, extra=PREVENT_EXTRA)`: a `Schema` that validates a mapping against a `TypedDict` and returns it typed as that `TypedDict` (nothing is constructed).
* `create_typeddict_schema(typeddict_type, additional_constraints=None, *, extra=PREVENT_EXTRA)`: the functional form, returning the same `Schema`.
* `Key(secret=False, alias=None, accept_canonical=True, forbidden=False, remove=False, inclusive=None, exclusive=None, required=None, description=None, msg=None)`: configure the marker a builder generates for an `Annotated` field, so a dataclass or `TypedDict` field carries the marker behavior (secret, alias, forbidden, remove, group membership, and the rest) without a hand-written dict schema. See the [dataclasses guide](/guides/dataclasses/).
A field without a default is `Required`, one with a `default`/`default_factory` is `Optional`. Annotations map deeply (`list[str]` to `[str]`, `X | None` to `Maybe(X)`, a nested dataclass to its own schema); `additional_constraints` adds a per-field validator with `All`.
## Self-validation
[Section titled “Self-validation”](#self-validation)
A type can carry its own validator, so it validates by more than `isinstance`:
* `__probatio_validate__`: a classmethod a type can define to validate (and coerce) a value itself. Whenever that type is compiled as a bare schema, Probatio calls `Type.__probatio_validate__(value)` instead of an `isinstance` check and uses the return value. Works anywhere a type is used, including a hand-written `Schema(Type)`. See the [custom validators guide](/guides/custom-validators/).
## Compile policy
[Section titled “Compile policy”](#compile-policy)
A hot schema compiles itself into a specialized validator; these names control that process-wide (see [Compiled schemas](/guides/compiled-schemas/)):
* `CompilePolicy`: the policy enum: `OFF`, `ON`, and `AUTO` (the default, which compiles a schema once it runs hot).
* `get_compile_policy()`: return the active policy.
* `set_compile_policy(policy)`: set the process-wide policy.
## Build policy
[Section titled “Build policy”](#build-policy)
Whether a schema compiles its declaration at construction or defers it to first validation; these names control that process-wide (see [Lazy building](/guides/lazy-building/)):
* `BuildPolicy`: the policy enum: `EAGER` (the default, compile at construction, matching voluptuous) and `LAZY` (defer to first validation, so a schema built but never validated never compiles). Under `LAZY` a definition error raises at first use rather than at construction.
* `get_build_policy()`: return the active policy.
* `set_build_policy(policy)`: set the process-wide policy.
## Loading and dumping
[Section titled “Loading and dumping”](#loading-and-dumping)
Probatio validates parsed Python objects; parsing and serialization stay with the caller. Parse with the library of your choice, then validate the result, and serialize a validated value with your own dumper (see the [loading and dumping guide](/guides/loading-and-dumping/)).
# Built-ins by role
> Every built-in grouped by the job it does, split by the axis that matters most, whether it checks a value and hands it back or returns a changed one.
The [API reference](/reference/) lists every name with its signature, and the [voluptuous compatibility](/reference/compatibility-matrix/) page maps each name to its status against voluptuous. This page is the third lens: what *role* each built-in plays, so you can find the right tool by the job you need done rather than by remembering its name.
Everything here splits into four roles:
* **Markers** are dictionary keys that carry intent (required, optional, secret).
* **Combinators** compose or wrap other validators (`All`, `Any`, `Maybe`).
* **Validators** check a value and return it unchanged.
* **Transformers** return a changed value.
The line that matters most runs between the last two. A validator is a gate: the value passes through untouched or it raises. A transformer rewrites the value, so the thing that comes out is not the thing that went in. Transformers split again by whether the *type* changes:
* A **converter** changes the type (`"5"` to `5`, `"192.0.2.1"` to an `IPv4Address`). Its name says so: `As` converts to a named type, `From` decodes a source, `Coerce` is the generic one.
* A **normalizer** keeps the type and cleans the value (`"ABC"` to `"abc"`, a number clamped into a range). Its name reads as an action on the value (`Lower`, `Strip`, `Clamp`).
Knowing the role tells you where a name belongs in a chain. Validators and transformers run left to right inside `All`, so a converter early in the chain decides the type every later step sees.
## Markers
[Section titled “Markers”](#markers)
Dictionary keys that carry intent. Each compares and hashes by its underlying key, so a marker stands in for the bare key.
| Name | Note |
| ----------- | ---------------------------------------------- |
| `Required` | The key must be present. |
| `Optional` | The key may be present; a `default` fills in. |
| `Remove` | Drop matching keys from the output. |
| `Extra` | Catch-all key for the rest of a mapping. |
| `Inclusive` | Group of keys that must appear together. |
| `Exclusive` | Group of keys of which at most one may appear. |
| `Self` | A recursive reference to the schema. |
| `Forbidden` | The key must be absent. |
| `Alias` | Accept the key under alternate input names. |
| `Secret` | Redact the key’s value in error output. |
## Combinators and wrappers
[Section titled “Combinators and wrappers”](#combinators-and-wrappers)
Compose or wrap other validators.
| Name | Note |
| ------------- | --------------------------------------------------- |
| `All` | Run each in turn; the output of one feeds the next. |
| `Any` | The first that accepts wins. |
| `And` | Alias of `All`. |
| `Or` | Alias of `Any`. |
| `Union` | `Any` with an optional discriminant. |
| `Switch` | Alias of `Union`. |
| `TaggedUnion` | Route on one key’s value to the matching schema. |
| `SomeOf` | Between `min_valid` and `max_valid` must pass. |
| `Maybe` | Accept `None`, otherwise the wrapped validator. |
| `Msg` | Override a validator’s error message. |
## Validators
[Section titled “Validators”](#validators)
Check a value and return it unchanged.
### Value and type
[Section titled “Value and type”](#value-and-type)
| Name | Note |
| --------------- | ------------------------------------ |
| `Literal` | Equal to a literal value. |
| `Equal` | Equal to a target. |
| `In` | A member of a container. |
| `NotIn` | Not a member of a container. |
| `Contains` | Contains a target value. |
| `Match` | Matches a regular expression. |
| `IsRegex` | A compilable regular expression. |
| `Object` | Attributes validated like a mapping. |
| `ExactSequence` | A sequence matching a fixed shape. |
| `Unordered` | A sequence in any order. |
| `Unique` | No duplicate elements. |
| `Sorted` | Already in ascending order. |
| `Length` | Length within bounds. |
| `NonEmpty` | Not empty. |
### Numbers
[Section titled “Numbers”](#numbers)
| Name | Note |
| ------------- | --------------------------------------------------- |
| `Range` | Within a numeric range. |
| `Number` | A number; `yield_decimal=True` returns a `Decimal`. |
| `Positive` | Greater than zero. |
| `Negative` | Less than zero. |
| `NonNegative` | Zero or greater. |
| `MultipleOf` | An exact multiple of a base. |
| `Byte` | An integer from 0 to 255. |
| `SmallFloat` | A float from 0 to 1. |
| `Latitude` | A valid latitude. |
| `Longitude` | A valid longitude. |
| `Percentage` | A number or `"NN%"` string in 0 to 100. |
### Strings and formats
[Section titled “Strings and formats”](#strings-and-formats)
| Name | Note |
| ---------------- | ------------------------------------------------------------------------------------------- |
| `Email` | An email address. |
| `Url` | A URL. |
| `FqdnUrl` | A URL with a fully qualified domain. |
| `Slug` | A URL slug. |
| `Alpha` | Letters only. |
| `Alphanumeric` | Letters and digits only. |
| `ASCII` | ASCII characters only. |
| `PrintableASCII` | Printable ASCII only. |
| `NoWhitespace` | No whitespace characters. |
| `StartsWith` | Begins with a prefix. |
| `EndsWith` | Ends with a suffix. |
| `ByteLength` | UTF-8 byte length within bounds. |
| `HexColor` | A hex color; compose `Lower`/`Upper` for case. |
| `Hex` | Hexadecimal; checks, does not decode. |
| `Base64` | Base64; checks, does not decode. |
| `JSONString` | A JSON string; value via `FromJSONString`. |
| `ULID` | A ULID; compose `Upper` for canonical case. |
| `DataURI` | A `data:` URI. |
| `Datetime` | A datetime string; string in, string out. |
| `Date` | A date string; string in, string out. |
| `Time` | A time string; string in, string out. |
| `Duration` | A duration (also a timedelta, a number of seconds, or a mapping); object via `AsTimedelta`. |
| `TimeZone` | A UTC offset; object via `AsTimezone`. |
| `TimeZoneInfo` | An IANA name; object via `Coerce(ZoneInfo)`. |
### Network and identifier
[Section titled “Network and identifier”](#network-and-identifier)
Check the format and return the string unchanged. The parsed object is opt-in through `Coerce`, since each of these types constructs from its string.
| Name | Note |
| ------------- | ----------------------------------------------------------------------------------------------------------- |
| `Hostname` | A hostname. |
| `Fqdn` | A fully qualified domain name. |
| `Port` | A TCP/UDP port number; returns an `int`. |
| `MacAddress` | A MAC address; canonical form via `NormalizeMacAddress`. |
| `UUID` | A UUID; object via `Coerce(uuid.UUID)`. |
| `IPv4Address` | Object via `Coerce(ipaddress.IPv4Address)`. |
| `IPv6Address` | Object via `Coerce(ipaddress.IPv6Address)`. |
| `IPAddress` | Object via `Coerce(ipaddress.ip_address)`. |
| `IPNetwork` | Object via `Coerce(lambda v: ipaddress.ip_network(v, strict=False))`; plain `ip_network` rejects host bits. |
### Filesystem
[Section titled “Filesystem”](#filesystem)
| Name | Note |
| --------------- | ---------------------- |
| `IsDir` | An existing directory. |
| `IsFile` | An existing file. |
| `PathExists` | An existing path. |
| `IsSymlink` | A symbolic link. |
| `IsSocket` | A socket. |
| `IsFifo` | A FIFO. |
| `IsBlockDevice` | A block device. |
### Truthiness
[Section titled “Truthiness”](#truthiness)
| Name | Note |
| --------- | --------------- |
| `IsTrue` | A truthy value. |
| `IsFalse` | A falsy value. |
### Cross-field and mapping rules
[Section titled “Cross-field and mapping rules”](#cross-field-and-mapping-rules)
Run over a whole mapping, usually after a dict schema with `All`.
| Name | Note |
| ----------------- | ----------------------------------------- |
| `RequiredWith` | Required when another key is present. |
| `RequiredWithout` | Required when another key is absent. |
| `RequiredIf` | Required when a condition holds. |
| `Check` | An arbitrary predicate over the value. |
| `AtLeastOne` | At least one of a set of keys is present. |
| `AtMostOne` | At most one of a set of keys is present. |
| `ExactlyOne` | Exactly one of a set of keys is present. |
| `AllOrNone` | All of a set of keys, or none. |
| `Immutable` | Rejects a change against the prior value. |
| `WriteOnce` | Rejects a second write. |
## Transformers
[Section titled “Transformers”](#transformers)
Return a changed value.
### Converters
[Section titled “Converters”](#converters)
The type changes.
| Name | Result |
| ---------------- | ----------------------------------------- |
| `Coerce` | `type(value)`, the generic convert. |
| `Boolean` | A string to a `bool`. |
| `Set` | An iterable to a `set`. |
| `EnsureList` | A scalar to a one-element `list`. |
| `AsDatetime` | A string to a `datetime`. |
| `AsDate` | A string to a `date`. |
| `AsTime` | A string to a `time`. |
| `AsTimedelta` | A duration to a `timedelta`. |
| `AsTimezone` | An offset to a `datetime.timezone`. |
| `FromEpoch` | A timestamp to a `datetime`. |
| `FromPercentage` | `"80%"` to a `float`. |
| `HexInt` | A hex string to an `int`. |
| `FromJSONString` | A JSON string to the decoded value. |
| `DefaultTo` | `None` to a default (substitution). |
| `EmptyToNone` | An empty string or container to `None`. |
| `Map` | A value to its entry in a lookup table. |
| `SetTo` | Anything to a fixed value (substitution). |
### Normalizers
[Section titled “Normalizers”](#normalizers)
The type stays; the value is cleaned.
| Name | Result |
| --------------------- | ---------------------------------------------- |
| `Lower` | Lowercase. |
| `Upper` | Uppercase. |
| `Capitalize` | Capitalized. |
| `Title` | Title-cased. |
| `Strip` | Whitespace trimmed. |
| `CollapseWhitespace` | Whitespace runs squeezed to one space. |
| `RemovePrefix` | A leading prefix removed if present. |
| `RemoveSuffix` | A trailing suffix removed if present. |
| `Truncate` | Cut to at most `max_length` characters. |
| `Replace` | Regular-expression replace. |
| `Clamp` | A number pinned into a range. |
| `NormalizeMacAddress` | A MAC address in canonical form. |
| `CreditCard` | A card number (Luhn), stripped to bare digits. |
| `IBAN` | An IBAN (mod-97), compacted and upper-cased. |
| `E164` | An E.164 phone number, grouping stripped. |
### Arithmetic
[Section titled “Arithmetic”](#arithmetic)
Compute a new number from the input. The result follows Python’s own arithmetic type, so an integer operation on an `int` stays an `int`, while a divisor or a fractional operand yields a `float` (`Divide` and `Remap` always do).
| Name | Result |
| ----------- | ------------------------------------------------ |
| `Multiply` | A number times a factor. |
| `Divide` | A number divided by a divisor. |
| `Offset` | A number plus an amount. |
| `Round` | A number rounded to decimals, or to an integer. |
| `RoundUp` | A number’s ceiling (a whole integer). |
| `RoundDown` | A number’s floor (a whole integer). |
| `Snap` | A number rounded to the nearest step. |
| `Abs` | A number’s absolute value. |
| `Modulo` | A number reduced modulo `n` (wrap-around). |
| `Remap` | A number mapped from one range onto another. |
| `Scale` | A number rescaled (`factor`/`divisor`/`offset`). |
### Collections
[Section titled “Collections”](#collections)
Reshape a list rather than validate it. `Sort` and `Dedupe` are the doing siblings of the `Sorted` and `Unique` validators.
| Name | Result |
| --------- | ------------------------------------------------- |
| `Split` | A delimited string into a list. |
| `Join` | A list into a delimited string. |
| `Sort` | A list ordered (a new list). |
| `Dedupe` | A list without duplicates, first-seen order kept. |
| `First` | The first item of a sequence. |
| `Last` | The last item of a sequence. |
| `Without` | A list without the listed values. |
`CreditCard`, `IBAN`, and `E164` normalize by default; pass `normalize=False` to validate and return the value unchanged.
# voluptuous compatibility
> A name-by-name map of voluptuous's public API and its status in Probatio, including the few intentional deviations.
The drop-in promise is the point of Probatio. This page makes it checkable. Below is voluptuous’s meaningful public surface, name by name, with its status in Probatio and a short note. Every name in voluptuous 0.16.0’s public surface was checked against Probatio with `hasattr(probatio, name)`, not from memory.
The status column uses three values:
* **Supported**: same name, same behavior.
* **Supported, with deviation**: present and compatible, but Probatio behaves differently in one documented way. Each one is a deliberate improvement.
* **Not mirrored**: Probatio does not expose this name. For everything on this page that is “not mirrored”, the reason is that the name is not part of voluptuous’s intended public API. See [What Probatio does not mirror](#what-probatio-does-not-mirror).
For the narrative version, read [Migrating from voluptuous](/getting-started/migrating-from-voluptuous/) and [Compatibility](/getting-started/compatibility/).
## Schema and constants
[Section titled “Schema and constants”](#schema-and-constants)
| Name | Status | Notes |
| ----------------- | --------- | ----------------------------------------------------------------- |
| `Schema` | Supported | Same calling convention, including positional `required`/`extra`. |
| `Schemable` | Supported | The type alias for anything `Schema` accepts. |
| `ALLOW_EXTRA` | Supported | Extra-key policy constant. |
| `PREVENT_EXTRA` | Supported | The default extra-key policy. |
| `REMOVE_EXTRA` | Supported | Extra-key policy constant. |
| `UNDEFINED` | Supported | The “no value” sentinel for marker defaults. |
| `Undefined` | Supported | The type of `UNDEFINED`. |
| `default_factory` | Supported | Wraps a callable as a marker default factory. |
## Markers
[Section titled “Markers”](#markers)
| Name | Status | Notes |
| ----------- | ------------------------- | ----------------------------------------------------------------------------------------------------- |
| `Marker` | Supported | Base class for all markers. |
| `Required` | Supported | The key must be present. |
| `Optional` | Supported | The key may be present; `default` fills it in. |
| `Remove` | Supported | Drop matching keys from the output. |
| `Extra` | Supported | The catch-all key. |
| `Inclusive` | Supported, with deviation | Keys in a group appear together or not at all. See [Intentional deviations](#intentional-deviations). |
| `Exclusive` | Supported, with deviation | At most one key from a group. See [Intentional deviations](#intentional-deviations). |
| `Self` | Supported, with deviation | Recursive schema reference. See [Intentional deviations](#intentional-deviations). |
## Combinators
[Section titled “Combinators”](#combinators)
| Name | Status | Notes |
| -------- | --------- | ----------------------------------------------------------- |
| `All` | Supported | Every validator must pass, output feeding the next. |
| `Any` | Supported | First accepting validator wins. |
| `And` | Supported | Alias of `All`. |
| `Or` | Supported | Alias of `Any`. |
| `Union` | Supported | Like `Any`, with an optional `discriminant`. |
| `Switch` | Supported | Alias of `Union`. |
| `SomeOf` | Supported | Pass between `min_valid` and `max_valid` of the validators. |
## Validators
[Section titled “Validators”](#validators)
| Name | Status | Notes |
| --------------- | --------- | ------------------------------------------------------------------------------ |
| `Coerce` | Supported | Convert with `type(value)`. |
| `Boolean` | Supported | Factory (`Boolean()`), as in voluptuous. Reads truthy/falsy strings as `bool`. |
| `Literal` | Supported | Require equality with a literal. |
| `Equal` | Supported | Require equality with a target. |
| `In` | Supported | Membership in a container. |
| `NotIn` | Supported | Non-membership in a container. |
| `Contains` | Supported | A collection must contain an item. |
| `Match` | Supported | Match a regular expression. |
| `Replace` | Supported | Replace regex matches in a string. |
| `Range` | Supported | Numeric bounds, inclusive by default. |
| `Clamp` | Supported | Pin a value into a range. |
| `Number` | Supported | Validate a numeric string, optionally precision and scale. |
| `Length` | Supported | Bound the length of a sized value. |
| `Unique` | Supported | Require distinct items. |
| `Set` | Supported | Convert an iterable into a `set`. |
| `ExactSequence` | Supported | Validate a fixed-length sequence by position. |
| `Unordered` | Supported | Validate a sequence in any order. |
| `Object` | Supported | Validate an object’s attributes like a mapping. |
| `Maybe` | Supported | Allow `None`, otherwise validate. |
| `Lower` | Supported | Lowercase transform. |
| `Upper` | Supported | Uppercase transform. |
| `Capitalize` | Supported | Capitalize transform. |
| `Title` | Supported | Title-case transform. |
| `Strip` | Supported | Whitespace-strip transform. |
| `Email` | Supported | Backtracking-safe email format check. |
| `Url` | Supported | Backtracking-safe URL format check. |
| `FqdnUrl` | Supported | URL with a fully qualified domain. |
| `Datetime` | Supported | Validate a datetime string (default `%Y-%m-%dT%H:%M:%S.%fZ`). |
| `Date` | Supported | Validate a date string (default `%Y-%m-%d`). |
| `IsDir` | Supported | An existing directory path. |
| `IsFile` | Supported | An existing file path. |
| `PathExists` | Supported | A path that exists. |
| `IsTrue` | Supported | The value must be truthy. |
| `IsFalse` | Supported | The value must be falsy. |
| `DefaultTo` | Supported | Replace `None` with a default. |
| `SetTo` | Supported | Always produce a fixed value. |
| `Msg` | Supported | Replace a validator’s failure message. |
## Error classes
[Section titled “Error classes”](#error-classes)
| Name | Status | Notes |
| ---------------------- | ------------------------- | -------------------------------------------------------------- |
| `Error` | Supported | Base for everything Probatio raises. |
| `SchemaError` | Supported | The schema definition itself is invalid. |
| `Invalid` | Supported | A single validation failure, with a `path`. |
| `MultipleInvalid` | Supported, with deviation | Collects several errors and proxies the first. See note below. |
| `RequiredFieldInvalid` | Supported | Semantic subclass of `Invalid`. |
| `ObjectInvalid` | Supported | Semantic subclass of `Invalid`. |
| `DictInvalid` | Supported | Semantic subclass of `Invalid`. |
| `SequenceTypeInvalid` | Supported | Semantic subclass of `Invalid`. |
| `TypeInvalid` | Supported | Semantic subclass of `Invalid`. |
| `ValueInvalid` | Supported | Semantic subclass of `Invalid`. |
| `ScalarInvalid` | Supported | Semantic subclass of `Invalid`. |
| `LiteralInvalid` | Supported | Semantic subclass of `Invalid`. |
| `CoerceInvalid` | Supported | Semantic subclass of `Invalid`. |
| `AnyInvalid` | Supported | Semantic subclass of `Invalid`. |
| `AllInvalid` | Supported | Semantic subclass of `Invalid`. |
| `MatchInvalid` | Supported | Semantic subclass of `Invalid`. |
| `RangeInvalid` | Supported | Semantic subclass of `Invalid`. |
| `LengthInvalid` | Supported | Semantic subclass of `Invalid`. |
| `InInvalid` | Supported | Semantic subclass of `Invalid`. |
| `NotInInvalid` | Supported | Semantic subclass of `Invalid`. |
| `ContainsInvalid` | Supported | Semantic subclass of `Invalid`. |
| `ExactSequenceInvalid` | Supported | Semantic subclass of `Invalid`. |
| `ExclusiveInvalid` | Supported | Semantic subclass of `Invalid`. |
| `InclusiveInvalid` | Supported | Semantic subclass of `Invalid`. |
| `TrueInvalid` | Supported | Semantic subclass of `Invalid`. |
| `FalseInvalid` | Supported | Semantic subclass of `Invalid`. |
| `BooleanInvalid` | Supported | Semantic subclass of `Invalid`. |
| `UrlInvalid` | Supported | Semantic subclass of `Invalid`. |
| `EmailInvalid` | Supported | Semantic subclass of `Invalid`. |
| `DirInvalid` | Supported | Semantic subclass of `Invalid`. |
| `FileInvalid` | Supported | Semantic subclass of `Invalid`. |
| `PathInvalid` | Supported | Semantic subclass of `Invalid`. |
| `DatetimeInvalid` | Supported | Semantic subclass of `Invalid`. |
| `DateInvalid` | Supported | Semantic subclass of `Invalid`. |
| `NotEnoughValid` | Supported | Raised by `SomeOf`. |
| `TooManyValid` | Supported | Raised by `SomeOf`. |
Note
`MultipleInvalid` carries an extra `error_type` attribute that proxies the first error’s type, where voluptuous has no such attribute. It is additive, so it does not break voluptuous code: code that never touched `error_type` keeps working, and code that reads it gets the first error’s `error_type`.
## Helpers
[Section titled “Helpers”](#helpers)
| Name | Status | Notes |
| -------------------------------- | --------- | ---------------------------------------------------------------------------------------- |
| `validate` | Supported | Decorator that validates a function’s arguments and `__return__`. |
| `message` | Supported | Decorator turning a `ValueError`-raising function into a configurable validator factory. |
| `truth` | Supported | Decorator turning a predicate into a validator that returns the value when truthy. |
| `raises` | Supported | Context manager asserting a block raises a given exception. |
| `humanize_error` | Supported | In `probatio.humanize`. Render an error against the data. |
| `validate_with_humanized_errors` | Supported | In `probatio.humanize`. Validate, raising a humanized message. |
## Additions beyond voluptuous
[Section titled “Additions beyond voluptuous”](#additions-beyond-voluptuous)
These names and options are not in voluptuous; they are Probatio additions, so they do not affect the drop-in promise (existing code keeps working). Each carries forward an upstream request.
* `Forbidden`: a marker requiring a key to be absent (`{Forbidden("password"): object}`). Carries forward [issue #193](https://github.com/alecthomas/voluptuous/issues/193).
* `TaggedUnion`: a discriminated union that routes on one key’s value to the matching schema (`TaggedUnion("type", {"grid": ..., "solar": ...})`), reporting the chosen branch’s own errors and naming the valid tags on a miss. Cases are a `{tag: schema}` mapping, or a list of branches that each pin the tag as a literal (read once from the branch); the routing table is built once either way. The readable form of a `Union` with a hand-written `discriminant`, and the direct equivalent of Home Assistant’s `cv.key_value_schemas`.
* `Alias`: a key marker accepting a value under one or more alias names and emitting it under a canonical name (`{Alias("user_name", "user-name"): str}`). Multiple aliases, first-present-wins by declaration order, optional strict rename. voluptuous has no equivalent.
* `DataclassSchema`, `create_dataclass_schema`, `is_dataclass`: build a schema from a dataclass, validate a mapping, and construct an instance. Carries forward draft [PR #533](https://github.com/alecthomas/voluptuous/pull/533).
* `TypedDictSchema`, `create_typeddict_schema`: build a schema from a TypedDict and return the validated mapping typed as that TypedDict (nothing constructed, since a TypedDict is a plain dict at runtime). voluptuous has no equivalent.
* `Exclusive(..., required=True)`: an exclusive group must hold exactly one key (an empty group is an error). Carries forward [issue #115](https://github.com/alecthomas/voluptuous/issues/115).
* `Exclusive(..., default=...)`: fill a group member when the exclusive group is empty. Carries forward [issue #245](https://github.com/alecthomas/voluptuous/issues/245).
* `IPv4Address`, `IPv6Address`, `IPAddress`, `IPNetwork`, `MacAddress`, `UUID`, `Hostname`, `Fqdn`: network and identifier validators; they validate and return the value unchanged (wrap with `Coerce` for the parsed object). `NormalizeMacAddress` returns a MAC in canonical form, and `Port` returns an `int`. Common across ecosystems, reinvented across Home Assistant and ESPHome.
* `Time`, `Duration`, `TimeZoneInfo`, `TimeZone`: time-of-day (the sibling of `Date`/`Datetime`), duration validation, IANA zone validation, and UTC offset validation, each returning the value unchanged. `Time` is voluptuous [issue #335](https://github.com/alecthomas/voluptuous/issues/335); the others recur across ecosystems.
* `AsDatetime`, `AsDate`, `AsTime`, `AsTimedelta`, `AsTimezone`: the object-returning siblings of `Datetime`/`Date`/`Time`/`Duration`/`TimeZone`, parsing to a `datetime`/`date`/`time`/`timedelta`/`datetime.timezone` instead of validating and passing the value through. The date and time ones parse ISO 8601 by default (standard library only, for deterministic results), or a `strptime` format; `AsDatetime` can require a timezone-aware result. (`TimeZoneInfo` has no `As*` sibling: `Coerce(zoneinfo.ZoneInfo)` gives the object, since the constructor takes the name.)
* `FromEpoch`: parse a Unix timestamp (seconds or milliseconds) into a timezone-aware UTC `datetime`. Common in APIs and device payloads.
* `EnsureList`, `Slug`, `Positive`, `Negative`, `NonNegative`, `MultipleOf`, `Percentage`, `FromPercentage`: list-wrapping, slug format, sign conveniences, integer-multiple, and a 0 to 100 percentage (`Percentage` validates and returns the value; `FromPercentage` parses it to a `float`). Common config helpers.
* `Secret`: a key marker that redacts the key’s value from validation error output (`` instead of the value), so a rejected credential does not leak into rendered validation error output. Composes with the presence markers by nesting (`Optional(Secret("password"))`). Voluptuous has no equivalent.
* `NonEmpty`, `Byte`, `SmallFloat`, `IsRegex`: a non-empty check, 0 to 255 and 0 to 1 bounded numbers, and a “value is a compilable regex” check.
* `Multiply`, `Divide`, `Offset`, `Round`, `RoundUp`, `RoundDown`, `Snap`, `Abs`, `Modulo`, `Remap`, `Scale`: arithmetic mutators for the small calculations common on device and sensor fields (a gain, a unit divide, a shift, rounding up/down or to a nearest step, a magnitude, a modulo wrap-around, a range remap like the Arduino `map()`, or the whole affine transform in one `Scale`). Readable, composable stand-ins for `Coerce(lambda ...)`.
* `CollapseWhitespace`, `RemovePrefix`, `RemoveSuffix`, `Truncate`: string normalizers beyond the plain transforms (squeeze internal whitespace, strip a known prefix or suffix, cap the length). Unlike `Lower`/`Strip`, they reject a non-string rather than coercing it.
* `Map`, `EmptyToNone`: `Map` translates a value through a table you supply (a device status code to a name), the bring-your-own-mapping answer to a domain conversion so probatio never picks a disputed table; `EmptyToNone` turns an empty string or container into `None` (the “empty means unset” shape), the near-reverse of `DefaultTo`.
* `Split`, `Join`, `Sort`, `Dedupe`, `First`, `Last`, `Without`: collection shapers. `Split`/`Join` convert between a delimited string and a list; `Sort` and `Dedupe` are the doing siblings of the `Sorted` and `Unique` validators; `First`/`Last` pick an element; `Without` prunes listed values (so `Without(None)` clears the holes).
* `JSONString`, `FromJSONString`: validate a JSON string (and optionally the decoded value against an inner schema); `JSONString` returns the string unchanged, while `FromJSONString` returns the decoded value.
* `IsSymlink`, `IsSocket`, `IsFifo`, `IsBlockDevice`: filesystem predicates for the special file types, alongside `IsDir`/`IsFile`.
* `Alpha`, `Alphanumeric`, `ASCII`, `PrintableASCII`, `NoWhitespace`, `StartsWith`, `EndsWith`, `ByteLength`, `HexColor`: character-class and affix string checks, UTF-8 byte length, and a hex color.
* `Base64`, `Hex`, `Sorted`, `ULID`, `Latitude`, `Longitude`: base64/hex validation, an ascending-order check, a ULID, and geographic coordinate bounds.
* `CreditCard`, `IBAN`, `DataURI`, `E164`: format and checksum validators (Luhn, ISO 13616 mod-97, RFC 2397 data URIs, international phone format). Pure checks, no network or extra dependency. voluptuous has none of these.
* `RequiredWith`, `RequiredWithout`, `RequiredIf`, `Check`: cross-field rules over a whole mapping (used after a dict schema with `All`), for conditional requirements (on a key’s presence, absence, or value, combined with an any/all mode) and an arbitrary predicate. voluptuous has no equivalent.
* `AtLeastOne`, `AtMostOne`, `ExactlyOne`, `AllOrNone`: key-group presence rules over a mapping, for how many of a set of keys may or must appear. The dict-level form of `Inclusive`/`Exclusive`. They reject a non-mapping by default (`require_mapping=False` to opt out). Home Assistant and ESPHome roll their own `has_at_least_one_key`/`has_none_or_all_keys`; voluptuous has no equivalent.
* `Immutable`, `WriteOnce`: transition rules that compare new data against its previous value (passed as the call’s `context`), rejecting a change to an immutable field or a second write to a write-once one. voluptuous has no equivalent.
* `ExtraKeysInvalid` with `.candidates`: an unknown key under `PREVENT_EXTRA` reports the closest known keys (“did you mean …?”) and carries them on the error, instead of a bare “extra keys not allowed”. ESPHome forks voluptuous internals for this; first-class here, so any consumer gets the suggestion.
* `__probatio_validate__`: a protocol a type defines (a classmethod) to validate a raw value of itself, used in place of the bare `isinstance` check when that type is a schema. `EnumInvalid` is the error its built-in consumer (an enum class) raises. voluptuous has no such hook.
* `current_context` with `schema(data, context=...)`: an optional call argument that hands per-call state to validators that read `current_context()`, so one compiled schema validates against state known only at call time. Additive (a plain `schema(data)` is unchanged). voluptuous has no equivalent.
## Intentional deviations
[Section titled “Intentional deviations”](#intentional-deviations)
Compatibility is the target, so this list is short. Each entry is a deliberate improvement over a sharp edge, not a regression.
Each entry reads the same way: the behavior, then what voluptuous does, then what Probatio does.
* **An `int` in a `float` schema (`Schema(float)(5)`, ADR-017).** voluptuous validates `float` by `isinstance`, so an `int` is rejected. Probatio honors the PEP 484 numeric tower: an `int` is a valid `float` value, so it is accepted and normalized to `float` (`5` becomes `5.0`), while `bool`, a string, and `None` are still rejected. This aligns the validator with the `{"type": "number"}` that `to_json_schema` already emits. A strict widening: only a value voluptuous would have rejected is affected, and a `float` matcher now matches an `int` too (so `Remove(float)` in a list drops ints, and a `{float: ...}` type key matches an int key).
* **A validator as the key of an `Inclusive`/`Exclusive` group.** A group member is usually a literal key (`Inclusive("host", "server")`), but the marker accepts any key schema, including an `Any` of names (`Inclusive(Any("hours", "minutes"), "duration")`). voluptuous never counts such a member as present, which leaves `Inclusive` unsatisfiable (no input containing another member of the group is ever accepted, whatever the `Any` names hold) and `Exclusive` a silent no-op (the member never triggers the exclusion). Probatio treats the key as one member satisfied by any of the names it lists, so `Inclusive(Any("hours", "minutes"), "d")` with `Inclusive("name", "d")` accepts `{"hours": 1, "name": "tea"}` and rejects either one alone. A strict improvement: the voluptuous readings make the group either impossible or meaningless.
* **The rendered error string, `str(error)` (ADR-015).** voluptuous renders `expected int for dictionary value @ data['server']['port']`. Probatio renders the same error as `expected int at 'server.port'`: the path is a dotted trail (sequence indices as `[n]`, so `at 'hosts[2].name'`), and the error-type clause is not rendered. A non-mapping is rejected as “expected a mapping” instead of “expected a dictionary” (the engine accepts any `Mapping`, and the message now says so). The attributes are unchanged: `path`, `msg`, `error_message`, and `error_type` keep their voluptuous semantics, so code that reads them (rather than parsing the rendered string) is unaffected. Code that string-matches `str(error)` should switch to `path` and `error_message`, or to `as_dict()`.
* **Reworded default messages (ADR-015).** A handful of voluptuous default messages were broken or developer-speak; Probatio words them for the person reading them. `Literal` reads `expected {lit}` instead of `{value} not match for {lit}`. `Unordered` reads “expected a sequence”, “expected a sequence of N items”, and “item N (…) does not match any validator” instead of “Value … is not sequence!”, “List lengths differ, value:N != target:M”, and “Element #N (…) is not valid against any validator”. The filesystem predicates lose their stray capitals (“not a directory”, not “Not a directory”). `Boolean` reads “expected a boolean”. Classes, paths, and translation keys are unaffected; only the sentence changed.
* **Recursive `Self` on cyclic or very deep data.** voluptuous crashes with `RecursionError`. Probatio raises a clean `Invalid` with a path, caught with the rest of your errors. The depth limit tracks a fraction of the interpreter’s recursion limit, so raising `sys.setrecursionlimit` allows deeper data, up to the point where the operating system’s own stack is the real ceiling; past that a very high limit can still overflow, so keep the limit within reason.
* **`from_json_schema` on untrusted input.** voluptuous can hang or overflow the stack. Probatio refuses a catastrophically backtracking `pattern` and a pathologically deep document with `SchemaError`.
* **`MultipleInvalid.error_type`.** voluptuous has no such attribute. Probatio proxies the first error’s type. Additive, so existing code is unaffected.
* **Comparing a `Remove` marker (`Remove("b") == "b"`).** voluptuous compares a `Remove` to the key underneath it, so that is `True`, while hashing it by identity, so equal markers hash differently and the hash/equality contract is broken. Probatio both compares and hashes `Remove` by identity, so it is `False`. This covers every direct `==` against a `Remove`, whatever is on the other side: a bare key, or another marker for the same key (`Remove("b") == Required("b")`). Schema behavior is unaffected: a `Remove` still replaces the key it shadows when merged with `extend`, and several `Remove` markers still coexist in one schema. Every other marker compares to its underlying key the way voluptuous does.
* **Missing complex required key (`Required(Any("a", "b"))`).** voluptuous reports it twice (the “at least one of \[…]” error plus a redundant “required key not provided”). Probatio reports the single meaningful error; the first error matches voluptuous.
* **Non-dict `Mapping` input (a `MappingProxyType`, a multidict, a custom mapping).** voluptuous rejects it with “expected a dictionary”. Probatio validates any object implementing the `Mapping` protocol and returns a plain `dict`. A genuine `dict` subclass, on the other hand, is preserved as its own class, the same as voluptuous, so a `NodeDictClass` and similar metadata-carrying subclasses survive validation. A strict superset, so existing dict code is unaffected.
* **A callable validator raising `ValueError("reason")`.** voluptuous reports a bare “not a valid value”, dropping the reason. Probatio appends it: “not a valid value: reason”. A `ValueError` with no message stays bare.
* **An enum member as a schema or mapping key (`{Color.RED: str}`).** voluptuous rejects it with `SchemaError` (added upstream in [PR #537](https://github.com/alecthomas/voluptuous/pull/537), not yet released). Probatio accepts it: a member is matched by equality as a scalar, and works as a literal key, with `Required`/`Optional`, and as a value.
* **An enum class as a schema (`Schema(Color)`).** voluptuous treats it as a type and validates by `isinstance`, so it accepts only an already-built member and rejects the member’s value. Probatio accepts a member unchanged or any value that maps to one, returning the member, so the string a loader hands you (`"red"`) validates and you get `Color.RED` back. A strict widening: input that was valid before still is, and the rejection now carries the list of valid values. Acceptance is whatever the enum’s own constructor allows, so an `IntEnum` takes a numeric equivalent (`1.0`, `True`) and an `IntFlag` builds a combined value from any integer (`Schema(Perm)(3)` is `Perm.R | Perm.W`), the same as `Coerce` of that enum. Use an explicit `In([...])` when you need to accept only the exact listed members.
* **`Coerce(T)` on a value already of type `T` whose constructor rejects it (`Coerce(uuid.UUID)(a_uuid)`).** `uuid.UUID(a_uuid)` raises, so voluptuous reports a `CoerceInvalid`. Probatio returns the value unchanged, making `Coerce` idempotent, so re-validating an already-coerced value does not fail. This only affects a value that voluptuous would have rejected (a value that converts, or is not already a `T`, is unchanged), so it is a strict widening.
* **A built-in validator on a wrong-typed value (`Replace("a", "b")(42)`, `Number()(None)`).** voluptuous leaks a raw `TypeError`/`ValueError` from the underlying call (fixed upstream in [PR #540](https://github.com/alecthomas/voluptuous/pull/540) and [PR #539](https://github.com/alecthomas/voluptuous/pull/539), not yet released). Probatio raises a clean `Invalid` (a `MatchInvalid` for `Replace`, an `Invalid` for `Number`).
* **A signaling `Decimal` (`Decimal("sNaN")`) compared by a numeric validator (`Range`, `In`, `Equal`, `MultipleOf`).** A signaling NaN raises `decimal.InvalidOperation` on any comparison, which voluptuous leaks. Probatio catches the `ArithmeticError` and reports a clean `Invalid` (the value fails the check), so a crafted value cannot leak a raw exception.
* **A quoted email local part (`"a..b"@example.com`).** voluptuous’s email regex accepts the RFC quoted-string form of the local part; Probatio’s `Email` does not (it validates the common unquoted form with plain string checks, no backtracking regex). The quoted form is effectively never used in configuration.
* **`extend` with another `Schema` (`base.extend(other_schema)`).** voluptuous raises `AssertionError` (added upstream in [PR #538](https://github.com/alecthomas/voluptuous/pull/538), not yet released). Probatio merges the extension’s keys with its `required` intent preserved (recursively); its `extra` must match the result’s, so pass its `.schema` dict for a raw merge.
* **A list with several failing items (`[{"name": str}]` against two bad items).** voluptuous stops at the first failing nested item, reporting one error (open request, [issue #171](https://github.com/alecthomas/voluptuous/issues/171)). Probatio reports every failing item, each with its index in the path. Existing error-iterating code is unaffected.
* **A set schema with a transforming element (`{Coerce(int)}`).** voluptuous returns the set with its elements untransformed (bug, [issue #400](https://github.com/alecthomas/voluptuous/issues/400)). Probatio applies the element schema to set items like it does to a list, so a `Coerce` actually coerces.
* **A failing `Any` whose branches are all concrete (`Any(int, str, None)`).** voluptuous reports only the first branch (“expected int”) or “not a valid value” (open request, [issue #412](https://github.com/alecthomas/voluptuous/issues/412)). Probatio lists every expected branch (“expected int or str or None”) as `AnyInvalid`. A branch that is an arbitrary validator has no label, so `Any` still surfaces that branch’s error.
* **An unknown key under `PREVENT_EXTRA` (`{"name": str}` given `{"nmae": ...}`).** voluptuous raises a bare `Invalid("extra keys not allowed")`. Probatio raises `ExtraKeysInvalid("not a valid option, did you mean 'name'?")`, carrying the close matches on `.candidates`. The stable `code` stays `extra_keys_not_allowed`, so code matching on it is unaffected; only the rendered string changed.
* **Keyword parameter names.** Probatio renames a few of voluptuous’s parameters for clarity: `Lower`, `Upper`, `Capitalize`, `Title`, `Strip` take `value` (voluptuous `v`); `Marker`/`Remove` take `schema` (voluptuous `schema_`); `Msg` takes `validator` (voluptuous `schema`); `DefaultTo` takes `default` (voluptuous `default_value`). Positional calls are unaffected; only keyword callers using the old names need to update.
* **`Inclusive` positional argument order.** Probatio is `Inclusive(key, group, msg=None, default=UNDEFINED, description=None)`, matching the other markers; voluptuous puts `description` before `default`. Pass `default` and `description` by keyword to be safe across both.
* **`discriminant` on `All`/`Any`.** A tagged-union discriminant is honored on `Union` (alias `Switch`), not on `All`/`Any`; passing it to `All`/`Any` has no effect. Use `Union` for a discriminated union.
* **An `ExactSequence` element error carries the element’s index in the path** (`at '[1]'`), where voluptuous reports the bare parent path. Probatio’s path is more precise; existing path-walking code sees a deeper path.
* **The order of multiple missing-required-key errors** is the schema’s definition order in Probatio, and hash (set) order in voluptuous. The first error still matches whenever any non-missing error is present; only the ordering among several all-missing keys differs.
These match what the [Compatibility](/getting-started/compatibility/) and [Migrating from voluptuous](/getting-started/migrating-from-voluptuous/) pages describe. If you hit a difference that is not here, treat it as a bug.
## What Probatio does not mirror
[Section titled “What Probatio does not mirror”](#what-probatio-does-not-mirror)
Voluptuous has no `__all__`, so `dir(voluptuous)` returns everything the module happens to import: standard-library modules, regex constants, and internal helpers, none of which are part of its intended public API. Probatio defines an explicit `__all__` and does not re-export these. Reaching for one of them was already reaching into voluptuous internals.
The notable names, so you are not surprised:
* `Enum`: the standard-library `enum.Enum`, leaked into the namespace. Reach for `import enum`.
* `Decimal`, `InvalidOperation`: the standard-library `decimal` types. Reach for `from decimal import Decimal`.
* `Generator`: `collections.abc.Generator`. Reach for `from collections.abc import Generator`.
* `collections`, `datetime`, `inspect`, `itertools`, `os`, `re`, `sys`, `typing`, `urlparse`: imported standard-library modules. Import them yourself.
* `DOMAIN_REGEX`, `USER_REGEX`, `primitive_types`: internal regex and type constants. Not public; do not depend on them.
* `DefaultFactory`, `VirtualPathComponent`: internal type aliases and helpers. Not public.
* `extra`, `cache`, `wraps`, `contextmanager`, `basestring`: internal helpers and imported builtins. Not public.
* `er`: an alias of `voluptuous.error`. Use the `error` submodule directly.
None of these are validators or markers. If your code uses one, it is coupled to a voluptuous implementation detail, and you want to import the real thing directly.
## Submodule imports
[Section titled “Submodule imports”](#submodule-imports)
The submodules resolve too, so code that imports voluptuous internals has a Probatio counterpart.
| voluptuous module | Probatio module |
| --------------------------- | --------------------- |
| `voluptuous` | `probatio` |
| `voluptuous.error` | `probatio.error` |
| `voluptuous.validators` | `probatio.validators` |
| `voluptuous.humanize` | `probatio.humanize` |
| `voluptuous.schema_builder` | `probatio.schema` |
For dependencies that import voluptuous internals under the `voluptuous` name (not Probatio), `probatio.compat.install_as_voluptuous()` aliases Probatio into `sys.modules` so every later `import voluptuous` resolves to probatio. The [Compatibility](/getting-started/compatibility/) page covers what it registers and how and when to call it.
# Errors
> The error hierarchy Probatio raises, the fields on Invalid, and the default code for every semantic subclass.
This page is the reference for what Probatio raises. For the how-to (catching, reading paths, humanizing), see the [error handling guide](/guides/error-handling/).
## The error model
[Section titled “The error model”](#the-error-model)
Everything Probatio raises descends from one base, `Error`. Below it sit two unrelated kinds of failure: a broken schema and bad data.
* `Error`: the base for everything Probatio raises. Catch this to catch all of it.
* `SchemaError`: the schema *definition* is wrong. That is a programming mistake, not bad input, so it is kept separate from `Invalid`.
* `Invalid`: the data did not match the schema. Carries a `path` to the offending value.
* `MultipleInvalid`: a collection of `Invalid` errors from one validation pass. It is itself an `Invalid`, and it proxies its first error, so `msg`, `path`, `code`, and the rest read the first failure without reaching into `errors`.
A `Schema` always raises `MultipleInvalid` at the top, even for a single failure. So when you catch one, the individual failures live in `err.errors`, and `err.errors[0]` is the first.
## The fields on Invalid
[Section titled “The fields on Invalid”](#the-fields-on-invalid)
Every `Invalid` carries two layers. The voluptuous-compatible layer (`msg`, `path`, `error_message`, `error_type`) keeps its original wording and behavior, so code that reads those fields still works. Only the rendered `str(error)` differs from voluptuous: it appends the path as a dotted trail and drops the error-type clause. The structured layer (`code`, `context`, `translation_key`, `placeholders`, `as_dict()`) is additive: it changes none of the legacy output.
* `msg`: the human-readable message.
* `path`: the list of keys and indices to walk to the offending value.
* `error_message`: the bare message, without the path appended.
* `error_type`: the kind of value that failed, such as `dictionary value` (may be `None`). Kept for compatibility; it is no longer rendered in `str(error)`.
* `code`: the stable machine-readable code (the class default unless overridden).
* `secret`: whether the offending value must be redacted, not echoed, when rendered. Set for an error under a [`Secret`](/guides/dict-schemas-and-markers/#redacting-secret-values) key, so `humanize_error` (and any consumer building its own output) shows a placeholder instead of the value.
* `context`: a dict of structured detail about the failure, such as the expected type.
* `translation_key`: the stable key naming the message’s sentence, filled by every built-in validator (see the [translation keys reference](/reference/translation-keys/)); `None` on a bare `Invalid` raised without one.
* `placeholders`: the raw values the message interpolates, like `{"min": 10}`.
* `as_dict()`: the structured layer rendered as a serializable dict, handy for a JSON API.
Read `.path`, `.code`, and `.as_dict()` off a caught error. Remember the schema raises `MultipleInvalid`, so reach into `err.errors[0]`:
```python
from probatio import Schema, MultipleInvalid
schema = Schema({"server": {"ports": [int]}})
try:
schema({"server": {"ports": [80, "nope"]}})
except MultipleInvalid as err:
first = err.errors[0]
print(first.path) # ['server', 'ports', 1]
print(first.code) # type
print(first.as_dict()["context"]) # {'expected': 'int'}
```
On a `MultipleInvalid`, `as_dict()` renders the whole collection instead, under an `errors` key:
```python
from probatio import Schema, MultipleInvalid
schema = Schema({"a": int})
try:
schema({"a": "x"})
except MultipleInvalid as err:
print(err.as_dict()["errors"][0]["code"]) # type
```
To render a path the way `str(error)` does, use `render_path` from `probatio.error`. A consumer building its own error output stays consistent with the library’s rendering:
```python
from probatio.error import render_path
print(render_path(["server", "ports", 1])) # server.ports[1]
```
## Semantic subclasses
[Section titled “Semantic subclasses”](#semantic-subclasses)
The subclasses let callers branch on what went wrong. They mirror voluptuous, so the names match what you already catch. Each carries a stable `default_code`.
Each entry shows the class, its meaning, and its `default_code` in parentheses.
* `RequiredFieldInvalid` (`required`): a required key was missing from the data.
* `ObjectInvalid` (`object`): the value is not the expected object.
* `DictInvalid` (`not_a_dictionary`): the value is not a mapping.
* `ExtraKeysInvalid` (`extra_keys_not_allowed`): a key matched no schema key under `PREVENT_EXTRA`; carries `candidates`, the close matches.
* `SequenceTypeInvalid` (`not_a_sequence`): the value is not the expected sequence type (list, tuple, set).
* `TypeInvalid` (`type`): the value is not of the expected type.
* `ValueInvalid` (`value`): a validator rejected the value.
* `ScalarInvalid` (`not_valid`): the value does not match a scalar literal.
* `LiteralInvalid` (`not_valid`): the value does not match a `Literal`.
* `CoerceInvalid` (`coerce`): a value could not be coerced to the requested type; for an `Enum` with string values, carries `candidates`, the close matches.
* `EnumInvalid` (`enum`): the value is not a member of the expected enum, nor one of its values (raised by an enum class used as a schema).
* `AnyInvalid` (`no_match`): the value matched none of the candidates.
* `AllInvalid` (`all`): the value failed one of a chain of validators.
* `MatchInvalid` (`match`): the value does not match the expected pattern.
* `RangeInvalid` (`range`): the value falls outside the allowed range.
* `LengthInvalid` (`length`): the value’s length falls outside the allowed bounds.
* `InInvalid` (`not_in_list`): the value is not a member of the allowed set; carries `candidates`, the close matches among string members.
* `NotInInvalid` (`in_list`): the value is a member of a disallowed set.
* `ContainsInvalid` (`contains`): the collection does not contain the required element.
* `ExactSequenceInvalid` (`exact_sequence`): the sequence does not match the expected exact sequence.
* `ExclusiveInvalid` (`exclusive`): more than one key from a mutually exclusive group was provided.
* `InclusiveInvalid` (`inclusive`): some, but not all, keys from a co-dependent group were provided.
* `TrueInvalid` (`not_true`): the value is not truthy.
* `FalseInvalid` (`not_false`): the value is not falsy.
* `BooleanInvalid` (`boolean`): the value could not be read as a boolean.
* `UrlInvalid` (`url`): the value is not a valid URL.
* `EmailInvalid` (`email`): the value is not a valid email address.
* `DirInvalid` (`not_a_directory`): the value is not an existing directory.
* `FileInvalid` (`not_a_file`): the value is not an existing file.
* `PathInvalid` (`not_a_path`): the value is not an existing path.
* `SymlinkInvalid` (`not_a_symlink`): the value is not an existing symbolic link.
* `SocketInvalid` (`not_a_socket`): the value is not an existing socket.
* `FifoInvalid` (`not_a_fifo`): the value is not an existing named pipe (FIFO).
* `BlockDeviceInvalid` (`not_a_block_device`): the value is not an existing block device.
* `DatetimeInvalid` (`datetime`): the value is not a valid datetime.
* `DateInvalid` (`date`): the value is not a valid date.
* `TimeInvalid` (`time`): the value is not a valid time of day.
* `DurationInvalid` (`duration`): the value is not a valid duration.
* `TimeZoneInvalid` (`time_zone`): the value is not a valid time zone: an IANA name (`TimeZoneInfo`) or a UTC offset (`TimeZone`, `AsTimezone`).
* `EpochInvalid` (`epoch`): the value is not a valid Unix epoch timestamp.
* `IpInvalid` (`ip`): the value is not a valid IP address or network.
* `MacAddressInvalid` (`mac_address`): the value is not a valid MAC address.
* `UuidInvalid` (`uuid`): the value is not a valid UUID.
* `HostnameInvalid` (`hostname`): the value is not a valid hostname or domain name.
* `SlugInvalid` (`slug`): the value is not a valid slug.
* `MultipleOfInvalid` (`multiple_of`): the value is not a multiple of the factor.
* `JsonInvalid` (`json`): the value is not valid JSON.
* `ImmutableInvalid` (`immutable`): a field that may not change (`Immutable` or `WriteOnce`) was changed.
* `NotEnoughValid` (`not_enough_valid`): too few of a `SomeOf` group’s validators passed.
* `TooManyValid` (`too_many_valid`): too many of a `SomeOf` group’s validators passed.
Note
`ScalarInvalid` and `LiteralInvalid` share the code `not_valid`. The `code` is a classification, not a unique identifier per class. Branch on the exception type when you need to tell them apart.
The subclasses live in `probatio.error`. Catch them there:
```python
from probatio import Schema, Range, MultipleInvalid
from probatio.error import RangeInvalid
schema = Schema(Range(min=0, max=10))
try:
schema(99)
except MultipleInvalid as err:
first = err.errors[0]
print(isinstance(first, RangeInvalid)) # True
print(first.code) # range
```
# Performance
> How Probatio validates fast, how it compiles a hot schema, how it compares to voluptuous and mashumaro, and how to measure it yourself.
Probatio is pure Python with no native extension. That is a deliberate choice (see [ADR-001](https://github.com/frenck/probatio/blob/main/adr/001-clean-room-reimplementation-of-voluptuous.md) and [ADR-011](https://github.com/frenck/probatio/blob/main/adr/011-opt-in-compiled-schema.md)): the drop-in promise and easy installation matter more than a native core. Even so, the interpreted engine is well ahead of voluptuous, and a hot schema compiles itself into a specialized validator for more speed. This page explains the cost model and how to measure it, with honest numbers.
## The cost model: build once, call cheap
[Section titled “The cost model: build once, call cheap”](#the-cost-model-build-once-call-cheap)
A `Schema` compiles its definition once, when you construct it. After that, calling it is the cheap part. So the rule is simple: build the schema once, then reuse it.
```python
from probatio import Schema, All, Coerce, Range
# Built once. Reused for every value.
PORT = Schema(All(Coerce(int), Range(min=1, max=65535)))
PORT("443") # 443
PORT(8080) # 8080
```
The expensive work (walking the definition, resolving markers, wiring up the validators) happens in the constructor. If you rebuild the schema on every call, you pay that cost every time for nothing.
Tip
Build schemas at module scope, as module-level constants. Import and call them. Do not construct a `Schema` inside a function that runs in a hot loop.
## Two engines: interpreted and compiled
[Section titled “Two engines: interpreted and compiled”](#two-engines-interpreted-and-compiled)
Validation runs on one interpreted engine. On top of that, a schema can compile itself into a flat function generated for its exact shape, with the per-key work unrolled and the common validators inlined. By default this is automatic: a hot schema compiles itself once it has been called enough to be worth it, and a one-shot schema stays interpreted. Compiling only changes speed; the result is identical, because anything off the happy path bails to the interpreted engine.
You do not have to do anything to get this. To opt a specific schema in eagerly, or to turn the behavior off for an unusual workload, see [Compiled schemas](/guides/compiled-schemas/).
## How it compares to voluptuous
[Section titled “How it compares to voluptuous”](#how-it-compares-to-voluptuous)
On the bundled benchmark scenarios, the interpreted engine runs roughly 2.3 to 3.4 times faster than the same schema in voluptuous, and the compiled path lands between 6.7 and 7.4 times faster on the same scenarios. The exact ratio moves with the machine, the Python version, and the shape of the schema, so treat these as ballparks, not promises.
| Scenario | voluptuous | Probatio | Probatio compiled |
| -------------------------------- | ---------- | -------- | ----------------- |
| Flat types (`{str, int, ...}`) | 3.7 µs | 1.1 µs | 0.5 µs |
| Config (coerce, range, in, list) | 6.0 µs | 2.2 µs | 0.9 µs |
| Nested mapping | 7.0 µs | 3.0 µs | 1.0 µs |
Microseconds per validation, lower is faster. Measured July 2026 on one machine with Python 3.13.

Read the benchmark honestly
`bench/bench.py` is a rough, single-machine comparison. It builds equivalent schemas, validates a fixed payload many times in a plain loop, and prints the ratio. It is useful for a sanity check and for catching a large regression. It is not a rigorous benchmark, and the numbers are machine-dependent. Do not quote them as guarantees.
The tracked benchmarks are the CodSpeed ones in `bench/test_benchmarks.py`. Those run per pull request, so a performance regression shows up in review. They pin each case to a known engine (interpreted or compiled) so the two are tracked separately. The tracked benchmarks live on [CodSpeed](https://codspeed.io/frenck/probatio), so you can look at the history yourself.
## How a dataclass compares to mashumaro
[Section titled “How a dataclass compares to mashumaro”](#how-a-dataclass-compares-to-mashumaro)
For the dataclass path there is also a comparison against [mashumaro](https://github.com/Fatal1ty/mashumaro), which generates a `from_dict` per class. It is not a like-for-like comparison, and that is the point: mashumaro deserializes and largely trusts the declared types, while Probatio *validates* every field against its type and then constructs. mashumaro does strictly less work, so it is faster on already-correct input. The compiled Probatio path lands within roughly 1.4x of it on a small dataclass while still validating, and the gap all but closes as the field count grows (about 1.1x on a wide one).
| Dataclass | mashumaro | Probatio | Probatio compiled |
| ---------------- | --------- | -------- | ----------------- |
| Small (4 fields) | 0.5 µs | 1.5 µs | 0.7 µs |
| Wide (12 fields) | 1.2 µs | 2.8 µs | 1.3 µs |

The two libraries pair well: validate untrusted input with Probatio, then hand trusted dataclasses to mashumaro to serialize. See [Comparison to alternatives](/project/comparison/) for where each one fits.
## How it compares across libraries
[Section titled “How it compares across libraries”](#how-it-compares-across-libraries)
There is also a cross-library comparison on the dict-to-object path, in the spirit of mashumaro’s benchmark suite: one representative nested record, handed to each library as the same dict, timed turning it into a validated or constructed object.
Be honest about what is being compared, because these libraries do different amounts of work. *Validators* (Probatio, pydantic, marshmallow) check every field against its type and would reject a mismatch; voluptuous validates but returns a dict rather than constructing an object. *Deserializers* (mashumaro, cattrs, dacite, dataclasses-json) build the object and largely trust the declared types. Probatio appears in both groups: its normal call validates, and its opt-in [`construct`](/guides/dataclasses/#trusted-construction-without-validation) builds from trusted input without validating, the same job the deserializers do.

Two honest readings. On the **validate** path, compiled Probatio lands in the leading group, ahead of voluptuous, pydantic v1, dacite, marshmallow, and dataclasses-json, and within about 1.7x of the Rust-cored pydantic v2, while staying pure Python and validating every field. On the **trust the types** path, `construct` is the fastest in the whole field, ahead of mashumaro and cattrs, because it is a purpose-built constructor generated for that one dataclass. Run it yourself with `just bench-world`.
Narrowing to the like-for-like group, only the libraries that actually validate, makes the placement clearer:

Among validators, the only thing faster than compiled Probatio is pydantic v2, and it gets there with a native core (the Rust `pydantic-core`). Probatio is right behind it and ahead of every other validator, while being pure Python, no extension to build, nothing to compile at install. That is the trade Probatio is built around: the speed is close to the native option, the install is a plain `pip install`.
Where the native core actually wins
pydantic v2’s lead here comes from the workload: this record is parsing-heavy (ISO datetime strings, enums, decoded out of JSON), which is exactly what a Rust core is good at. The advantage is not uniform. It shrinks on a schema dominated by your own custom validators, because a native core cannot run a Python validator natively, it has to call it across the Rust boundary (efficiently, but it is still a Python call), so the speedup is muted right where the work is your code. The Rust win is real in the parsing-heavy regime, not everywhere.
## Measuring it yourself
[Section titled “Measuring it yourself”](#measuring-it-yourself)
The rough comparison against voluptuous:
```bash
just bench
```
The dataclass comparison against mashumaro:
```bash
just bench-dataclass
```
The cross-library comparison (syncs the extra libraries on first run):
```bash
just bench-world
```
These print a small table of per-validation times and ratios. Run them a few times; the absolute numbers wander. The charts on this page are regenerated from the same data with `just charts`.
The tracked CodSpeed benchmarks (walltime locally, tracked in CI):
```bash
just codspeed
```
These are not part of the normal test run. They exercise the hot paths interpreted and compiled: validating a config-style mapping, a dataclass, a nested mapping, a composed schema, and a combinator under the automatic compile policy; calling a `@probatio`-decorated function; compiling a schema from scratch; and the generation step itself.
To dig into a hot spot, the profiler wraps cProfile (and py-spy) over the generator and the validators it produces:
```bash
just profile run-config # see bench/profiling.py for the targets
```
## Practical tips
[Section titled “Practical tips”](#practical-tips)
* Build each schema once, at module scope, and reuse it. This is the single biggest win, and it is also what lets a hot schema compile itself.
* Never rebuild a schema inside a hot loop. Compilation is the expensive part.
* Reuse nested schemas too. A `Schema` you reference from another schema is built once and shared.
* Let `AUTO` do its job. You rarely need to reach for the `compile` flag by hand; the hot schemas speed themselves up. See [Compiled schemas](/guides/compiled-schemas/) for when you do.
## Reuse and concurrency
[Section titled “Reuse and concurrency”](#reuse-and-concurrency)
A compiled `Schema` is meant to be built once and called many times, and that is also what makes it safe to share. Validating reads the compiled checks and returns a new value without changing what the schema accepts, so the same `Schema` instance can be called from multiple threads at once. The one thing a call can change is the engine under the hood: a hot schema compiles itself on first use (see [Compiled schemas](/guides/compiled-schemas/)), and that one-time swap is synchronized, so two threads racing a cold schema resolve it once and both get the right result. Build it once at import time and let every worker use it.
The one caveat is your own code. If you write a custom validator that keeps mutable state, or pass a `default` factory that is not safe to call concurrently, that state is yours to make thread-safe. Probatio’s own validators hold no per-call state.
# Translation keys
> Every translation key the built-in validators emit, with its English template and placeholders.
Every error a built-in validator raises carries a `translation_key` naming the exact sentence of its default message, and `placeholders`, the raw values that sentence interpolates. Together they let a consumer render errors in its own words or language without parsing strings; see [Custom error messages](/guides/custom-error-messages/) for the pattern.
The keys are public API: renaming or removing one is a breaking change, the same as renaming a function. Where two validators produce the same sentence, they share the key, so one translation covers both. A custom `msg=` on a validator replaces the rendered text but keeps the key.
Templates are `str.format` style. The canonical source is the catalog in `probatio._messages`; this table mirrors it. One entry is a fragment rather than a sentence: `did_you_mean` is appended to a suggestion-carrying error’s message, with the close matches also available raw on `context["candidates"]` (see [how suggestions compose](/guides/custom-error-messages/#suggestions-compose-on-top)).
| Key | English template |
| ---------------------------------- | --------------------------------------------------------------------------------------------------- |
| `allowed_at_most_one_of` | `at most one of {keys} is allowed` |
| `allowed_one_of` | `exactly one of {keys} is allowed` |
| `byte_length_out_of_bounds` | `byte length out of bounds` |
| `cannot_be_changed` | `{field!r} cannot be changed` |
| `cannot_convert_to_set` | `cannot be converted to a set: {detail}` |
| `contains_duplicate_items` | `contains duplicate items: {items}` |
| `contains_unhashable_elements` | `contains unhashable elements: {detail}` |
| `did_you_mean` | `, did you mean {candidates}?` |
| `does_not_match_pattern` | `does not match the expected pattern` |
| `element_not_valid` | `item {position} ({item}) does not match any validator` |
| `exclusive_group` | `two or more values in the same group of exclusion {group!r}` |
| `expected_alpha` | `expected only ASCII letters` |
| `expected_alphanumeric` | `expected only ASCII letters and digits` |
| `expected_ascii` | `expected only ASCII characters` |
| `expected_base64_string` | `expected a Base64 string` |
| `expected_boolean` | `expected a boolean` |
| `expected_cidr` | `expected a CIDR network` |
| `expected_collection` | `expected a collection: {detail}` |
| `expected_credit_card_number` | `expected a credit card number` |
| `expected_data_uri` | `expected a data URI` |
| `expected_discriminator` | `expected {key} to be one of {values}` |
| `expected_duration` | `expected a duration` |
| `expected_duration_detailed` | `expected a duration like H:MM, H:MM:SS, an ISO 8601 duration like PT1H30M, or a number of seconds` |
| `expected_email_address` | `expected an email address` |
| `expected_fqdn` | `expected a fully-qualified domain name` |
| `expected_hex_color` | `expected a hex color like #rrggbb` |
| `expected_hex_string` | `expected a hex string` |
| `expected_hexadecimal_integer` | `expected a hexadecimal integer` |
| `expected_hostname` | `expected a hostname` |
| `expected_iana_time_zone` | `expected an IANA time zone` |
| `expected_iban` | `expected an IBAN` |
| `expected_ip` | `expected an IP address` |
| `expected_ipv4` | `expected an IPv4 address` |
| `expected_ipv6` | `expected an IPv6 address` |
| `expected_iso_date` | `expected an ISO 8601 date` |
| `expected_iso_datetime` | `expected an ISO 8601 datetime` |
| `expected_iso_time` | `expected an ISO 8601 time` |
| `expected_json_string` | `expected a JSON string` |
| `expected_mac_address` | `expected a MAC address` |
| `expected_mapping` | `expected a mapping` |
| `expected_no_whitespace` | `expected no whitespace` |
| `expected_object` | `expected a {cls!r}` |
| `expected_percentage` | `expected a percentage between 0 and 100` |
| `expected_phone_number` | `expected a phone number` |
| `expected_port` | `expected a port number between 1 and 65535` |
| `expected_printable_ascii` | `expected only printable ASCII characters` |
| `expected_sequence` | `expected a {expected}` |
| `expected_sequence_of_items` | `expected a sequence of {count} items` |
| `expected_slug` | `expected a slug` |
| `expected_string` | `expected a string` |
| `expected_timezone_aware_datetime` | `expected a timezone-aware datetime` |
| `expected_type` | `expected {expected}` |
| `expected_type_or_one_of` | `expected {expected} or one of {values}` |
| `expected_ulid` | `expected a ULID` |
| `expected_unix_timestamp` | `expected a Unix timestamp` |
| `expected_url` | `expected a URL` |
| `expected_url_with_fqdn` | `expected a URL with a fully-qualified domain name` |
| `expected_utc_offset` | `expected a UTC offset like +01:00, Z, or UTC` |
| `expected_uuid` | `expected a UUID` |
| `expected_uuid_version` | `expected a version {version} UUID` |
| `expected_valid_regex` | `expected a valid regular expression` |
| `inclusive_group` | `some but not all values in the same group of inclusion {group!r}` |
| `invalid_credit_card_number` | `invalid credit card number` |
| `invalid_data_uri` | `invalid data URI` |
| `invalid_iban` | `invalid IBAN` |
| `invalid_json` | `invalid JSON` |
| `invalid_phone_number` | `invalid phone number` |
| `invalid_value` | `invalid value` |
| `invalid_value_or_type` | `invalid value or type` |
| `key_not_allowed` | `key not allowed` |
| `length_max` | `length of value must be at most {max}` |
| `length_min` | `length of value must be at least {min}` |
| `max_contains` | `expected at most {max} matching item(s)` |
| `min_contains` | `expected at least {min} matching item(s)` |
| `must_not_match_not_schema` | `value must not match the 'not' schema` |
| `no_valid_value` | `no valid value found` |
| `not_a_block_device` | `not a block device` |
| `not_a_collection` | `value is not a collection` |
| `not_a_directory` | `not a directory` |
| `not_a_fifo` | `not a FIFO` |
| `not_a_file` | `not a file` |
| `not_a_path` | `not a path` |
| `not_a_socket` | `not a socket` |
| `not_a_symlink` | `not a symlink` |
| `not_a_valid_option` | `not a valid option` |
| `not_a_valid_value` | `not a valid value` |
| `not_a_valid_value_detail` | `not a valid value: {detail}` |
| `path_does_not_exist` | `path does not exist` |
| `precision_must_equal` | `precision must be equal to {precision}` |
| `range_max` | `value must be at most {max}` |
| `range_max_exclusive` | `value must be lower than {max}` |
| `range_min` | `value must be at least {min}` |
| `range_min_exclusive` | `value must be higher than {min}` |
| `recursion_too_deep` | `data is nested too deeply for this recursive schema` |
| `required` | `required key not provided` |
| `required_any_of` | `at least one of {keys} is required` |
| `required_none_or_all_of` | `either none or all of {keys} are required` |
| `required_one_of` | `exactly one of {keys} is required` |
| `required_when_absent` | `{key!r} is required when {triggers} is absent` |
| `required_when_present` | `{key!r} is required when {triggers} is present` |
| `required_when_value` | `{key!r} is required when {conditions}` |
| `scale_must_equal` | `scale must be equal to {scale}` |
| `too_many_valid` | `value matched {passed} alternatives, expected at most {max}` |
| `value_does_not_match_format` | `value does not match expected format {format}` |
| `value_has_no_precision` | `value has no precision` |
| `value_multiple_of` | `value must be a multiple of {factor}` |
| `value_must_be_number_string` | `value must be a number enclosed in a string` |
| `value_must_contain` | `value must contain {item!r}` |
| `value_must_end_with` | `value must end with {suffix!r}` |
| `value_must_start_with` | `value must start with {prefix!r}` |
| `value_no_length` | `value has no length` |
| `value_not_allowed` | `value is not allowed` |
| `value_not_empty` | `value must not be empty` |
| `value_not_equal` | `value is not equal to {target!r}` |
| `value_not_one_of` | `value must not be one of {values}` |
| `value_not_sorted` | `value is not sorted` |
| `value_one_of` | `value must be one of {values}` |
| `value_was_not_false` | `value was not false` |
| `value_was_not_true` | `value was not true` |
| `write_once_already_set` | `{field!r} is write-once and already set` |
Three keys are JSON Schema-only: `max_contains`, `min_contains`, and `must_not_match_not_schema`. They fire only from schemas built by `from_json_schema` (see the [JSON Schema guide](/guides/json-schema/)); no hand-written validator emits them.
# Typing
> What static type checking does and does not give you when you validate with Probatio.
Probatio is typed, and it ships its types. What that buys you is honest but limited: your calls into Probatio are checked, but the *result* of a validation is not a statically known shape. This page spells out the difference, so you know what the type checker is actually doing for you.
## Probatio ships its types
[Section titled “Probatio ships its types”](#probatio-ships-its-types)
The package includes a `py.typed` marker, so type checkers read Probatio’s own annotations instead of treating it as untyped. Probatio itself is checked in CI with both [mypy](https://mypy-lang.org/) and [ty](https://github.com/astral-sh/ty), under mypy’s strict mode. The public API carries annotations, and they are kept honest by those runs.
That means when you call a validator or build a schema, the checker sees real signatures and flags the obvious mistakes.
```python
from probatio import Range
Range(min=0, max=10) # checked: a Range validator
```
## A schema cannot describe its result
[Section titled “A schema cannot describe its result”](#a-schema-cannot-describe-its-result)
Here is the part to be clear about. A schema is arbitrary data: a type, a literal, a callable, a nested container, any combination. The validated result depends on what that data says at runtime, including normalization like `Coerce` and default values. A static checker cannot read a schema value and work out the shape it produces.
So `Schema.__call__` returns `Any`. The checker will not infer that `schema({"name": "app"})` gave you a dict with a `name` key of type `str`. It gives you back something untyped, and from there you are on your own.
```python
from probatio import Schema, Required
schema = Schema({Required("name"): str})
result = schema({"name": "app"}) # result is typed as Any
result # {'name': 'app'}
```
Probatio validates at runtime. It checks that the data matches the schema, and raises if it does not. It does not hand you a statically typed model the way a model-class library (something built on dataclasses or a model base class) does. The guarantee is a runtime one.
Note
This is not a Probatio shortcoming to be fixed later. It follows from the design: a schema is a value, not a type declaration, so its result type is not knowable to a checker. voluptuous works the same way.
If you want a statically typed object after validation, construct it yourself from the validated data. Probatio confirms the data is well-formed; you turn it into a typed object:
```python
from dataclasses import dataclass
from probatio import Schema, Required
@dataclass
class Config:
name: str
port: int
schema = Schema({Required("name"): str, Required("port"): int})
def load(raw: object) -> Config:
"""Validate, then build a typed object the checker understands."""
data = schema(raw) # runtime-checked, typed Any
return Config(name=data["name"], port=data["port"])
load({"name": "app", "port": 8080}) # Config(name='app', port=8080)
```
After the `Config(...)` line the checker knows the type again, because you told it. The validation guarantees the values are present and of the right kind at runtime, so the construction is safe.
## Typed results with `DataclassSchema`
[Section titled “Typed results with DataclassSchema”](#typed-results-with-dataclassschema)
There is a shorter path that does the same thing for you. `DataclassSchema` reads a dataclass and builds the schema from its annotations, and it is generic in the dataclass type, so the checker infers the result. `DataclassSchema(Config)` is a `DataclassSchema[Config]`, and calling it is typed as returning a `Config`, not `Any`:
```python
from dataclasses import dataclass
from probatio import DataclassSchema
@dataclass
class Config:
name: str
port: int = 8080
schema = DataclassSchema(Config)
config = schema({"name": "app"}) # config is typed as Config
config.port # 8080, and the checker knows .port is an int
```
This carries a static type because the schema came from a type (the dataclass) rather than arbitrary data. You write the shape once, as the dataclass, and get both the runtime validation and the static type from it. See [schemas from dataclasses](/guides/dataclasses/) for the full field mapping.
## Typed results with `TypedDictSchema`
[Section titled “Typed results with TypedDictSchema”](#typed-results-with-typeddictschema)
`TypedDictSchema` does the same from a `TypedDict`, and it is the interesting case for typing. A `TypedDict` is a plain dict at runtime, so there is nothing to construct: the validated dict *is* the result, and the schema is generic, so the checker types it as the `TypedDict`. You get the static type at no runtime cost, and `result["key"]` access keeps working, because it really is a dict.
```python
from typing import TypedDict
from probatio import TypedDictSchema
class Config(TypedDict):
name: str
port: int
schema = TypedDictSchema(Config)
config = schema({"name": "app", "port": 8080}) # typed as Config
config["port"] # 8080, and the checker knows it is an int
```
So there are two ways to get a typed result, and they differ in what comes back. `DataclassSchema` constructs an instance (attribute access, a real type, a construction cost). `TypedDictSchema` returns the validated dict unchanged, typed as the `TypedDict` (dict access, zero construction). Reach for the dataclass when you want an object, the TypedDict when you want typed dict-shaped data without changing how that data is used. Both are checked: the checker knows the keys, and flags an unknown one.
## `Schemable`
[Section titled “Schemable”](#schemable)
`Schemable` is the type alias for “anything you can use as a schema”. It is defined as an alias of `Any`:
```text
type Schemable = Any
```
It is exposed so you can annotate code that accepts or stores schemas, mirroring voluptuous. Because a schema can be any of the accepted forms (a type, a literal, a callable, a container, a nested `Schema`), there is no narrower type that fits, so the alias is `Any`. Use it for intent and readability, not for narrowing:
```python
from probatio import Schema, Schemable
def make_validator(definition: Schemable) -> Schema:
"""Build a Schema from any schemable definition."""
return Schema(definition)
make_validator({"port": int}) # a compiled Schema
```
The annotation documents what the argument is for. It does not constrain it beyond what `Any` allows.
## The rest of the API is typed
[Section titled “The rest of the API is typed”](#the-rest-of-the-api-is-typed)
Everything you call is annotated, so the call sites are checked even though the validated result is not. The validators, the markers, the combinators, the error classes: all carry signatures. Passing the wrong kind of argument to a validator, or mishandling an error, is caught.
```python
from probatio import All, Coerce, Range, Schema
schema = Schema(All(Coerce(int), Range(min=0))) # all checked at the call site
schema("42") # 42
```
## What you get, and what you do not
[Section titled “What you get, and what you do not”](#what-you-get-and-what-you-do-not)
What static typing gives you here:
* Probatio’s public API is annotated and ships `py.typed`, so your editor and type checker see it.
* Calls into Probatio (building schemas, calling validators, catching errors) are checked against real signatures.
* Probatio’s own code is checked under strict mypy and under ty in CI.
What it does not give you:
* A statically typed result. `schema(data)` returns `Any`; the checker does not know the shape of what comes back.
* Compile-time verification of a schema’s structure against your data’s type. That check happens at runtime, when you call the schema.
* A typed model object for free, with two exceptions: `DataclassSchema` is generic and returns its dataclass type, and `TypedDictSchema` is generic and returns its TypedDict type (the validated dict, typed, at no construction cost). For a plain `Schema`, build the typed object from the validated data yourself.
Tip
Validate at the boundary, construct your typed object right after, and keep the typed object for the rest of the program. You get the runtime guarantee from Probatio and the static guarantee from your own type.