Skip to main content

Handling Extra Fields and Totality

This guide demonstrates how to manage unexpected input fields and field requirement (totality) using CoreConfig and specific schema types like TypedDictSchema and ModelFieldsSchema.

Configuring Extra Fields Behavior​

You can control how the validator handles fields that are not defined in the schema using the extra_behavior setting. The available options are:

  • 'ignore': (Default) Extra fields are silently dropped.
  • 'allow': Extra fields are preserved and included in the output.
  • 'forbid': Validation fails if extra fields are present.

Allowing Extra Fields in TypedDicts​

To allow extra fields in a TypedDict, set extra_fields_behavior in the CoreConfig or extra_behavior directly in the typed_dict_schema.

from pydantic_core import SchemaValidator, core_schema

v = SchemaValidator(
    core_schema.typed_dict_schema(
        fields={
            'a': core_schema.typed_dict_field(schema=core_schema.int_schema()),
        },
        config=core_schema.CoreConfig(extra_fields_behavior='allow'),
    )
)

# Extra field 'b' is preserved in the output
assert v.validate_python({'a': 1, 'b': 2}) == {'a': 1, 'b': 2}

Forbidding Extra Fields in Models​

For ModelFieldsSchema, you can forbid extra fields to ensure strict adherence to the defined structure.

from pydantic_core import SchemaValidator, core_schema, ValidationError
import pytest

v = SchemaValidator(
    core_schema.model_fields_schema(
        fields={'a': core_schema.model_field(schema=core_schema.int_schema())},
        extra_behavior='forbid'
    )
)

with pytest.raises(ValidationError):
    v.validate_python({'a': 1, 'b': 2})

Handling Totality in TypedDicts​

Totality determines whether all fields defined in a TypedDict are required by default. This can be configured globally via CoreConfig or per schema.

Setting Global Totality​

Use typed_dict_total in CoreConfig to change the default requirement for all TypedDict fields.

from pydantic_core import SchemaValidator, core_schema

v = SchemaValidator(
    core_schema.typed_dict_schema(
        fields={
            'a': core_schema.typed_dict_field(schema=core_schema.int_schema()),
            'b': core_schema.typed_dict_field(schema=core_schema.int_schema()),
        },
        config=core_schema.CoreConfig(typed_dict_total=False),
    )
)

# Both fields are now optional
assert v.validate_python({'a': 1}) == {'a': 1}

Overriding Totality per Schema​

You can override the global configuration by setting the total attribute in TypedDictSchema.

v = SchemaValidator(
    core_schema.typed_dict_schema(
        fields={
            'a': core_schema.typed_dict_field(schema=core_schema.int_schema()),
        },
        total=True, # Overrides config or default
    )
)

Validating Extra Fields​

When extra_behavior is set to 'allow', you can provide an extras_schema to validate the values of the unexpected fields.

from pydantic_core import SchemaValidator, core_schema

v = SchemaValidator(
    core_schema.model_fields_schema(
        fields={'a': core_schema.model_field(schema=core_schema.int_schema())},
        extra_behavior='allow',
        extras_schema=core_schema.str_schema() # All extra fields must be strings
    )
)

# 'b' is validated as a string
data, extra, fields_set = v.validate_python({'a': 1, 'b': 'hello'})
assert extra == {'b': 'hello'}

Validation-Time Overrides​

The behavior for extra fields can be overridden at the time of validation using the extra parameter in validate_python.

from pydantic_core import SchemaValidator, core_schema

v = SchemaValidator(
    core_schema.typed_dict_schema(
        fields={'a': core_schema.typed_dict_field(schema=core_schema.int_schema())}
    )
)

# Override default 'ignore' behavior to 'forbid' for this specific call
try:
    v.validate_python({'a': 1, 'b': 2}, extra='forbid')
except ValidationError:
    print("Extra fields forbidden at runtime")

Troubleshooting and Return Types​

When working with extra_behavior='allow', be aware of the different return types between schema types:

  • TypedDictSchema: Returns a single dictionary containing both defined and extra fields.
  • ModelFieldsSchema: Returns a tuple of (data, extra, fields_set).
    • data: A dictionary of validated fields defined in the schema.
    • extra: A dictionary of validated extra fields.
    • fields_set: A set of keys that were present in the input.

Gotcha: Missing Extras Schema​

If you set extra_behavior='allow' but do not provide an extras_schema, the extra fields will be accepted without any validation (effectively treated as Any). If you need to ensure extra fields meet certain criteria (e.g., must be integers), always define an extras_schema.