Handling Nullable and Optional Values

In Python, None is frequently used to represent both a "null" value and the absence of a value. This ambiguity creates significant challenges for data validation libraries like Pydantic, which must distinguish between a field that was explicitly set to None and a field that was omitted entirely. To solve this, this project implements a specialized NullableSchema and a Some wrapper class to provide clear semantics for optionality and nullability.

The Nullable Schema

The NullableSchema is a core schema type that explicitly allows a value to be None. Rather than treating None as just another type in a union, the NullableSchema acts as a wrapper around another CoreSchema.

Defined in pydantic-core/python/pydantic_core/core_schema.py, the NullableSchema structure is as follows:

class NullableSchema(TypedDict, total=False):
    type: Required[Literal['nullable']]
    schema: Required[CoreSchema]
    strict: bool
    ref: str
    metadata: dict[str, Any]
    serialization: SerSchema

The primary way to create this schema is through the nullable_schema() helper function. When the validator encounters None, it returns it immediately as a valid result. If the input is anything else, it delegates validation to the inner schema.

Example: Basic Nullable Validation

This example from pydantic-core/tests/validators/test_nullable.py demonstrates how the nullable wrapper allows None while still enforcing the constraints of the inner schema for non-null values:

from pydantic_core import SchemaValidator, core_schema

# Wrap an integer schema to allow None
schema = core_schema.nullable_schema(core_schema.int_schema())
v = SchemaValidator(schema)

assert v.validate_python(None) is None
assert v.validate_python(1) == 1
assert v.validate_python('123') == 123

The strict parameter in nullable_schema allows you to enforce strict validation on the inner schema even if the global configuration is lax. This design ensures that the "nullability" of a field is decoupled from the validation logic of the value itself.

Distinguishing Missing vs. Null with 'Some'

While NullableSchema handles validation, the project needs a way to communicate the state of a value when it might be missing, null, or present. This is where the Some wrapper comes in.

The Some class (found in pydantic_core) is a generic container used primarily by SchemaValidator.get_default_value() and TypeAdapter.get_default_value(). It functions similarly to the Option::Some variant in Rust, wrapping a value to indicate its presence.

The Tri-state Problem

When querying for a default value, there are three possible states:

1. No default: The field is required.
2. Default is None: The field is optional and defaults to null.
3. Default is a value: The field is optional and has a specific default (e.g., 42).

If get_default_value() simply returned None for both "no default" and "default is None", the caller could not distinguish between them. The Some wrapper resolves this:

• Returns None: No default value exists.
• Returns Some(None): The default value is explicitly None.
• Returns Some(value): The default value is value.

Example: Handling Defaults

The following test case from pydantic-core/tests/validators/test_with_default.py illustrates this pattern:

from pydantic_core import SchemaValidator, core_schema, Some

def test_some() -> None:
    # Define a schema with an explicit default of 42
    s = core_schema.with_default_schema(core_schema.int_schema(), default=42)
    res = SchemaValidator(s).get_default_value()

    assert res is not None
    assert isinstance(res, Some)
    assert res.value == 42
    assert repr(res) == 'Some(42)'

Pattern Matching with 'Some'

To make handling these wrapped values more ergonomic, the Some class implements __match_args__, enabling Python's structural pattern matching (available in Python 3.10+). This allows developers to write clean, declarative logic for handling the different states of a value.

As seen in the project's test suite:

def handle_default(v: Some[Any] | None) -> str:
    match v:
        case Some(None):
            return 'Default is explicitly None'
        case Some(value):
            return f'Default value is: {value}'
        case None:
            return 'No default defined'

Design Tradeoffs and Constraints

The decision to use a wrapper like Some instead of a sentinel object (like PydanticUndefined) for return values reflects a preference for explicit container types over shared global constants.

1. Type Safety: Using Some[T] | None allows type checkers to enforce that the developer handles the "missing" case before accessing the underlying value.
2. Nesting: Some can wrap any value, including other sentinels, without collision.
3. Performance: While wrapping values in a Python object incurs a small overhead, it is only used in specific metadata-retrieval methods like get_default_value(), not in the hot path of the main validation loop (which is implemented in Rust).

By combining the NullableSchema for validation logic and the Some wrapper for value representation, the project provides a robust framework for handling the complexities of nullability in a type-safe and unambiguous manner.