ORMatic Overview

ORMatic Overview#

ORMatic is a subpackage of KRROOD designed to integrate Python dataclasses with the SQLAlchemy ORM. It operates by generating a SQLAlchemy declarative model layer, offering utilities for translating between domain objects (dataclasses) and database representations (rows).

This approach promotes a separation between the domain model and the persistence mechanism, allowing developers to concentrate on business logic while abstracting underlying database implementation and SQLAlchemy specifics.

System Overview#

The following diagram illustrates the interaction between the domain layer, the persistence layer, and the database:

        flowchart TB
    %% Title and Description
    accTitle: ORMatic High-Level System Overview
    accDescr: High-level architecture showing Domain Layer, Persistence Layer, and RDBMS

    %% Subgraphs (Clusters)
    subgraph Domain ["Domain Layer"]
        direction LR
        definitions["Domain Definitions (Python Dataclasses)"]
        data["Domain Objects (Python Objects)"]
    end

    subgraph Persistence ["Persistence Layer (Generated)"]
        direction LR
        interface["Data Access Object Definitions"]
        instances["Data Access Objects"]
    end

    %% Database Node
    db[("RDBMS (Supported by SQLAlchemy)")]

    %% Relationships & Layout Constraints
    definitions -->|"generate_orm()"| interface
    data -- "to_dao()" -->instances
    instances <-->|"SQLAlchemy Session"| db


    %% Internal Structure Hints
    definitions -.->|"Defines structure"| data
    interface -.->|"Defines structure"| instances
    instances -- "from_dao()" --> data

Fig. 1 ORMatic High-Level System Overview#

  • Domain Layer: This is where business logic and data structures are defined using standard Python dataclasses (Domain Definitions). Instances of these dataclasses are referred to as Domain Objects.

  • Persistence Layer: ORMatic automatically generates a corresponding set of Data Access Objects (DAOs) based on the domain definitions. These DAOs are SQLAlchemy declarative models that handle the mapping to the database.

  • Mapping & Conversion: - generate_orm(): Analyzes domain definitions to automatically create the DAO structure. - to_dao(): Converts domain objects into their DAO counterparts for persistence. - from_dao(): Reconstructs domain objects from retrieved DAO instances.

  • Database Interaction: The persistence layer communicates with the RDBMS via SQLAlchemy sessions, providing a clean abstraction over the physical storage.

I recommend to structure your Python package to work cleanly with ORMatic as follows:

domain_model/
├── scripts/
│   └── generate_orm.py               # Script to configure ORMatic
├── src/
│   └── domain_model/
│       └── orm/
│           ├── __init__.py           # Package initializer
│           ├── model.py              # Custom types and alternative mappings
│           └── ormatic_interface.py  # Generated target file for ORMatic
└── tests/
    └── conftest.py                   # Pytest config; automatically runs generate_orm.py

You can make your conftest automatically refresh the ormatic_interface.py by including something like this in your conftest.py:

def pytest_configure(config):
    # Ensure ORM classes are generated before tests run
    repository_root = Path(__file__).resolve().parents[1]
    generate_orm_path = (
        repository_root / "scripts" / "generate_orm.py"
    )
    # Execute the ORM generation script as a standalone module
    runpy.run_path(str(generate_orm_path), run_name="__main__")

import domain_model.orm.ormatic_interface

Core Assumptions and Modeling Rules#

To ensure unambiguous translation and an ergonomic generated ORM, ORMatic adheres to the following rules for dataclass modeling:

  • Protected Fields: Any dataclass field name beginning with _ is ignored for persistence. These protected fields are suitable for transient or derived runtime data.

  • Optional Types: The only guaranteed supported union is Optional[T]. Other unions are modeled using a shared superclass if there exists a common base that is not object.

  • Collections: Iterables must be non-optional and non-nested. For an “optional list,” use a default factory that returns an empty collection.

  • Inheritance and Polymorphism: Inheritance is supported and generates joined-table inheritance. Note that only the first base class in a multiple inheritance structure is considered for queries using abstract classes.

  • Type Discipline: Dataclass fields require concrete, non-ambiguous type annotations. Prefer small dataclasses or value objects over primitive types when modeling relationships.

When dataclasses cannot conform to these patterns, the following alternatives are available:

  • Alternative Mapping: Explicit mapping classes can be provided to control how a dataclass is persisted.

  • Type Decorator: Custom type decorators can be supplied for specialized value types.

Detailed information on these options is provided in the Dealing with unmappable dataclasses section.

What ORMatic Generates#

Execution of ORMatic on a set of classes produces a module containing:

  • Base: An SQLAlchemy DeclarativeBase that manages metadata and a type_mappings registry for custom types.

  • *DAO Classes: One class per dataclass (and per explicit mapping). Each is a SQLAlchemy declarative model that includes: - Columns for fields with built-in or custom types - Foreign keys and relationships inferred from nested dataclasses and collections - __mapper_args__ for inheritance and polymorphic configuration

ORMatic analyzes the dataclass structure to identify scalar fields, one-to-one, one-to-many, and many-to-many associations. This information is then used to generate a full, decoupled SQLAlchemy declarative layer via a Jinja template.

ORMatic Compatible Class Pattern#

If you want to generally know how to write classes that work out of the box with ORMatic, follow these simple rules:

  1. Everything is a dataclass

  2. Never write you own __init__.

  3. Apply side effects (e. g. setting back references) in the __post_init__

  4. Use protected fields, those whose name begins with _ only for computational state that doesnt need persistence (e. g. caching, indexing, …)

  5. Don’t use nested containers as public fields

  6. Don’t use dicts as public fields

  7. Never use optional containers, use empty containers by default.

Persisting Objects#

  • krrood.ormatic.dao.to_dao(): Converts a dataclass instance into its corresponding DAO object, including recursive conversion of nested elements.

  • krrood.ormatic.dao.DataAccessObject.from_dao(): Converts a loaded DAO instance back into the original dataclass, including nested components and collections.