waldur-site-agent

Plugin Architecture

The Waldur Site Agent uses a pluggable backend system that allows external developers to create custom backend plugins without modifying the core codebase.

Core architecture & plugin system

---
config:
  layout: elk
---
graph TB
    subgraph "Core Package"
        WA[waldur-site-agent<br/>Core Logic & Processing]
        BB[BaseBackend<br/>Abstract Interface]
        BC[BaseClient<br/>Abstract Interface]
        CU[Common Utils<br/>Entry Point Discovery]
    end

    subgraph "Plugin Ecosystem"
        PLUGINS[Backend Plugins<br/>SLURM, MOAB, MUP, etc.]
        UMANAGE[Username Management<br/>Plugins]
    end

    subgraph "Entry Point System"
        EP_BACKENDS[waldur_site_agent.backends]
        EP_USERNAME[waldur_site_agent.username_management_backends]
    end

    %% Core dependencies
    WA --> BB
    WA --> BC
    WA --> CU

    %% Plugin registration and discovery
    CU --> EP_BACKENDS
    CU --> EP_USERNAME
    EP_BACKENDS -.-> PLUGINS
    EP_USERNAME -.-> UMANAGE

    %% Plugin inheritance
    PLUGINS -.-> BB
    PLUGINS -.-> BC
    UMANAGE -.-> BB

    %% Styling - Dark mode compatible colors
    classDef corePackage fill:#1E3A8A,stroke:#3B82F6,stroke-width:2px,color:#FFFFFF
    classDef plugin fill:#581C87,stroke:#8B5CF6,stroke-width:2px,color:#FFFFFF
    classDef entrypoint fill:#065F46,stroke:#10B981,stroke-width:2px,color:#FFFFFF

    class WA,BB,BC,CU corePackage
    class PLUGINS,UMANAGE plugin
    class EP_BACKENDS,EP_USERNAME entrypoint

Agent modes & external systems

---
config:
  layout: elk
---
graph TB
    subgraph "Agent Modes"
        ORDER[agent-order-process<br/>Order Processing]
        REPORT[agent-report<br/>Usage Reporting]
        SYNC[agent-membership-sync<br/>Membership Sync]
        EVENT[agent-event-process<br/>Event Processing]
    end

    subgraph "Plugin Layer"
        PLUGINS[Backend Plugins<br/>SLURM, MOAB, MUP, etc.]
    end

    subgraph "External Systems"
        WALDUR[Waldur Mastermind<br/>REST API]
        BACKENDS[Cluster Backends<br/>CLI/API Systems]
        STOMP[STOMP Broker<br/>Event Processing]
    end

    %% Agent mode usage of plugins
    ORDER --> PLUGINS
    REPORT --> PLUGINS
    SYNC --> PLUGINS
    EVENT --> PLUGINS

    %% External connections
    ORDER <--> WALDUR
    REPORT <--> WALDUR
    SYNC <--> WALDUR
    EVENT <--> WALDUR
    EVENT <--> STOMP
    PLUGINS <--> BACKENDS

    %% Styling - Dark mode compatible colors
    classDef agent fill:#B45309,stroke:#F59E0B,stroke-width:2px,color:#FFFFFF
    classDef plugin fill:#581C87,stroke:#8B5CF6,stroke-width:2px,color:#FFFFFF
    classDef external fill:#C2410C,stroke:#F97316,stroke-width:2px,color:#FFFFFF

    class ORDER,REPORT,SYNC,EVENT agent
    class PLUGINS plugin
    class WALDUR,BACKENDS,STOMP external

Event processing architecture

The event_process mode uses WebSocket STOMP connections to receive real-time events from Waldur Mastermind via RabbitMQ. The main loop combines event-driven processing with periodic reconciliation to ensure data consistency even when STOMP messages are missed.

Event processing flow

---
config:
  layout: elk
---
graph TB
    subgraph "Startup"
        INIT[Run Initial<br/>Offering Processing]
        REG[Register Agent Identity<br/>& Event Subscriptions]
        STOMP_CONN[Connect WebSocket STOMP<br/>per Object Type]
    end

    subgraph "Main Loop (1-min tick)"
        TICK[Wake Up]
        HC_CHECK{Health Check<br/>interval elapsed?<br/>default: 30 min}
        HC[Send Health Checks<br/>for All Offerings]
        RC_CHECK{Reconciliation<br/>interval elapsed?<br/>default: 60 min}
        RC[Run Username<br/>Reconciliation]
        SLEEP[Sleep 60s]
    end

    subgraph "STOMP Event Handlers (daemon threads)"
        ORDER_H[Order Handler<br/>process orders]
        MEMBER_H[Membership Handler<br/>sync roles & users]
        OU_H[OfferingUser Handler<br/>sync usernames]
        IMPORT_H[Resource Import<br/>Handler]
        LIMITS_H[Periodic Limits<br/>Handler]
    end

    subgraph "External Systems"
        WALDUR[Waldur Mastermind<br/>REST API]
        RMQ[RabbitMQ<br/>WebSocket STOMP]
        BACKEND[Backend System<br/>SLURM / Waldur B / etc.]
    end

    %% Startup flow
    INIT --> REG --> STOMP_CONN

    %% Main loop
    STOMP_CONN --> TICK
    TICK --> HC_CHECK
    HC_CHECK -->|Yes| HC --> RC_CHECK
    HC_CHECK -->|No| RC_CHECK
    RC_CHECK -->|Yes| RC --> SLEEP
    RC_CHECK -->|No| SLEEP
    SLEEP --> TICK

    %% STOMP event handlers
    RMQ -->|events| ORDER_H
    RMQ -->|events| MEMBER_H
    RMQ -->|events| OU_H
    RMQ -->|events| IMPORT_H
    RMQ -->|events| LIMITS_H

    %% External connections
    ORDER_H --> WALDUR
    MEMBER_H --> WALDUR
    OU_H --> WALDUR
    HC --> WALDUR
    RC --> WALDUR
    RC --> BACKEND
    ORDER_H --> BACKEND
    MEMBER_H --> BACKEND
    STOMP_CONN --> RMQ

    %% Styling - Dark mode compatible colors
    classDef startup fill:#1E3A8A,stroke:#3B82F6,stroke-width:2px,color:#FFFFFF
    classDef loop fill:#B45309,stroke:#F59E0B,stroke-width:2px,color:#FFFFFF
    classDef handler fill:#581C87,stroke:#8B5CF6,stroke-width:2px,color:#FFFFFF
    classDef external fill:#C2410C,stroke:#F97316,stroke-width:2px,color:#FFFFFF
    classDef decision fill:#065F46,stroke:#10B981,stroke-width:2px,color:#FFFFFF

    class INIT,REG,STOMP_CONN startup
    class TICK,HC,RC,SLEEP loop
    class ORDER_H,MEMBER_H,OU_H,IMPORT_H,LIMITS_H handler
    class WALDUR,RMQ,BACKEND external
    class HC_CHECK,RC_CHECK decision

Periodic reconciliation

Event-driven processing can miss updates due to transient STOMP disconnections or message loss. The main loop includes a periodic reconciliation timer (default: 60 minutes, configurable via WALDUR_SITE_AGENT_RECONCILIATION_PERIOD_MINUTES environment variable) that runs sync_offering_user_usernames() for all STOMP-enabled offerings with a membership sync backend.

This reconciliation is lightweight — it only syncs usernames, not a full membership sync — and is idempotent, so running it has no side effects when data is already consistent.

STOMP subscription types

Each offering can subscribe to multiple object types depending on configuration:

Key plugin features

Plugin structure

Built-in plugin structure

plugins/{backend_name}/
├── pyproject.toml              # Entry point registration
├── waldur_site_agent_{name}/   # Plugin implementation
│   ├── backend.py             # Backend class inheriting BaseBackend
│   ├── client.py              # Client for external system communication
│   └── parser.py              # Data parsing utilities (optional)
└── tests/                     # Plugin-specific tests

Available plugins

SLURM plugin (waldur-site-agent-slurm)

MOAB plugin (waldur-site-agent-moab)

MUP plugin (waldur-site-agent-mup)

Waldur federation plugin (waldur-site-agent-waldur)

Basic username management (waldur-site-agent-basic-username-management)

Rancher plugin (waldur-site-agent-rancher)

Rancher CRD-driven plugin (waldur-site-agent-rancher-kc-crd)

Creating custom plugins

For comprehensive plugin development instructions, including:

See Plugin Development Guide.

A ready-to-use plugin template is available at docs/plugin-template/.

Plugin discovery mechanism

The core system automatically discovers plugins through Python entry points:

from importlib.metadata import entry_points

BACKENDS = {
    entry_point.name: entry_point.load()
    for entry_point in entry_points(group="waldur_site_agent.backends")
}

This enables:

Configuration integration

Plugins integrate through offering configuration:

offerings:
  - name: "Example Offering"
    backend_type: "slurm"                    # Legacy setting
    order_processing_backend: "slurm"        # Order processing via SLURM
    reporting_backend: "custom-api"          # Custom reporting backend
    membership_sync_backend: "slurm"         # Membership sync via SLURM
    username_management_backend: "custom"    # Custom username generation

This allows:

This site is open source. Improve this page.