mapping_networks documentation

mapping_networks is a PyTorch package for training target models through low-dimensional, generated parameter spaces. It is inspired by the paper Mapping Networks and is being built as a model-agnostic, stateless toolkit rather than a reproduction tied to one experiment.

Current Status

The package is under active development. The following layers are fully implemented:

  • Parameter runtime: Named parameter metadata and storage (ParameterSpec and ParameterTree).

  • Stateless target execution: Frozen target wrapper and stateless execution using torch.func.functional_call.

  • Mappers & modulation: Orthogonal and residual MLP mappers, plus additive, affine, and low-rank weight modulation.

  • Generators & strategies: Single-latent vector (SLVT), layerwise, and custom grouped strategy/generator mappings.

  • MappingModel: High-level MappingModel interface that orchestrates stateless generation and execution.

  • Loss system: Task loss plus stability, Jacobian smoothness, and cosine alignment regularizations.

  • Configuration & registries: Pydantic configurations and registries for custom components.

  • Trainer & callbacks: Training loops, customizable batch adapters, and lifecycle callback observers.

  • Checkpointing: Versioned compact checkpoint save/load with architecture validation and training resumption.

  • Scaling & distributed: DDP wrapping, lazy generation, memory benchmarking, and torch.compile compatibility.

See progress.md for the complete specification and implementation checklist.

Getting Started

User Guides

API Reference