Generators and strategies

Generators own trainable latent vectors and use mappers to produce complete ParameterTree instances. Strategies are small declarative builders that select a generator arrangement.

Layer-wise generation

LayerwiseGenerator is the default architecture. Parameters are grouped by the module that owns them, so a layer’s weight and bias share one latent mapper:

from torch import nn

from mapping_networks import LayerwiseGenerator, ParameterSpec

target = nn.Sequential(nn.Linear(4, 8), nn.ReLU(), nn.Linear(8, 2))
spec = ParameterSpec.from_module(target)
generator = LayerwiseGenerator(spec, latent_dim=16)
parameters = generator.generate_parameters()

assert tuple(name for name, _ in generator.named_latent_vectors()) == ("0", "2")
spec.validate_tree(parameters)

Only LayerGenerator.latent tensors are optimizer-visible parameters. Default MLP mapper weights remain buffers. Each group is reconstructed independently, reducing peak fixed-projection size relative to one giant mapper.

Single-vector generation

SingleVectorGenerator implements the paper’s SLVT baseline. One latent and one projection generate all target parameters:

from mapping_networks import SingleVectorGenerator

generator = SingleVectorGenerator(spec, latent_dim=16)
parameters = generator()

The projection contains approximately latent_dim * target_parameter_count elements. Construction raises MemoryError when that exceeds max_projection_elements (100 million by default). Passing allow_large=True is an explicit acknowledgement, not a memory optimization.

Explicit groups

GroupedGenerator accepts a complete, non-overlapping mapping of group names to parameter names:

from mapping_networks import GroupedGenerator

groups = {
    "input": ("0.weight", "0.bias"),
    "output": ("2.weight", "2.bias"),
}
generator = GroupedGenerator(
    spec,
    groups,
    latent_dim={"input": 12, "output": 8},
)

Every spec name must occur exactly once. A single integer applies one latent dimension to all groups; a mapping assigns dimensions individually.

Strategies and extension

LayerwiseStrategy, SLVTStrategy, and GroupedStrategy construct their corresponding generators through a shared GenerationStrategy.build() contract. ParameterGenerator subclasses implement generate_parameters() and named_latent_vectors().

Custom mapper creation is supported with mapper_factory(latent_dim, output_dim). The returned mapper dimensions must match the group contract. Custom generator outputs are always validated against their target ParameterSpec before stateless execution.

Lazy generation and shared block modulation for pretrained fine-tuning remain scaling extensions; the current grouped API establishes their ownership and naming boundary without pretending that an ordinary monolithic target forward can consume weights lazily.