zuko.flows.neural#

Neural flows and transformations.

Classes#

`NeuralAutoregressiveTransform`	Creates a lazy neural autoregressive transformation.
`NAF`	Creates a neural autoregressive flow (NAF).
`UnconstrainedNeuralAutoregressiveTransform`	Creates a lazy unconstrained neural autoregressive transformation.
`UNAF`	Creates an unconstrained neural autoregressive flow (UNAF).

Descriptions#

class zuko.flows.neural.NeuralAutoregressiveTransform(features, context=0, signal=8, network={}, **kwargs)#

Creates a lazy neural autoregressive transformation.

The monotonic neural network is parametrized by its internal positive weights, which are independent of the features and context. To modulate its behavior, it receives as input a signal that is autoregressively dependent on the features and context.

See also

zuko.transforms.MonotonicTransform

References

Neural Autoregressive Flows (Huang et al., 2018)

https://arxiv.org/abs/1804.00779

Parameters:

features (int) – The number of features.
context (int) – The number of context features.
signal (int) – The number of signal features of the monotonic network.
network (Dict[str, Any]) – Keyword arguments passed to zuko.nn.MonotonicMLP.
kwargs – Keyword arguments passed to MaskedAutoregressiveTransform.

Example

>>> t = NeuralAutoregressiveTransform(3, 4)
>>> t
NeuralAutoregressiveTransform(
  (base): MonotonicTransform()
  (order): [0, 1, 2]
  (hyper): MaskedMLP(
    (0): MaskedLinear(in_features=7, out_features=64, bias=True)
    (1): ReLU()
    (2): MaskedLinear(in_features=64, out_features=64, bias=True)
    (3): ReLU()
    (4): MaskedLinear(in_features=64, out_features=24, bias=True)
  )
  (network): MonotonicMLP(
    (0): MonotonicLinear(in_features=9, out_features=64, bias=True)
    (1): TwoWayELU(alpha=1.0)
    (2): MonotonicLinear(in_features=64, out_features=64, bias=True)
    (3): TwoWayELU(alpha=1.0)
    (4): MonotonicLinear(in_features=64, out_features=1, bias=True)
  )
)
>>> x = torch.randn(3)
>>> x
tensor([-2.3267,  1.4581, -1.6776])
>>> c = torch.randn(4)
>>> y = t(c)(x)
>>> t(c).inv(y)
tensor([-2.3267,  1.4581, -1.6776])

class zuko.flows.neural.NAF(features, context=0, transforms=3, randperm=False, **kwargs)#

Creates a neural autoregressive flow (NAF).

Warning

Invertibility is only guaranteed for features within the interval \([-10, 10]\). It is recommended to standardize features (zero mean, unit variance) before training.

References

Neural Autoregressive Flows (Huang et al., 2018)

https://arxiv.org/abs/1804.00779

Parameters:

features (int) – The number of features.
context (int) – The number of context features.
transforms (int) – The number of autoregressive transformations.
randperm (bool) – Whether features are randomly permuted between transformations or not. If False, features are in ascending (descending) order for even (odd) transformations.
unconstrained – Whether to use unconstrained or regular monotonic networks.
kwargs – Keyword arguments passed to NeuralAutoregressiveTransform.

class zuko.flows.neural.UnconstrainedNeuralAutoregressiveTransform(features, context=0, signal=8, network={}, **kwargs)#

Creates a lazy unconstrained neural autoregressive transformation.

The integrand neural network is parametrized by its internal weights, which are independent of the features and context. To modulate its behavior, it receives as input a signal that is autoregressively dependent on the features and context. The integration constant has the same dependencies as the signal.

References

Unconstrained Monotonic Neural Networks (Wehenkel et al., 2019)

https://arxiv.org/abs/1908.05164

Parameters:

features (int) – The number of features.
context (int) – The number of context features.
signal (int) – The number of signal features of the integrand network.
network (Dict[str, Any]) – Keyword arguments passed to zuko.nn.MLP.
kwargs – Keyword arguments passed to MaskedAutoregressiveTransform.

Example

>>> t = UnconstrainedNeuralAutoregressiveTransform(3, 4)
>>> t
UnconstrainedNeuralAutoregressiveTransform(
  (base): UnconstrainedMonotonicTransform()
  (order): [0, 1, 2]
  (hyper): MaskedMLP(
    (0): MaskedLinear(in_features=7, out_features=64, bias=True)
    (1): ReLU()
    (2): MaskedLinear(in_features=64, out_features=64, bias=True)
    (3): ReLU()
    (4): MaskedLinear(in_features=64, out_features=27, bias=True)
  )
  (integrand): MLP(
    (0): Linear(in_features=9, out_features=64, bias=True)
    (1): ELU(alpha=1.0)
    (2): Linear(in_features=64, out_features=64, bias=True)
    (3): ELU(alpha=1.0)
    (4): Linear(in_features=64, out_features=1, bias=True)
    (5): Softplus(beta=1, threshold=20)
  )
)
>>> x = torch.randn(3)
>>> x
tensor([-0.0103, -1.0871, -0.0667])
>>> c = torch.randn(4)
>>> y = t(c)(x)
>>> t(c).inv(y)
tensor([-0.0103, -1.0871, -0.0667])

class zuko.flows.neural.UNAF(features, context=0, transforms=3, randperm=False, **kwargs)#

Creates an unconstrained neural autoregressive flow (UNAF).

Warning

Invertibility is only guaranteed for features within the interval \([-10, 10]\). It is recommended to standardize features (zero mean, unit variance) before training.

References

Unconstrained Monotonic Neural Networks (Wehenkel et al., 2019)

https://arxiv.org/abs/1908.05164

Parameters:

features (int) – The number of features.
context (int) – The number of context features.
transforms (int) – The number of autoregressive transformations.
randperm (bool) – Whether features are randomly permuted between transformations or not. If False, features are in ascending (descending) order for even (odd) transformations.
kwargs – Keyword arguments passed to UnconstrainedNeuralAutoregressiveTransform.