Noise layers¶
-
class
lasagne.layers.
DropoutLayer
(incoming, p=0.5, rescale=True, shared_axes=(), **kwargs)[source]¶ Dropout layer
Sets values to zero with probability p. See notes for disabling dropout during testing.
Parameters: incoming : a
Layer
instance or a tuplethe layer feeding into this layer, or the expected input shape
p : float or scalar tensor
The probability of setting a value to zero
rescale : bool
If
True
(the default), scale the input by1 / (1 - p)
when dropout is enabled, to keep the expected output mean the same.shared_axes : tuple of int
Axes to share the dropout mask over. By default, each value can be dropped individually.
shared_axes=(0,)
uses the same mask across the batch.shared_axes=(2, 3)
uses the same mask across the spatial dimensions of 2D feature maps.See also
dropout_channels
- Drops full channels of feature maps
spatial_dropout
- Alias for
dropout_channels()
dropout_locations
- Drops full pixels or voxels of feature maps
Notes
The dropout layer is a regularizer that randomly sets input values to zero; see [R40], [R41] for why this might improve generalization.
The behaviour of the layer depends on the
deterministic
keyword argument passed tolasagne.layers.get_output()
. IfTrue
, the layer behaves deterministically, and passes on the input unchanged. IfFalse
or not specified, dropout (and possibly scaling) is enabled. Usually, you would usedeterministic=False
at train time anddeterministic=True
at test time.References
[R40] (1, 2) Hinton, G., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. R. (2012): Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580. [R41] (1, 2) Srivastava Nitish, Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2014): Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research, 5(Jun)(2), 1929-1958. -
get_output_for
(input, deterministic=False, **kwargs)[source]¶ Propagates the given input through this layer (and only this layer).
Parameters: input : Theano expression
The expression to propagate through this layer.
Returns: output : Theano expression
The output of this layer given the input to this layer.
Notes
This is called by the base
lasagne.layers.get_output()
to propagate data through a network.This method should be overridden when implementing a new
Layer
class. By default it raises NotImplementedError.
-
lasagne.layers.
dropout_channels
(incoming, *args, **kwargs)[source]¶ Convenience function to drop full channels of feature maps.
Adds a
DropoutLayer
that sets feature map channels to zero, across all locations, with probability p. For convolutional neural networks, this may give better results than independent dropout [R42].Parameters: incoming : a
Layer
instance or a tuplethe layer feeding into this layer, or the expected input shape
*args, **kwargs
Any additional arguments and keyword arguments are passed on to the
DropoutLayer
constructor, except for shared_axes.Returns: layer :
DropoutLayer
instanceThe dropout layer with shared_axes set to drop channels.
References
[R42] (1, 2) J. Tompson, R. Goroshin, A. Jain, Y. LeCun, C. Bregler (2014): Efficient Object Localization Using Convolutional Networks. https://arxiv.org/abs/1411.4280
-
lasagne.layers.
spatial_dropout
(incoming, *args, **kwargs)[source]¶ alias of
dropout_channels()
-
lasagne.layers.
dropout_locations
(incoming, *args, **kwargs)[source]¶ Convenience function to drop full locations of feature maps.
Adds a
DropoutLayer
that sets feature map locations (i.e., pixels or voxels) to zero, across all channels, with probability p.Parameters: incoming : a
Layer
instance or a tuplethe layer feeding into this layer, or the expected input shape
*args, **kwargs
Any additional arguments and keyword arguments are passed on to the
DropoutLayer
constructor, except for shared_axes.Returns: layer :
DropoutLayer
instanceThe dropout layer with shared_axes set to drop locations.
-
class
lasagne.layers.
GaussianNoiseLayer
(incoming, sigma=0.1, **kwargs)[source]¶ Gaussian noise layer.
Add zero-mean Gaussian noise of given standard deviation to the input [R43].
Parameters: incoming : a
Layer
instance or a tuplethe layer feeding into this layer, or the expected input shape
sigma : float or tensor scalar
Standard deviation of added Gaussian noise
Notes
The Gaussian noise layer is a regularizer. During training you should set deterministic to false and during testing you should set deterministic to true.
References
[R43] (1, 2) K.-C. Jim, C. Giles, and B. Horne (1996): An analysis of noise in recurrent neural networks: convergence and generalization. IEEE Transactions on Neural Networks, 7(6):1424-1438.