[source]

Layer

keras.layers.core.Layer()

Abstract base layer class.

All Keras layers accept certain keyword arguments:

  • trainable: boolean. Set to "False" before model compilation to freeze layer weights (they won't be updated further during training).
  • input_shape: a tuple of integers specifying the expected shape of the input samples. Does not includes the batch size. (e.g. (100,) for 100-dimensional inputs).
  • batch_input_shape: a tuple of integers specifying the expected shape of a batch of input samples. Includes the batch size (e.g. (32, 100) for a batch of 32 100-dimensional inputs).

Methods

clear_previous(reset_weights=True)

Unlink a layer from its parent in the computational graph.

This is only allowed if the layer has an input attribute.

count_params()

Return the total number of floats (or ints) composing the weights of the layer.

get_config()

Return the parameters of the layer, as a dictionary.

get_input(train=False)

get_output(train=False)

get_output_mask(train=None)

For some models (such as RNNs) you want a way of being able to mark some output data-points as "masked", so they are not used in future calculations. In such a model, get_output_mask() should return a mask of one less dimension than get_output() (so if get_output is (nb_samples, nb_timesteps, nb_dimensions), then the mask is (nb_samples, nb_timesteps), with a one for every unmasked datapoint, and a zero for every masked one.

If there is no masking then it shall return None. For instance if you attach an Activation layer (they support masking) to a layer with an output_mask, then that Activation shall also have an output_mask. If you attach it to a layer with no such mask, then the Activation's get_output_mask shall return None.

Some layers have an output_mask even if their input is unmasked, notably Embedding which can turn the entry "0" into a mask.

get_weights()

Return the weights of the layer, as a list of numpy arrays.

set_input_shape(input_shape)

set_previous(layer, reset_weights=True)

Connect a layer to its parent in the computational graph.

set_weights(weights)

Set the weights of the layer.

  • weights: a list of numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).
supports_masked_input()

Whether or not this layer respects the output mask of its previous layer in its calculations. If you try to attach a layer that does not support masked_input to a layer that gives a non-None output_mask(), an error will be raised.

[source]

Masking

keras.layers.core.Masking(mask_value=0.0)

Mask an input sequence by using a mask value to identify padding.

This layer copies the input to the output layer with identified padding replaced with 0s and creates an output mask in the process.

At each timestep, if the values all equal mask_value, then the corresponding mask value for the timestep is 0 (skipped), otherwise it is 1.

[source]

Merge

keras.layers.core.Merge(layers, mode='sum', concat_axis=-1, dot_axes=-1)

Merge the output of a list of layers or containers into a single tensor.

Arguments

  • mode: one of {sum, mul, concat, ave, join, cos, dot}.
    • sum: sum the outputs (shapes must match)
    • mul: multiply the outputs element-wise (shapes must match)
    • concat: concatenate the outputs along the axis specified by concat_axis
    • ave: average the outputs (shapes must match)
    • join: places the outputs in an OrderedDict (inputs must be named)
  • concat_axis: axis to use in concat mode.
  • dot_axes: axis or axes to use in dot mode (see the Numpy documentation for more details).

Examples

left = Sequential()
left.add(Dense(50, input_shape=(784,)))
left.add(Activation('relu'))

right = Sequential()
right.add(Dense(50, input_shape=(784,)))
right.add(Activation('relu'))

model = Sequential()
model.add(Merge([left, right], mode='sum'))

model.add(Dense(10))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy', optimizer='rmsprop')

model.fit([X_train, X_train], Y_train, batch_size=128, nb_epoch=20,
      validation_data=([X_test, X_test], Y_test))

[source]

TimeDistributedMerge

keras.layers.core.TimeDistributedMerge(mode='sum')

Sum/multiply/average over the outputs of a TimeDistributed layer.

Input shape

3D tensor with shape: (samples, steps, features).

Output shape

2D tensor with shape: (samples, features).

Arguments

  • mode: one of {'sum', 'mul', 'ave'}

[source]

Dropout

keras.layers.core.Dropout(p)

Apply Dropout to the input. Dropout consists in randomly setting a fraction p of input units to 0 at each update during training time, which helps prevent overfitting.

Arguments

  • p: float between 0 and 1. Fraction of the input units to drop.

References

[source]

Activation

keras.layers.core.Activation(activation)

Apply an activation function to an output.

Input shape

Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.

Output shape

Same shape as input.

  • _Arguments_:

  • activation: name of activation function to use

    • (see: activations), or alternatively, a Theano or TensorFlow operation.

[source]

Reshape

keras.layers.core.Reshape(dims)

Reshape an output to a certain shape.

Input shape

Arbitrary, although all dimensions in the input shaped must be fixed. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.

Output shape

(batch_size,) + dims

Arguments

  • dims: target shape. Tuple of integers, does not include the samples dimension (batch size).

[source]

Permute

keras.layers.core.Permute(dims)

Permute the dimensions of the input according to a given pattern.

Useful for e.g. connecting RNNs and convnets together.

Input shape

Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.

Output shape

Same as the input shape, but with the dimensions re-ordered according to the specified pattern.

Arguments

  • dims: Tuple of integers. Permutation pattern, does not include the samples dimension. Indexing starts at 1. For instance, (2, 1) permutes the first and second dimension of the input.

[source]

Flatten

keras.layers.core.Flatten()

Flatten the input. Does not affect the batch size.

Input shape

Arbitrary, although all dimensions in the input shape must be fixed. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.

Output shape

(batch_size,)

[source]

RepeatVector

keras.layers.core.RepeatVector(n)

Repeat the input n times.

Input shape

2D tensor of shape (nb_samples, features).

Output shape

3D tensor of shape (nb_samples, n, features).

Arguments

  • n: integer, repetition factor.

[source]

Dense

keras.layers.core.Dense(output_dim, init='glorot_uniform', activation='linear', weights=None, W_regularizer=None, b_regularizer=None, activity_regularizer=None, W_constraint=None, b_constraint=None, input_dim=None)

Just your regular fully connected NN layer.

Input shape

2D tensor with shape: (nb_samples, input_dim).

Output shape

2D tensor with shape: (nb_samples, output_dim).

Arguments

  • output_dim: int > 0.
  • init: name of initialization function for the weights of the layer (see initializations), or alternatively, Theano function to use for weights initialization. This parameter is only relevant if you don't pass a weights argument.
  • activation: name of activation function to use (see activations), or alternatively, elementwise Theano function. If you don't specify anything, no activation is applied (ie. "linear" activation: a(x) = x).
  • weights: list of numpy arrays to set as initial weights. The list should have 2 elements, of shape (input_dim, output_dim) and (output_dim,) for weights and biases respectively.
  • W_regularizer: instance of WeightRegularizer (eg. L1 or L2 regularization), applied to the main weights matrix.
  • b_regularizer: instance of WeightRegularizer, applied to the bias.
  • activity_regularizer: instance of ActivityRegularizer, applied to the network output.
  • W_constraint: instance of the constraints module (eg. maxnorm, nonneg), applied to the main weights matrix.
  • b_constraint: instance of the constraints module, applied to the bias.
  • input_dim: dimensionality of the input (integer). This argument (or alternatively, the keyword argument input_shape) is required when using this layer as the first layer in a model.

[source]

TimeDistributedDense

keras.layers.core.TimeDistributedDense(output_dim, init='glorot_uniform', activation='linear', weights=None, W_regularizer=None, b_regularizer=None, activity_regularizer=None, W_constraint=None, b_constraint=None, input_dim=None, input_length=None)

Apply a same Dense layer for each dimension[1] (time_dimension) input. Especially useful after a recurrent network with 'return_sequence=True'.

Input shape

3D tensor with shape (nb_sample, time_dimension, input_dim).

Output shape

3D tensor with shape (nb_sample, time_dimension, output_dim).

Arguments

  • output_dim: int > 0.
  • init: name of initialization function for the weights of the layer (see initializations), or alternatively, Theano function to use for weights initialization. This parameter is only relevant if you don't pass a weights argument.
  • activation: name of activation function to use (see activations), or alternatively, elementwise Theano function. If you don't specify anything, no activation is applied (ie. "linear" activation: a(x) = x).
  • weights: list of numpy arrays to set as initial weights. The list should have 2 elements, of shape (input_dim, output_dim) and (output_dim,) for weights and biases respectively.
  • W_regularizer: instance of WeightRegularizer (eg. L1 or L2 regularization), applied to the main weights matrix.
  • b_regularizer: instance of WeightRegularizer, applied to the bias.
  • activity_regularizer: instance of ActivityRegularizer, applied to the network output.
  • W_constraint: instance of the constraints module (eg. maxnorm, nonneg), applied to the main weights matrix.
  • b_constraint: instance of the constraints module, applied to the bias.
  • input_dim: dimensionality of the input (integer). This argument (or alternatively, the keyword argument input_shape) is required when using this layer as the first layer in a model.

[source]

ActivityRegularization

keras.layers.core.ActivityRegularization(l1=0.0, l2=0.0)

Layer that passes through its input unchanged, but applies an update to the cost function based on the activity.

Input shape

Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.

Output shape

Same shape as input.

Arguments

  • l1: L1 regularization factor.
  • l2: L2 regularization factor.

[source]

AutoEncoder

keras.layers.core.AutoEncoder(encoder, decoder, output_reconstruction=True, weights=None)

A customizable autoencoder model.

Input shape

Same as encoder input.

Output shape

If output_reconstruction = True then dim(input) = dim(output) else dim(output) = dim(hidden).

Arguments

  • encoder: A layer or layer container.
  • decoder: A layer or layer container.
  • output_reconstruction: If this is False, the output of the autoencoder is the output of the deepest hidden layer. Otherwise, the output of the final decoder layer is returned.
  • weights: list of numpy arrays to set as initial weights.

Examples

from keras.layers import containers, AutoEncoder, Dense
from keras import models

# input shape: (nb_samples, 32)
encoder = containers.Sequential([Dense(16, input_dim=32), Dense(8)])
decoder = containers.Sequential([Dense(16, input_dim=8), Dense(32)])

autoencoder = AutoEncoder(encoder=encoder, decoder=decoder, output_reconstruction=True)
model = models.Sequential()
model.add(autoencoder)

# training the autoencoder:
model.compile(optimizer='sgd', loss='mse')
model.fit(X_train, X_train, nb_epoch=10)

# predicting compressed representations of inputs:
autoencoder.output_reconstruction = False  # the model has to be recompiled after modifying this property
model.compile(optimizer='sgd', loss='mse')
representations = model.predict(X_test)

# the model is still trainable, although it now expects compressed representations as targets:
model.fit(X_test, representations, nb_epoch=1)  # in this case the loss will be 0, so it's useless

# to keep training against the original inputs, just switch back output_reconstruction to True:
autoencoder.output_reconstruction = True
model.compile(optimizer='sgd', loss='mse')
model.fit(X_train, X_train, nb_epoch=10)

[source]

MaxoutDense

keras.layers.core.MaxoutDense(output_dim, nb_feature=4, init='glorot_uniform', weights=None, W_regularizer=None, b_regularizer=None, activity_regularizer=None, W_constraint=None, b_constraint=None, input_dim=None)

A dense maxout layer.

A MaxoutDense layer takes the element-wise maximum of nb_feature Dense(input_dim, output_dim) linear layers. This allows the layer to learn a convex, piecewise linear activation function over the inputs.

Note that this is a linear layer; if you wish to apply activation function (you shouldn't need to --they are universal function approximators), an Activation layer must be added after.

Input shape

2D tensor with shape: (nb_samples, input_dim).

Output shape

2D tensor with shape: (nb_samples, output_dim).

References

[source]

Lambda

keras.layers.core.Lambda(function, output_shape=None, arguments={})

Used for evaluating an arbitrary Theano / TensorFlow expression on the output of the previous layer.

Input shape

Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.

Output shape

Specified by output_shape argument.

Arguments

  • function: The function to be evaluated. Takes one argument: the output of previous layer
  • output_shape: Expected output shape from function. Could be a tuple or a function of the shape of the input
  • arguments: optional dictionary of keyword arguments to be passed to the function.

[source]

LambdaMerge

keras.layers.core.LambdaMerge(layers, function, output_shape=None, arguments={})

LambdaMerge layer for evaluating an arbitrary Theano / TensorFlow function over multiple inputs.

Output shape

Specified by output_shape argument

Arguments

layers - Input layers. Similar to layers argument of Merge function - The function to be evaluated. Takes one argument: list of outputs from input layers output_shape - Expected output shape from function. Could be a tuple or a function of list of input shapes - arguments: optional dictionary of keyword arguments to be passed to the function.

[source]

Siamese

keras.layers.core.Siamese(layer, inputs, merge_mode='concat', concat_axis=1, dot_axes=-1, is_graph=False)

Share a layer accross multiple inputs.

For instance, this allows you to applied e.g. a same Dense layer to the output of two different layers in a graph.

Output shape

Depends on merge_mode argument

Arguments

  • layer: The layer to be shared across multiple inputs
  • inputs: Inputs to the shared layer
  • merge_mode: Same meaning as mode argument of Merge layer
  • concat_axis: Same meaning as concat_axis argument of Merge layer
  • dot_axes: Same meaning as dot_axes argument of Merge layer
  • is_graph: Should be set to True when used inside Graph

[source]

Highway

keras.layers.core.Highway(init='glorot_uniform', transform_bias=-2, activation='linear', weights=None, W_regularizer=None, b_regularizer=None, activity_regularizer=None, W_constraint=None, b_constraint=None, input_dim=None)

Densely connected highway network, a natural extension of LSTMs to feedforward networks.

Input shape

2D tensor with shape: (nb_samples, input_dim).

Output shape

2D tensor with shape: (nb_samples, input_dim).

Arguments

  • init: name of initialization function for the weights of the layer (see initializations), or alternatively, Theano function to use for weights initialization. This parameter is only relevant if you don't pass a weights argument.
  • transform_bias: value for the bias to take on initially (default -2)
  • activation: name of activation function to use (see activations), or alternatively, elementwise Theano function. If you don't specify anything, no activation is applied (ie. "linear" activation: a(x) = x).
  • weights: list of numpy arrays to set as initial weights. The list should have 2 elements, of shape (input_dim, output_dim) and (output_dim,) for weights and biases respectively.
  • W_regularizer: instance of WeightRegularizer (eg. L1 or L2 regularization), applied to the main weights matrix.
  • b_regularizer: instance of WeightRegularizer, applied to the bias.
  • activity_regularizer: instance of ActivityRegularizer, applied to the network output.
  • W_constraint: instance of the constraints module (eg. maxnorm, nonneg), applied to the main weights matrix.
  • b_constraint: instance of the constraints module, applied to the bias.
  • input_dim: dimensionality of the input (integer). This argument (or alternatively, the keyword argument input_shape) is required when using this layer as the first layer in a model.

References