Usage of metrics

A metric is a function that is used to judge the performance of your model. Metric functions are to be supplied in the metrics parameter when a model is compiled.

A metric function is similar to an objective function, except that the results from evaluating a metric are not used when training the model.

You can either pass the name of an existing metric, or pass a Theano/TensorFlow symbolic function (see Custom metrics).


  • y_true: True labels. Theano/TensorFlow tensor.
  • y_pred: Predictions. Theano/TensorFlow tensor of the same shape as y_true.


Single tensor value representing the mean of the output array across all datapoints.

Available metrics


binary_accuracy(y_true, y_pred)

Calculates the mean accuracy rate across all predictions for binary classification problems


categorical_accuracy(y_true, y_pred)

Calculates the mean accuracy rate across all predictions for multiclass classification problems


sparse_categorical_accuracy(y_true, y_pred)

Same as categorical_accuracy, but useful when the predictions are for sparse targets


top_k_categorical_accuracy(y_true, y_pred, k=5)

Calculates the top-k categorical accuracy rate, i.e. success when the target class is within the top-k predictions provided


mean_squared_error(y_true, y_pred)

Calculates the mean squared error (mse) rate between predicted and target values


mean_absolute_error(y_true, y_pred)

Calculates the mean absolute error (mae) rate between predicted and target values


mean_absolute_percentage_error(y_true, y_pred)

Calculates the mean absolute percentage error (mape) rate between predicted and target values


mean_squared_logarithmic_error(y_true, y_pred)

Calculates the mean squared logarithmic error (msle) rate between predicted and target values


hinge(y_true, y_pred)

Calculates the hinge loss, which is defined as max(1 - y_true * y_pred, 0)


squared_hinge(y_true, y_pred)

Calculates the squared value of the hinge loss


categorical_crossentropy(y_true, y_pred)

Calculates the cross-entropy value for multiclass classification problems. Note: Expects a binary class matrix instead of a vector of scalar classes.


sparse_categorical_crossentropy(y_true, y_pred)

Calculates the cross-entropy value for multiclass classification problems with sparse targets. Note: Expects an array of integer classes. Labels shape must have the same number of dimensions as output shape. If you get a shape error, add a length-1 dimension to labels.


binary_crossentropy(y_true, y_pred)

Calculates the cross-entropy value for binary classification problems.


kullback_leibler_divergence(y_true, y_pred)

Calculates the Kullback-Leibler (KL) divergence between prediction and target values


poisson(y_true, y_pred)

Calculates the poisson function over prediction and target values.


cosine_proximity(y_true, y_pred)

Calculates the cosine similarity between the prediction and target values.


matthews_correlation(y_true, y_pred)

Calculates the Matthews correlation coefficient measure for quality of binary classification problems.


fbeta_score(y_true, y_pred, beta=1)

Computes the F score, the weighted harmonic mean of precision and recall.

This is useful for multi-label classification where input samples can be tagged with a set of labels. By only using accuracy (precision) a model would achieve a perfect score by simply assigning every class to every input. In order to avoid this, a metric should penalize incorrect class assignments as well (recall). The F-beta score (ranged from 0.0 to 1.0) computes this, as a weighted mean of the proportion of correct class assignments vs. the proportion of incorrect class assignments.

With beta = 1, this is equivalent to a F-measure. With beta < 1, assigning correct classes becomes more important, and with beta > 1 the metric is instead weighted towards penalizing incorrect class assignments.

Custom metrics

Custom metrics can be defined and passed via the compilation step. The function would need to take (y_true, y_pred) as arguments and return either a single tensor value or a dict metric_name -> metric_value.

# for custom metrics
import keras.backend as K

def mean_pred(y_true, y_pred):
    return K.mean(y_pred)

def false_rates(y_true, y_pred):
    false_neg = ...
    false_pos = ...
    return {
        'false_neg': false_neg,
        'false_pos': false_pos,

              metrics=['accuracy', mean_pred, false_rates])