Linear

tflearn.activations.linear (x)

f(x) = x

Arguments

  • x : A Tensor with type float, double, int32, int64, uint8, int16, or int8.

Returns

The incoming Tensor (without changes).


Tanh

tflearn.activations.tanh (x)

Computes hyperbolic tangent of x element-wise.

Arguments

  • x: A Tensor with type float, double, int32, complex64, int64, or qint32.

Returns

A Tensor with the same type as x if x.dtype != qint32 otherwise the return type is quint8.


Sigmoid

tflearn.activations.sigmoid (x)

Computes sigmoid of x element-wise. Specifically, y = 1 / (1 + exp(-x)).

Arguments

  • x: A Tensor with type float, double, int32, complex64, int64, or qint32.

Returns

A Tensor with the same type as x if x.dtype != qint32 otherwise the return type is quint8.


Softmax

tflearn.activations.softmax (x)

Computes softmax activations.

For each batch i and class j we have

softmax[i, j] = exp(logits[i, j]) / sum(exp(logits[i]))

Arguments

  • x: A Tensor. Must be one of the following types: float32, float64. 2-D with shape [batch_size, num_classes].

Returns

A Tensor. Has the same type as x. Same shape as x.


Softplus

tflearn.activations.softplus (x)

Computes softplus: log(exp(features) + 1).

Arguments

  • x: A Tensor. Must be one of the following types: float32, float64, int32, int64, uint8, int16, int8, uint16.

Returns

A Tensor. Has the same type as x.


Softsign

tflearn.activations.softsign (x)

Computes softsign: features / (abs(features) + 1).

Arguments

  • x: A Tensor. Must be one of the following types: float32, float64, int32, int64, uint8, int16, int8, uint16.

Returns

A Tensor. Has the same type as x.


ReLU

tflearn.activations.relu (x)

Computes rectified linear: max(features, 0).

Arguments

  • x: A Tensor. Must be one of the following types: float32, float64, int32, int64, uint8, int16, int8, uint16.

Returns

A Tensor. Has the same type as x.


ReLU6

tflearn.activations.relu6 (x)

Computes Rectified Linear 6: min(max(features, 0), 6).

Arguments

  • x: A Tensor with type float, double, int32, int64, uint8, int16, or int8.

Returns

A Tensor with the same type as x.


LeakyReLU

tflearn.activations.leaky_relu (x, alpha=0.1, name='LeakyReLU')

Modified version of ReLU, introducing a nonzero gradient for negative input.

Arguments

  • x: A Tensor with type float, double, int32, int64, uint8, int16, or int8.
  • alpha: float. slope.
  • name: A name for this activation op (optional).

Returns

A Tensor with the same type as x.

References

Rectifier Nonlinearities Improve Neural Network Acoustic Models, Maas et al. (2013).

Links

http://web.stanford.edu/~awni/papers/relu_hybrid_icml2013_final.pdf


PReLU

tflearn.activations.prelu (x, channel_shared=False, weights_init='zeros', restore=True, name='PReLU')

Parametric Rectified Linear Unit.

Arguments

  • x: A Tensor with type float, double, int32, int64, uint8, int16, or int8.
  • channel_shared: bool. Single weight is shared by all channels
  • weights_init: str. Weights initialization. Default: zeros.
  • restore: bool. Restore or not alphas
  • name: A name for this activation op (optional).

Attributes

  • scope: str. This op scope.
  • alphas: Variable. PReLU alphas.

Returns

A Tensor with the same type as x.

References

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. He et al., 2014.

Links

http://arxiv.org/pdf/1502.01852v1.pdf


ELU

tflearn.activations.elu (x)

Exponential Linear Unit.

Arguments

  • x : A Tensor with type float, double, int32, int64, uint8, int16, or int8.
  • name : A name for this activation op (optional).

Returns

A tuple of tf.Tensor. This layer inference, i.e. output Tensors at training and testing time.

References

Fast and Accurate Deep Network Learning by Exponential Linear Units, Djork-Arné Clevert, Thomas Unterthiner, Sepp Hochreiter. 2015.

Links

http://arxiv.org/abs/1511.07289