Sequence Generator Model

tflearn.models.generator.SequenceGenerator (network, dictionary=None, seq_maxlen=25, clip_gradients=0.0, tensorboard_verbose=0, tensorboard_dir='/tmp/tflearn_logs/', checkpoint_path=None, max_checkpoints=None, session=None)

A deep neural network model for generating sequences.

Arguments

  • network: Tensor. Neural network to be used.
  • dictionary: dict. A dictionary associating each sample with a key ( usually integers). For example: {'a': 0, 'b': 1, 'c': 2, ...}.
  • seq_maxlen: int. The maximum length of a sequence.
  • tensorboard_verbose: int. Summary verbose level, it accepts different levels of tensorboard logs:
0 - Loss, Accuracy (Best Speed).
1 - Loss, Accuracy, Gradients.
2 - Loss, Accuracy, Gradients, Weights.
3 - Loss, Accuracy, Gradients, Weights, Activations, Sparsity.(Best visualization)
  • tensorboard_dir: str. Directory to store tensorboard logs. Default: "/tmp/tflearn_logs/"
  • checkpoint_path: str. Path to store model checkpoints. If None, no model checkpoint will be saved. Default: None.
  • max_checkpoints: int or None. Maximum amount of checkpoints. If None, no limit. Default: None.
  • session: Session. A session for running ops. If None, a new one will be created. Note: When providing a session, variables must have been initialized already, otherwise an error will be raised.

Attributes

  • trainer: Trainer. Handle model training.
  • predictor: Predictor. Handle model prediction.
  • session: Session. The current model session.

Methods

evaluate (X, Y, batch_size=128)

Evaluate model on given samples.

Arguments
  • X: array, list of array (if multiple inputs) or dict (with inputs layer name as keys). Data to feed to train model.
  • Y: array, list of array (if multiple inputs) or dict (with estimators layer name as keys). Targets (Labels) to feed to train model. Usually set as the next element of a sequence, i.e. for x[0] => y[0] = x[1].
  • batch_size: int. The batch size. Default: 128.
Returns

The metric score.

fit (X_inputs, Y_targets, n_epoch=10, validation_set=None, show_metric=False, batch_size=None, shuffle=None, snapshot_epoch=True, snapshot_step=None, excl_trainops=None, run_id=None)

Train model, feeding X_inputs and Y_targets to the network.

NOTE: When not feeding dicts, data assignations is made by input/estimator layers creation order (For example, the second input layer created will be feeded by the second value of X_inputs list).

Examples
model.fit(X, Y) # Single input and output
model.fit({'input1': X}, {'output1': Y}) # Single input and output
model.fit([X1, X2], Y) # Mutliple inputs, Single output

# validate with X_val and [Y1_val, Y2_val]
model.fit(X, [Y1, Y2], validation_set=(X_val, [Y1_val, Y2_val]))
# 10% of training data used for validation
model.fit(X, Y, validation_set=0.1)
Arguments
  • X_inputs: array, list of array (if multiple inputs) or dict (with inputs layer name as keys). Data to feed to train model.
  • Y_targets: array, list of array (if multiple inputs) or dict (with estimators layer name as keys). Targets (Labels) to feed to train model. Usually set as the next element of a sequence, i.e. for x[0] => y[0] = x[1].
  • n_epoch: int. Number of epoch to run. Default: None.
  • validation_set: tuple. Represents data used for validation. tuple holds data and targets (provided as same type as X_inputs and Y_targets). Additionally, it also accepts float (<1) to performs a data split over training data.
  • show_metric: bool. Display or not accuracy at every step.
  • batch_size: int or None. If int, overrides all network estimators 'batch_size' by this value.
  • shuffle: bool or None. If bool, overrides all network estimators 'shuffle' by this value.
  • snapshot_epoch: bool. If True, it will snapshot model at the end of every epoch. (Snapshot a model will evaluate this model on validation set, as well as create a checkpoint if 'checkpoint_path' specified).
  • snapshot_step: int or None. If int, it will snapshot model every 'snapshot_step' steps.
  • excl_trainops: list of TrainOp. A list of train ops to exclude from training process (TrainOps can be retrieve through tf.get_collection_ref(tf.GraphKeys.TRAIN_OPS)).
  • run_id: str. Give a name for this run. (Useful for Tensorboard).

generate (seq_length, temperature=0.5, seq_seed=None, display=False)

Generate a sequence. Temperature is controlling the novelty of the created sequence, a temperature near 0 will looks like samples used for training, while the higher the temperature, the more novelty. For optimal results, it is suggested to set sequence seed as some random sequence samples from training dataset.

Arguments
  • seq_length: int. The generated sequence length.
  • temperature: float. Novelty rate.
  • seq_seed: sequence. A sequence used as a seed to generate a new sequence. Suggested to be a sequence from data used for training.
  • display: bool. If True, print sequence as it is generated.
Returns

The generated sequence.

get_weights (weight_tensor)

Get a variable weights.

Examples

sgen = SequenceGenerator(...) w = sgen.get_weights(denselayer.W) -- get a dense layer weights

Arguments
  • weight_tensor: tf.Tensor. A Variable.
Returns

np.array. The provided variable weights.

load (model_file, **optargs)

Restore model weights.

Arguments
  • model_file: str. Model path.
  • optargs: optional extra arguments for trainer.restore (see helpers/trainer.py) These optional arguments may be used to limit the scope of variables restored, and to control whether a new session is created for the restored variables.

save (model_file)

Save model weights.

Arguments
  • model_file: str. Model path.

set_weights (tensor, weights)

Assign a tensor variable a given value.

Arguments
  • tensor: Tensor. The tensor variable to assign value.
  • weights: The value to be assigned.