PositionalEncoding< InputDataType, OutputDataType > Class Template Reference

Positional Encoding injects some information about the relative or absolute position of the tokens in the sequence. More...

Public Member Functions
	PositionalEncoding ()
	Create PositionalEncoding object. More...
	PositionalEncoding (const size_t embedDim, const size_t maxSequenceLength)
	Create the PositionalEncoding layer object using the specified parameters. More...
template<typename eT >
void	Backward (const arma::Mat< eT > &, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...
OutputDataType const &	Delta () const
	Get the delta. More...
OutputDataType &	Delta ()
	Modify the delta. More...
InputDataType const &	Encoding () const
	Get the positional encoding vector. More...
template<typename eT >
void	Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...
InputDataType const &	InputParameter () const
	Get the input parameter. More...
InputDataType &	InputParameter ()
	Modify the input parameter. More...
OutputDataType const &	OutputParameter () const
	Get the output parameter. More...
OutputDataType &	OutputParameter ()
	Modify the output parameter. More...
template<typename Archive >
void	serialize (Archive &ar, const unsigned int)
	Serialize the layer. More...

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::PositionalEncoding< InputDataType, OutputDataType >

Positional Encoding injects some information about the relative or absolute position of the tokens in the sequence.

The input and the output have the same shape: (embedDim * maxSequenceLength, batchSize). The embeddings are stored consequently.

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 37 of file positional_encoding.hpp.

Constructor & Destructor Documentation

PositionalEncoding

(

)

Create PositionalEncoding object.

PositionalEncoding	(	const size_t	embedDim,
		const size_t	maxSequenceLength
	)

Create the PositionalEncoding layer object using the specified parameters.

Parameters

embedDim	The length of the embedding vector.
maxSequenceLength	Number of tokens in each sequence.

Member Function Documentation

void Backward	(	const arma::Mat< eT > &	,
		const arma::Mat< eT > &	gy,
		arma::Mat< eT > &	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters

*	(input) The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

OutputDataType const& Delta ( ) const

inline

Get the delta.

Definition at line 89 of file positional_encoding.hpp.

OutputDataType& Delta ( )

inline

Modify the delta.

Definition at line 91 of file positional_encoding.hpp.

InputDataType const& Encoding ( ) const

inline

Get the positional encoding vector.

Definition at line 94 of file positional_encoding.hpp.

void Forward	(	const arma::Mat< eT > &	input,
		arma::Mat< eT > &	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

InputDataType const& InputParameter ( ) const

inline

Get the input parameter.

Definition at line 79 of file positional_encoding.hpp.

InputDataType& InputParameter ( )

inline

Modify the input parameter.

Definition at line 81 of file positional_encoding.hpp.

OutputDataType const& OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 84 of file positional_encoding.hpp.

OutputDataType& OutputParameter ( )

inline

Modify the output parameter.

Definition at line 86 of file positional_encoding.hpp.

void serialize	(	Archive &	ar,
		const unsigned	int
	)

Serialize the layer.

---

The documentation for this class was generated from the following file:

• src/mlpack/methods/ann/layer/positional_encoding.hpp