Speech Processing Group

Overview:

A fast tool for parallel training of neural networks for classification. There are two sets of tools, the CPU and GPU tools. The fast CPU training is based on multithread data-parallelization approach. The GPU training is is implemented by means of CUDA GPGPU, it can be viewed as o form of node-parallelization approach. The training algorithm is block-wise Stochastic Gradient Descent.

With the toolkit you may experiment with various NN tricks such as convolutive networks with shared weights, RBM pre-training by Contrastive Divergence and sequence-based MPE training.

Distributions:

The package contains sourcecodes, libraries and training example:
http://speech.fit.vutbr.cz/files/software/tnet/TNet_v1_8_r115.tar.gz

Various libraries are used in TNet:

• KaldiLib this is the subset of classes from kaldi 1.0 project, it contains matrix/vector classes, features/labels I/O and UI
• GotoBLAS is used for acceleration of linear algebra on CPU
• CUDA is used to facilitate GPGPU computing
• pthreads are used for multithreading and synchronisation
• STKLib is used for lattice rescoring in MPE trainig (most of the tools are STK-independent)

The neural networks are viewed as a sequence of linear/nonlinear transforms (BiasedLinearity, Biases, Sigmoid or Softmax), which are able to propagate, backpropagate and update. You may implement your own transforms, (see interfaces Component and UpdatableComponent). Cross-entropy training criterion is used, however also Mean Square Error criterion is implemented.
The feature/label files are HTK compatible.

Licence:

The tool is distributed under Apache v2.0 licence:
http://www.apache.org/licenses/LICENSE-2.0.txt