--------------------------------------------------------------------------------
TEclass - a tool for classification of consensus sequences of eukaryotic 
transposable elements. 

Version 2.1 (27th Jan 2011)
--------------------------------------------------------------------------------

TEclass is a Perl pipeline to classify the consensus sequences of unknown 
eukaryotic transposable elements according to their mechanism of transposition,
using machine learning methods. It can classify them into four categories: DNA 
transposons, LTRs, LINEs, SINEs, and it also identifies ORFs. Note that it 
assumes that ATG is the start codon (and TAG, TGA, TAA the stop codons); in 
case your sequences are from an organism with a different genetic code the ORF 
predictions will be less accurate.   


SYSTEM REQUIREMENTS
Linux, UNIX, Cygwin or Mac 

PREREQUISITES
Standard linux tools: Bash, Perl, Python, gcc, (gnu) make. On Mac you have to install 
the command line tools from Apple first. (https://developer.apple.com/opensource) 


INSTALLATION
1. Uncompress TEclass.tar.gz in your preferred destination directory
2. Change directory to TEclass-2.1.3/
3. type 'sh Download_dependencies.sh'  
4. type 'sh Compile_dependencies.sh'    
5. type 'perl Configure.pl'
6. type 'sh Install.sh  /some/directory'    (optional)

This will download and compile all dependencies in the TEclass directory, except 
Blastclust, which is avalialble as a precompiled binary for Linux, FreeBSD, 
Solaris, Mac and Windows. The compilation takes a few minutes on desktop 
(gcc 4.9.2). The Install.sh script creates symlinks in the specified directory 
to TEclassTest.pl and TEclassBuild.pl scripts, the default is /usr/local/bin. 
It doesn't copy anything.

In case you decide to install the dependencies on your own, skip steps 3 and 4, 
and run only the Configure.pl script (and eventually Install.sh). This will 
create a Configuration.pm file, where you will have to edit the paths to each 
dependency on your own. In case you have difficulties with compiling some of the 
tools, the Compile_dependencies.sh script may contain some helpful information. 
If you only plan to classify repeats, and have no intention to build your own 
classifiers (what is very time consuming), you will not need Blastclust, or the 
repeat databases (RepBase and RepeatMasker libraries), so there is no need to 
download them or edit their path variables in Configuration.pm.   

The dependencies are:
  
 - LIBSVM package - a library for Support Vector Machines
       http://www.csie.ntu.edu.tw/~cjlin/libsvm/
 - Glimmer3 - a gene prediction tool
       http://ccb.jhu.edu/software/glimmer/
 - LIBRF package - a library for random forest classification
       http://mtv.ece.ucsb.edu/benlee/librf.html
 - LVQ_PAK - Kohonen's Learning Vector Quantisation package 
       http://www.cis.hut.fi/research/som-research/nnrc-programs.shtml
 - Blastclust executable from the NCBI Blast package
       ftp://ftp.ncbi.nlm.nih.gov/blast/executables/release/2.2.26/


TEclass consists of the follwing scripts:  
 - TEclassTest.pl 
 - TEclassBuild.pl
 - TEclass.pm  
 - Build_ICM_model.pl
 - Build_LVQ_sets.pl
 - Build_RandomForests.pl
 - Build_SVM_models.pl
 - Configure.pl
 - Preprocesing.pl 
 - Download_dependencies.sh
 - Compile_dependencies.sh
 - Install.sh


TESTIG REPEATS
To test repeats you need classifiers. You can either build them on your own 
(see below), or download the prebuilt classifiers from 
http://www.compgen.uni-muenster.de/tools/teclass/download/classifiers.tar.gz. 
Note that when uncompressed, they are over 1 GB. Once the classifiers are 
available you can classify the unknown TE consensus sequences with the 
TEclassTest.pl script, that usually takes only minutes. The consensus 
sequences have to be provided in fasta format. Note that the program always 
assumes that the provided sequence is a TE, it cannot distinguish TEs 
from non-TEs.

Example: 
./TEclassTest.pl [flags] testfile.fa

You can check your installation by comparing the output with the contents of 
the testfile directory.

Optional flags:
   -h help
   -c '/some/directory' : uses a user specified path to the classifiers. 
      Without the flag the default is the TEclass-2.1.3/classifiers 
      directory.
   -o '/some/directory' : writes the output to a user specified directory. 
      Without the flag an output directory is created in the current folder 
      using the input filename and a timestamp.
   -t Use for testing purposes only. It uses a decision algorithm that 
      performs better when the SVM models were built with a realtively 
      small subset of RepBase (~ -s 300-500). 
   -r Reverse complements the input sequence in the *.lib file (and 
      adjusts the ORF cordinates accordingly) if it is predicted to be 
      in the reverse orientation.
 
The output directory contains four files : the inputfile, an HTML file 
with the separate results of each classification method and ORF prediction, 
a fasta file (*.lib) containing the final classification result, ORF coordinates 
and the sequence; and a text file (*.stat) with the classification statistics. 



BUILDING THE CLASSIFIERS
Building the classifiers using the entire RepBase can be very time consuming, it
may take several weeks on a single CPU (depending on settings), mostly due to 
building of SVM models with pentamers. To build them, you have to download the 
EMBL version of RepBase, and also the RepeatMasker libraries from GIRI
(http://www.girinst.org/repbase/index.html - password required). After 
downloading and unpacking them run the Configure.pl script (or edit the 
Configuration.pm module manually) to update the configuration.  
    
Once this is ready, one can build the classifiers with the TEclassBuild script:
    
    ./TEclassBuild.pl -x <int> [+ optional flags]
    
This will build the gene models, random forests, learning vectors, and 
SVM models used for repeat classification. These models can also be built 
separately, by running the corresponding Build script (after preprocessing). 
      
The -x flag specifies the way TEclass searches for optimal parameters (C and g) 
for the SVM classification, which is by far the most time consuming step. All 
other classifiers can be built within a few minutes or hours on a pc. 

If -x is set to 0, TEclass examines a full matrix of C-g combinations (alltogether 
99 combinations) for each model, and selects the best performing C-g pair for the 
training by five-fold cross validation. With the current size of RepBase, it 
takes about 2 months using 8 CPUs to build the classifiers with this option. 
(The prebuilt classifiers are made with the -x 0 option). 

If -x is set to 1, it examines only a list of C-g pairs which we found to 
perform well on repeats; it results in an ~ 8x speedup compared with the -x 0 
option, as much less pairs are tested. We recommend using this option, it is 
likely to result in essentially similar quality classifiers than the -x 0 option.

If -x is set to 2, TEclass does not search for the best C and g parameters, 
but uses either default ones (C=17 and g=-11), or user specified values, if 
provided. This requires significanlty less computing power than options 0 or 1, 
however also the quality of the models will also be worse.    

Other flags:                                
-C <int> : user defined C (only if -x is set to 2)
-g <int> : user defined g (only if -x is set to 2)   
-w 'workdir': Sets the working directory for tmp files. Default is ./workdir
-o 'outdir' : Sets the output directory. Default is TEclassdir/classifiers       
-s <int> : builds SVM models on samples of specified size, i.e. each classifier
           is built using a random sample of N sequences. Sample size influences 
           both the speed and quality of the model building, we recommend to use 
           it only for testing purposes (-s 300 to 500).   
-m <int> : sets the memory allocated to each CPU for SVM (default is 1024 MB)
-p <int> : sets the number of CPUs used in SVM training, default is 1. 
           If the -x 0 option is selected, one can use up to 99 CPUs, with 
           -x 1 up to 12, while the -x 2 option is not parallelized.                  
-h : help 


HOW TO CITE
Please cite the article  "Gyrgy Abrusn, Norbert Grundmann, Luc DeMester, and 
Wojciech Makalowski: TEclass: a tool for automated classification of unknown 
eukaryotic transposable elements". Bioinformatics 2009 25(10):1329-1330
(http://bioinformatics.oxfordjournals.org/)


DISCLAIMER
The program is provided "as is", with no warranty that it works on your machine, 
and without responsibility for any damage it may cause to your system. 

Gyorgy Abrusan (gyorgy.abrusan@gmail.com)
