Installation
The R software for running systemPipeR
can be downloaded from CRAN. The systemPipeR
environment can be installed from the R console using the biocLite
install command. The associated data package systemPipeRdata
can be installed the same way. The latter is a helper package for generating systemPipeR
workflow environments with a single command containing all parameter files and sample data required to quickly test and run workflows.
Loading package and documentation
Load sample data and workflow templates
The mini sample FASTQ files used by this overview vignette as well as the associated workflow reporting vignettes can be loaded via the systemPipeRdata
package as shown below. The chosen data set SRP010938
contains 18 paired-end (PE) read sets from Arabidposis thaliana (Howard et al., 2013). To minimize processing time during testing, each FASTQ file has been subsetted to 90,000-100,000 randomly sampled PE reads that map to the first 100,000 nucleotides of each chromosome of the A. thalina genome. The corresponding reference genome sequence (FASTA) and its GFF annotion files (provided in the same download) have been truncated accordingly. This way the entire test sample data set requires less than 200MB disk storage space. A PE read set has been chosen for this test data set for flexibility, because it can be used for testing both types of analysis routines requiring either SE (single end) reads or PE reads.
The following generates a fully populated systemPipeR
workflow environment (here for RNA-Seq) in the current working directory of an R session. At this time the package includes workflow templates for RNA-Seq, ChIP-Seq, VAR-Seq and Ribo-Seq. Templates for additional NGS applications will be provided in the future.
Download latest version of this tutorial
In case there is an updated version of this tutorial, download its systemPipeRNAseq.Rmd
source.
Now open the R markdown script systemPipeRNAseq.Rmd
in your R IDE (e.g. vim-r or RStudio) and
run the workflow as outlined below.
If you are on Windows please run the following command to move all input files into the right location. This problem will be fixed next week.
The working environment of the sample data loaded in the previous step contains the following preconfigured directory structure. Directory names are indicated in grey. Users can change this structure as needed, but need to adjust the code in their workflows accordingly.
- workflow/ (e.g. rnaseq/)
- This is the directory of the R session running the workflow.
- Run script ( *.Rnw or *.Rmd) and sample annotation (targets.txt) files are located here.
- Note, this directory can have any name (e.g. rnaseq, varseq). Changing its name does not require any modifications in the run script(s).
- Important subdirectories:
- param/
- Stores parameter files such as: *.param, *.tmpl and *_run.sh.
- data/
- FASTQ samples
- Reference FASTA file
- Annotations
- etc.
- results/
- Alignment, variant and peak files (BAM, VCF, BED)
- Tabular result files
- Images and plots
- etc.
- param/
The following parameter files are included in each workflow template:
targets.txt
: initial one provided by user; downstreamtargets_*.txt
files are generated automatically*.param
: defines parameter for input/output file operations, e.g.trim.param
,bwa.param
,vartools.parm
, …*_run.sh
: optional bash script, e.g.:gatk_run.sh
- Compute cluster environment (skip on single machine):
.BatchJobs
: defines type of scheduler forBatchJobs
*.tmpl
: specifies parameters of scheduler used by a system, e.g. Torque, SGE, StarCluster, Slurm, etc.
Structure of targets
file
The targets
file defines all input files (e.g. FASTQ, BAM, BCF) and sample comparisons of an analysis workflow. The following shows the format of a sample targets
file included in the package. It also can be viewed and downloaded from systemPipeR
’s GitHub repository here. In a target file with a single type of input files, here FASTQ files of single end (SE) reads, the first three columns are mandatory including their column names, while it is four mandatory columns for FASTQ files of PE reads. All subsequent columns are optional and any number of additional columns can be added as needed.
Structure of targets
file for single end (SE) samples
To work with custom data, users need to generate a targets
file containing the paths to their own FASTQ files and then provide under targetspath
the path to the corresponding targets
file.
Structure of targets
file for paired end (PE) samples
Sample comparisons
Sample comparisons are defined in the header lines of the targets
file starting with ‘# <CMP>
’.
The function readComp
imports the comparison information and stores it in a list
. Alternatively, readComp
can obtain the comparison information from the corresponding SYSargs
object (see below).
Note, these header lines are optional. They are mainly useful for controlling comparative analyses according to certain biological expectations, such as identifying differentially expressed genes in RNA-Seq experiments based on simple pair-wise comparisons.
Structure of param
file and SYSargs
container
The param
file defines the parameters of a chosen command-line software. The following shows the format of a sample param
file provided by this package.
The systemArgs
function imports the definitions of both the param
file
and the targets
file, and stores all relevant information in a SYSargs
object (S4 class). To run the pipeline without command-line software, one can
assign NULL
to sysma
instead of a param
file. In addition, one can
start systemPipeR
workflows with pre-generated BAM files by providing a
targets file where the FileName
column provides the paths to the BAM files.
Note, in the following example the usage of suppressWarnings()
is only relevant for
building this vignette. In typical workflows it should be removed.
Several accessor methods are available that are named after the slot names of the SYSargs
object.
Of particular interest is the sysargs()
method. It constructs the system
commands for running command-lined software as specified by a given param
file combined with the paths to the input samples (e.g. FASTQ files) provided
by a targets
file. The example below shows the sysargs()
output for
running TopHat2 on the first PE read sample. Evaluating the output of
sysargs()
can be very helpful for designing and debugging param
files
of new command-line software or changing the parameter settings of existing
ones.
The content of the param
file can also be returned as JSON object as follows (requires rjson
package).