Creative Commons License
This blog by Tommy Tang is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

My github papge

Friday, February 24, 2017

use bash associate array to generate commands

The problem

I am running pyclone recently to determine the clonality of our collected tumor samples. It needs tumor purity (estimated from sequenza) for input. I have 24 samples, and I want to generate pyclone commands for all of them.

The solution

I usually generate command by bash and then use another bash wrapper to generate pbs files on HPC ((Thanks to @SBAmin). now for each patient, I have two samples. How should I generate the commands? I am still better in R and Unix than python, so I used the associated array in bash.
First, generate a file containing the tumor purity for each tumor:
head -6 all_tumor_purity_no_header.txt
0.69    Pa25T1
0.26    Pa25T2
0.49    Pa26T1
0.37    Pa26T2
0.9 Pa27T1
0.92    Pa27T2
This bash script uses associate array to contain tumor purity. read more at http://www.artificialworlds.net/blog/2012/10/17/bash-associative-array-examples/
make_commands.sh:
#! /bin/bash
set -euo pipefail

## build the array to contain the tumor purity, like a python dict
## have to declare by -A
declare -A cols

while read purity sample
do
    cols[$sample]=$purity
done < all_purity_no_header.txt 

echo ${cols[@]}

## generate commands
for i in Pa{25..37}
do
   echo PyClone run_analysis_pipeline --in_file ${i}T1_pyclone.tsv ${i}T2_pyclone.tsv --tumour_contents ${cols[${i}T1]} ${cols[${i}T2]} --samples ${i}T1 ${i}T2 --density pyclone_binomial --working_dir ${i}T_pyclone_analysis --min_cluster_size 2 --seed 123 --num_iters 50000 > ${i}_pyclone_commands.txt
done
chmod u+x make_commands.sh
./make_commands.sh
what you get: cat *commands.txt
PyClone run_analysis_pipeline --in_file Pa25T1_pyclone.tsv Pa25T2_pyclone.tsv --tumour_contents 0.69 0.26 --samples Pa25T1 Pa25T2 --density pyclone_binomial --working_dir Pa25T_pyclone_analysis --min_cluster_size 2 --seed 123 --num_iters 50000
PyClone run_analysis_pipeline --in_file Pa26T1_pyclone.tsv Pa26T2_pyclone.tsv --tumour_contents 0.49 0.37 --samples Pa26T1 Pa26T2 --density pyclone_binomial --working_dir Pa26T_pyclone_analysis --min_cluster_size 2 --seed 123 --num_iters 50000
PyClone run_analysis_pipeline --in_file Pa27T1_pyclone.tsv Pa27T2_pyclone.tsv --tumour_contents 0.9 0.92 --samples Pa27T1 Pa27T2 --density pyclone_binomial --working_dir Pa27T_pyclone_analysis --min_cluster_size 2 --seed 123 --num_iters 50000
....
then:
use the makemsub wrapper:


find *commands.txt | parallel 'makemsub -a {} -j {/.} -o a -c 4 -t "48:00:00" -m 16g > {/.}.pbs'

for pbs in *pbs
do
  msub $pbs
  sleep 1
done
I usually do it for simple tasks. e.g. only one or two commands are evoked. For more complex workflows, a workflow tool such as snakemake is better.

Wednesday, February 1, 2017

How to enable the scroll mode for tmux

tmux config file

it changed the key binding from control + b to control + a if you are familiar with the screen shortcuts.
control + a + c will create a new window. control + a + Space will move to previous window. control + a + n will move to next window.
control + a + ?
will show you all the shortcuts.

scroll mode

One problem with screen or tmux is that you have to press control + a + [ to enter the copy mode, and and control + a + ] to paste it. I want to just use the mouse to scroll up and down and copy/paste.
read this long thread github issue: https://github.com/tmux/tmux/issues/145 The solution that worked for me:
git clone https://github.com/tmux-plugins/tpm ~/.tmux/plugins/tpm
Put this at the bottom of .tmux.conf:
# List of plugins
set -g @plugin 'tmux-plugins/tpm'
set -g @plugin 'tmux-plugins/tmux-sensible'

# Other examples:
# set -g @plugin 'github_username/plugin_name'
# set -g @plugin 'git@github.com/user/plugin'
# set -g @plugin 'git@bitbucket.com/user/plugin'

# Initialize TMUX plugin manager (keep this line at the very bottom of tmux.conf)
run '~/.tmux/plugins/tpm/tpm'

open your .tmux.conf.
To enable mouse-mode in tmux 2.1+, put the following line in your ~/.tmux.conf:
set-option -g mouse on

then add the following line to your .tmux.conf file:
set -g @plugin 'nhdaly/tmux-better-mouse-mode'
  • install it
# start a new session
tmux

# install plugin
`control + a + I (captial)` to install all the plugins.

Now if you scroll up with your mouse, you will enter into copy mode automatically, and when you scroll down to the end of the current screen, you will exit the copy mode automatically.
  • copy and paste
If you scroll up and select the text you want to copy by left-click and drag, you will exit the copy mode instantly, and the content you selected will be copied in the buffer. You just need to control + a + ] to paste it. very cool!

Monday, January 30, 2017

How to make a heatmap using character matrix

People are willing to read colors. I will show you how to repesent characters in heatmap. ComplexHeatmap can directly translate a character matrix into heatmap. It is good to learn basics of grid package which ComplexHeatmap and ggplot2 build on.
grid.rect is the function used to draw rectangles.
library(ComplexHeatmap)
colors<- structure(circlize::rand_color(4), names = c("a", "b", "c", "d"))
set.seed(123)
mat_letters<- matrix(sample(letters[1:4], 100, replace = TRUE), 10, 10)
## default
Heatmap(mat_letters, col = colors)
I want to have some gap between the cells:
cell_fun<-  function(j, i, x, y, width, height, fill) {
    grid.rect(x = x *0.6, y = y, width = width * 0.6, height = height, 
        gp = gpar(col = "grey", fill = fill))
}
Heatmap(mat_letters, rect_gp = gpar(type = "none"), cell_fun = cell_fun, col = colors)
It may look different in an interactive graphical device. save the figure in a pdf and you will see the effect.
## always square and have gaps between cells
cell_fun2<- function(j, i, x, y, width, height, fill) {
    s = min(unit.c(convertWidth(width, "cm"), convertHeight(height, "cm")))
    grid.rect(x = x*0.8, y = y*0.8, width = s * 0.8, height = s*0.8, 
        gp = gpar(col = "grey", fill = fill))
}
Heatmap(mat_letters, rect_gp = gpar(type = "none"), cell_fun = cell_fun2, col = colors)
cell_fun3<-  function(j, i, x, y, width, height, fill) {
    grid.rect(x = x*0.8, y = y, width = width * 0.8, height = height, 
        gp = gpar(col = "grey", fill = fill))
}
Heatmap(mat_letters, rect_gp = gpar(type = "none"), cell_fun = cell_fun3, col = colors)

make cells thiner

cell_fun4<- function(j, i, x, y, width, height, fill) {
        s = min(unit.c(convertWidth(width, "cm"), convertHeight(height, "cm")))
    grid.rect(x = x*0.6 , y = y, width = s* 0.6, height = height, 
        gp = gpar(col = "grey", fill = fill))
}
Heatmap(mat_letters, rect_gp = gpar(type = "none"), cell_fun = cell_fun4, col = colors, show_heatmap_legend = FALSE)