1.1   Intra DOG1

by boolean KW, with FLC, FRI, peak at chr4 1.3Mb.

1.2   2009-2-15 Boolean KW

for pairs resulting high pvalues under a certain boolean operator, why is that? can you do more?

proportion of each operator as the dominant operator in different pvalue range.

Figure Intra-Gene (SNPs within 10k vicinity of the gene) Top Boolean Operator Percentage by log pvalue

Figure Inter-Top5%-SNP Top Boolean Operator Percentage by log pvalue

1.3   2009-2-15 linear model epistasis

questions:

1. figure out how many SNP pairs among all don't have all 4-allele combination.
2. epistasis pvalue=0, what happened?

2.1   GADA/mGADAr2 parameters

Choose the cutoff to call duplication/deletion based on the histogram of amplitudes for known duplication/deletions.

2.2   QC

2.2.1   position of insertion matters

One CNV probe (1, 5923348) is adjacent to one insertion from 1_5923357_1 until 1_5923357_23. But the insertion is on the border of the probe with 4 base-pair overlap. The first 2 and the 4th base of the insertion matches the reference sequence and with 1_5923357_0 a polymorphism not matching the insertion. The probe has 0 or 1 base mismatch.

Figure 1 Alignment 265 with 2nd insertion at position (1,5923357).

Figure 2 12-color legend for figure 1.

2.3   2009-6-5 Regions from Karel Riha

http://www.gmi.oeaw.ac.at/en/research/research-groups/karel-riha/

code to draw CNV matrices underlying it:

#use CNV discrete calls (1/-1) as SNP matrix
DrawSNPRegion.py -i /tmp/CycB25CycB23Region.tsv
-I /Network/Data/250k/tmp-yh/CNV/call_method_17_CNV_array_intensity_norm_chr1_GADA_M10_discrete_m.3125.tsv
-N /Network/Data/250k/tmp-yh/phenotype.tsv -l 28 -o /Network/Data/250k/tmp-yh/snp_region_CNV_M10_discrete_m.3125
-j /Network/Data/250k/tmp-yh/at_gene_model_pickelf -e 17 -u yh -s -V 2

#use CNV amplitude (float) output as SNP matrix
DrawSNPRegion.py -i /tmp/CycB25CycB23Region.tsv
-I /Network/Data/250k/tmp-yh/CNV/call_method_17_CNV_array_intensity_norm_chr1_GADA_M10_out_amp.tsv
-N /Network/Data/250k/tmp-yh/phenotype.tsv -l 28 -o /Network/Data/250k/tmp-yh/snp_region_CNV_M10_amp
-j /Network/Data/250k/tmp-yh/at_gene_model_pickelf -e 17 -u yh -s -V 3

2.4   2009-7-31 compile mGADAr2 (Pique-Regi2009) mex on hpc-cmb

All the code has been imported into svn, https://dl324b-1.cmb.usc.edu/projects/hapmap/browser/variation/trunk/bin/GADA/mGADAr2. Read the wiki page regarding how to check it out.

Roger uses c++ comment style // in his c files. By default, matlab adds -ansi to CFLAGS which forces gcc to exit upon encountering //. So need to override that option first.

1. Run below and choose gccopts.sh

   /usr/usc/matlab/default/bin/mex -setup

2. modify ~/.matlab/R2008a/mexopts.sh to remove -ansi from CFLAGS (there're several of them for different architectures).

3. Run MakeMex.m in matlab to compile all c extentions.

4. Run testFitGadaModel.m to do a test on simulation data.

5. Read svn documentation of testFitGadaModelVersion2.m and figure out how to scale it

2.5   2009-10-12 CNV normalization

Col-0 (reference genome) intensity ranges from 1.6x to 4.7x (log transformed). There are apparent duplications (probes are not well-chosen to be unique in ref genome or ref genome/TAIR 8 contains errors). Same probe appears with high intensity in different arrays for the same Col-0.

The normalization procedure proposed by Roger (Quantile Normalization + Column Median subtraction + Row Median subtraction) seems to cause the genome-wide pattern to become more zigzag. The column and row median subtraction step is questionable. The idea is to get rid of reference intensity so that zero corresponds to reference, while column and row median are not good reference candidate.

A better idea is to use intensity from multiple Col-0 arrays as row or column reference.

2.6   2009-11-04 False negative/positive rate with PE-deletions as gold standard

Compare tiling-array derived deletions with Paired-end derived ones.

Besides criteria above, if a tiling-deletion is within a PE-deletion , it's also called a match. The rationale is because tiling-deletions haven't undergone a merging process. A large deletion might be fragmented into several small adjacent deletions due to probe intensity fluctuation.

* Cvi-0 PE data has only two chromosomes (1&4). FPR is inflated.

3.1   SNP pairs with LD (r2) >= 0.95

31591 pairs with LD (r2) >=0.95. around 4500 pairs with r2=1.

13 out of 31591 are inter-chromosomal which roughly correspond to 4 regional links. 3rd column is number of base-pairs that are identical by dynamic programming, match=1, mismatch=-1, del=-2. 4th column is r2.:

link 1:

1-14286837, 5-8441445, 11, 0.96324

1-15607026, 5-8441445, 16, 0.96324

link 2:

1, 17398608, 5, 12107290, 15, 0.95658

1, 17398608, 5, 12283101, 9, 0.97071

1, 17398608, 5, 12350652, 16, 0.95578

1, 17398608, 5, 12442929, 13, 0.97024

1, 17398608, 5, 12579638, 13, 0.95658

1, 17398608, 5, 12623022, 13, 0.95658

1, 17398608, 5, 13072637, 11, 0.95658

link 3:

1, 25993897, 3, 15752587, 9, 0.96007

1, 25997144, 3, 15752587, 23, 0.96007

1, 25997609, 3, 15752587, 13/14, 0.96998

link 4:

1, 26287598, 4, 18405157, 15, 0.95170

500 out of 31578 are over 130,046 bp apart on the same chromosome. 250 out of 31578 are over 296,165 bp apart on the same chromosome.

3.2   2009-4-12 genotype combinations of SNP pairs

For two SNPs, look at the genotype combinations across accessions with non-NA phenotype. This is from call method 29 and phenotype 7.:

mpiexec ~/script/variation/src/MpiInterSNPPairAsso.py -i ~/panfs/250k/call_method_29_binary.tsv -n
~/panfs/250k/snps_context_g0_m10000  -p ~/panfs/250k/phenotype.tsv -o ~/panfs/250k/InterSNPCount -w 7 -y3
-u yh -a yh324 -t ~/panfs/db/results/type_1/ -l 29 -f 23000

out of 6,014,661,600 (need to figure out why it's this number) pairs, majority (5,877,321,290=97.7%) are 4-combos.

5.1   2009-5-1 db dump outline

at, chip, dbsnp, genome, stock

analysis_method array_info biology_category call_info call_info_pc_values call_method call_method_eigen_values
call_phenotype_qq_plots call_qc candidate_gene_list candidate_gene_list_type candidate_list_super_type 
contaminant_type maf_vs_score_plot  phenotype_avg phenotype_hist_plots phenotype_method probes 
qc_method readme results results_gene results_gene2type results_method
results_method_json results_method_type score_rank_histogram_type snp_annotation
snp_annotation_type snps snps_context web_help 
null_distribution_type score_rank_histogram_type candidate_gene_top_snp_test_rm_type  view_array view_call view_qc 

view_array, view_call, view_qc

5.2   2009-4-8 Phenotypes to be published

id as in table phenotype_method: 1-35,39-48,57-82,158-159,161-179,182-186,272-274,277-283

non-flowering time ids: 8-35,60-79,158-159,161-179,182-186

5.3   2009-5-1 procedures

outlined the procedures of chopping the full 250k database down to the public one. (instead of create a select dump out of the full 250k, i'll use copy and delete. it's easier.)

1. dataset: only keep the 192 version (actually encompassing 199 accessions)
2. remove analysis methods other than KW and EMMA
3. associations results: only the association results based on 192 dataset(above)
4. candidate gene lists: only recently added ones, id>=130.
5. phenotypes: Delete phenotypes not supposed to be public. For some phenotypes, delete the phenotype data beyond "192" or even "96".

sql:

delete from candidate_gene_list where id <130;
delete from call_method where id not in (29, 32);
delete from analysis_method where id not in (1,4,7);
delete from phenotype_method where id not in ();

5.4   2009-5-4 todo

1. add on delete cascade on foreign keys in table results_method. 2009-5-19 done.

2. create a master-slave relationship from cypress to papaya. Before that, tables that need to be synchronized.

   1. Tables that are changed on cypress

      1. stock_250k: web_help phenotype_method

   2. Tables to be changed on cypress

      1. stock_250k: remove phenotype values outside 199 accessions in table phenotype_avg:

         delete from phenotype_avg where ecotype_id not in (select ecotype_id from view_call where call_method_id=32);
         

   3. Tables that are changed on papaya

      1. dbsnp: new table accession2call_method, accession, call_method. (re-dump)
      2. stock_250k: qc_method (two new columns and 3 new entries)

3. After master-slave is established

   1. update stock.ecotype on cypress based on chicago's version.
   2. add index (tax_id, gene_symbol) to table gene_symbol2id in db genome