logo

This session will guide you to use DESeq2 to identify differentially expressed genes and become familiar with design matrices. The session will also introduce concepts essential for working with real-world data, such as incorporating covariates and interaction terms into models.

The objectives of this session are:

  • Obtain familiarity with the DESeq2 workflow
  • Design and configure differential expression models appropriately for RNA-seq data

Note: All the code in this tutorial will be performed in R (or Rstudio).

Dataset

This workshop will use processed RNA-seq counts from this study, where they studied the transcriptional response to SARS-CoV-2 infection in three tissues (cornea, limbus, sclera). The dataset consists of 18 samples, with three replicates per tissue and condition (mock vs CoV-2).

The data can be found here:

ls ../data/
c2.cp.kegg_legacy.v2025.1.Hs.symbols.gmt
GSE164073_Eye_count_matrix.tsv
GSE164073_Eye_count_meta.tsv
h.all.v2025.1.Hs.symbols.gmt
res_Cornea_CoV2_vs_Mock.rds

DEseq2 workflow

Load libraries

library(DESeq2)
library(vsn) # Used for standard deviation vs mean plots
library(hexbin) # Used for standard deviation vs mean plots
library(pheatmap) 
library(RColorBrewer)
library(EnhancedVolcano)
library(ggplot2)
library(ggbeeswarm)
library(ashr) # Used for log-fold shrinkage
library(knitr) # Used for formatting output

Load the data

Let’s load the dataset that consists of a raw count matrix and a metadata.

raw_counts<- read.table("../data/GSE164073_Eye_count_matrix.tsv", header=T, row.names=1)
meta<- read.table("../data/GSE164073_Eye_count_meta.tsv", row.names=1)
kable(head(raw_counts[,1:5]))
MW1 MW2 MW3 MW4 MW5
A1BG 91 131 86 77 69
A1BG-AS1 292 284 271 232 250
A1CF 0 0 0 0 0
A2M 255 273 263 150 176
A2M-AS1 8 9 9 11 16
A2ML1 0 0 0 0 0
kable(head(meta))
Tissue Condition
MW1 cornea mock
MW2 cornea mock
MW3 cornea mock
MW4 cornea CoV2
MW5 cornea CoV2
MW6 cornea CoV2

Create DESeq2 object

The first step is to create a design matrix. Let’s start by creating a very simple design matrix.

# Converting Tissue into a factor
meta$Tissue<- factor(meta$Tissue)
# Adjusting levels for Condition, so that mock is the reference
meta$Condition<- factor(meta$Condition, levels=c("mock", "CoV2"))
meta_design<- model.matrix( ~ Condition, data=meta)
kable(head(meta_design))
(Intercept) ConditionCoV2
MW1 1 0
MW2 1 0
MW3 1 0
MW4 1 1
MW5 1 1
MW6 1 1

This design matrix is answering the question: What are the overall differences between Mock and CoV2, without considering the tissues?

Now, let’s create a DESeq2 object.

dds <- DESeqDataSetFromMatrix(countData = raw_counts,
                              colData = meta,
                              design= meta_design)
dds
class: DESeqDataSet 
dim: 27946 18 
metadata(1): version
assays(1): counts
rownames(27946): A1BG A1BG-AS1 ... ZZEF1 ZZZ3
rowData names(0):
colnames(18): MW1 MW2 ... MW17 MW18
colData names(2): Tissue Condition

Pre-filtering

Removing low expressed counts is generally recommended for memory and speed reasons.

smallestGroupSize <- 3
keep <- rowSums(counts(dds) >= 10) >= smallestGroupSize
dds <- dds[keep,]
dds
class: DESeqDataSet 
dim: 14700 18 
metadata(1): version
assays(1): counts
rownames(14700): A1BG A1BG-AS1 ... ZZEF1 ZZZ3
rowData names(0):
colnames(18): MW1 MW2 ... MW17 MW18
colData names(2): Tissue Condition

Quality control

VST

We can obtain a variance stabilizing transformation

# Obtain the variance stabilizing transformation
vsd <- vst(dds)
# Let's compare it to the normal transformation
ntd <- normTransform(dds)
meanSdPlot(assay(ntd), ranks=F) # Normal

meanSdPlot(assay(vsd), ranks=F) # VST

We observe that the vst removes the dependence of the variance on the mean, which is evident for the low expressed genes.

Heatmap of the sample-to-sample distances

We can obtain distances between samples and then plot them as a heatmap.

sampleDists <- dist(t(assay(vsd)))
sampleDistMatrix <- as.matrix(sampleDists)
rownames(sampleDistMatrix) <- paste(vsd$Tissue, vsd$Condition, sep="-")
colnames(sampleDistMatrix) <- NULL
colors <- colorRampPalette( rev(brewer.pal(9, "Blues")) )(255)
pheatmap(sampleDistMatrix,
         clustering_distance_rows=sampleDists,
         clustering_distance_cols=sampleDists,
         col=colors)

We observe that in general, samples from the same tissue cluster together, as expected.

Principal component of the samples

We can also perform a PCA analysis of the VST counts.

plotPCA(vsd, intgroup=c("Tissue", "Condition")) +
    geom_point(size=0.5)
using ntop=500 top features by variance

Again, we observe that the first PCA is mostly associated with tissue, although samples can also subcluster between the mock and CoV2 conditions.

Differential expression

The standard DE is performed by the DESeq function.

dds <- DESeq(dds)
using supplied model matrix
estimating size factors
estimating dispersions
gene-wise dispersion estimates
mean-dispersion relationship
final dispersion estimates
fitting model and testing
dds
class: DESeqDataSet 
dim: 14700 18 
metadata(1): version
assays(5): counts mu H cooks originalCounts
rownames(14700): A1BG A1BG-AS1 ... ZZEF1 ZZZ3
rowData names(23): baseMean baseVar ... maxCooks replace
colnames(18): MW1 MW2 ... MW17 MW18
colData names(4): Tissue Condition sizeFactor replaceable
resultsNames(dds)
[1] "Intercept"     "ConditionCoV2"
Intercept
ConditionCoV2
res <- results(dds, name="ConditionCoV2") # This function provides the lof2Fold Changes and p-values for a given coefficient name
res
log2 fold change (MLE): ConditionCoV2 
Wald test p-value: ConditionCoV2 
DataFrame with 14700 rows and 6 columns
          baseMean log2FoldChange     lfcSE      stat    pvalue      padj
         <numeric>      <numeric> <numeric> <numeric> <numeric> <numeric>
A1BG       90.4475     -0.1439495  0.118021 -1.219695  0.222581  0.803126
A1BG-AS1  278.3007     -0.0480946  0.112076 -0.429126  0.667832  0.963881
A2M       873.3212     -0.8092580  0.741401 -1.091525  0.275042  0.833286
A2M-AS1    11.3404      0.4892287  0.256631  1.906348  0.056605  0.531853
A4GALT    775.8491      0.1133252  0.223626  0.506761  0.612322  0.951852
...            ...            ...       ...       ...       ...       ...
ZYG11A     17.6651     -0.3825263 0.3813743 -1.003021  0.315851  0.855794
ZYG11B   1385.1616      0.0993295 0.1068925  0.929246  0.352761  0.874910
ZYX      6722.2007     -0.2497730 0.1098916 -2.272903  0.023032  0.359851
ZZEF1    1364.0281      0.1296682 0.0923057  1.404769  0.160090  0.741104
ZZZ3     1187.3067      0.1200589 0.0744052  1.613582  0.106618  0.661463

Then, a results table can be generated. In this case, we provide a value in the ‘name’ argument in the results function, which correspondds to the CoV2 effect encoded in our design matrix. We will also explore other ways to access results in the follow up sections.

Log-Fold shrinkage

Shrinkage of the log-fold changes estimates is recommended for visualization and ranking of genes.

resLFC <- lfcShrink(dds, coef="ConditionCoV2", type="apeglm")
using 'apeglm' for LFC shrinkage. If used in published research, please cite:
    Zhu, A., Ibrahim, J.G., Love, M.I. (2018) Heavy-tailed prior distributions for
    sequence count data: removing the noise and preserving large differences.
    Bioinformatics. https://doi.org/10.1093/bioinformatics/bty895

The lfcShrink has various shrinkage methods, here we will use apeglm. Again, we look for the CoV2 effect model by setting up the coef argument in the lfcShrink function.

MA plots

Let’s compare the MA plots between the shrunken FC and the regular FC.

plotMA(res)

plotMA(resLFC)

You can see that the shrunken log2 fold changes remove the noise associated with log2 fold changes from low count genes.

Volcano plot

EnhancedVolcano(resLFC,
                lab = rownames(resLFC),
                x = 'log2FoldChange',
                y = 'pvalue')

We can explore also the results using a Volcano Plot.

Plot counts

geneCounts <- plotCounts(dds, gene = "SOD2", intgroup = c("Condition","Tissue"),returnData = T)
ggplot(geneCounts, aes(x = Condition, y = log2(count), color = Tissue)) +  geom_beeswarm(cex = 3)

We can also plot the normalized count values for a given gene.

Heatmaps

sig_genes<- rownames(resLFC[with(resLFC, abs(log2FoldChange > 1) & padj < 0.05),])
colors <- colorRampPalette(c("darkblue","white","darkred"))(255)
pheatmap(assay(vsd[sig_genes,]), annotation_col=meta,col=colors, scale="row")

Finally, we can generate a heatmap for a subset of genes with significant DE changes (padj < 0.05 and log2FoldChange >1).

Tissue-specific effects

So far, we haven’t considered the different tissues into our model. One way to study the tissue-specific effects is through a mean group model.

meta$group<- factor(paste0(".", meta$Tissue, "_", meta$Condition))
meta_design<- model.matrix( ~ 0 + group, data=meta) # We remove the intercept in this model
kable(head(meta_design))
group.cornea_CoV2 group.cornea_mock group.limbus_CoV2 group.limbus_mock group.sclera_CoV2 group.sclera_mock
MW1 0 1 0 0 0 0
MW2 0 1 0 0 0 0
MW3 0 1 0 0 0 0
MW4 1 0 0 0 0 0
MW5 1 0 0 0 0 0
MW6 1 0 0 0 0 0

This design is asking the question: What is the mean expression level in each tissue-condition group? Then, via contrasts we can specify comparisons, like, what is the difference in expression between CoV2 and mock in Cornea

design(dds)<- meta_design
dds <- DESeq(dds)
using supplied model matrix
using pre-existing size factors
estimating dispersions
found already estimated dispersions, replacing these
gene-wise dispersion estimates
mean-dispersion relationship
final dispersion estimates
fitting model and testing
dds
class: DESeqDataSet 
dim: 14700 18 
metadata(1): version
assays(5): counts mu H cooks originalCounts
rownames(14700): A1BG A1BG-AS1 ... ZZEF1 ZZZ3
rowData names(38): baseMean baseVar ... deviance maxCooks
colnames(18): MW1 MW2 ... MW17 MW18
colData names(4): Tissue Condition sizeFactor replaceable
resultsNames(dds)
[1] "group.cornea_CoV2" "group.cornea_mock" "group.limbus_CoV2"
[4] "group.limbus_mock" "group.sclera_CoV2" "group.sclera_mock"
group.cornea_CoV2
group.cornea_mock
group.limbus_CoV2
group.limbus_mock
group.sclera_CoV2
group.sclera_mock

We need to specify the contrast, asking what is the difference between CoV2 and mock in cornea. There are multiple ways to establish a contrast, one of them is to supply the contrast argument a list of two character vectors, a list of 2 character vectors: the names of the fold changes for the numerator, and the names of the fold changes for the denominator.

res<- results(dds, contrast=list("group.cornea_CoV2", "group.cornea_mock")) # Let's obtain the results, as these contain the wald st
resLFC <- lfcShrink(dds, contrast=list("group.cornea_CoV2", "group.cornea_mock"), type="ashr")
using 'ashr' for LFC shrinkage. If used in published research, please cite:
    Stephens, M. (2016) False discovery rates: a new deal. Biostatistics, 18:2.
    https://doi.org/10.1093/biostatistics/kxw041
EnhancedVolcano(resLFC,
                lab = rownames(resLFC),
                x = 'log2FoldChange',
                y = 'pvalue')

Another way is to use a a numeric contrast vector with one element for each element in ‘resultsNames(object)’ (most general case).

resultsNames(dds)
[1] "group.cornea_CoV2" "group.cornea_mock" "group.limbus_CoV2"
[4] "group.limbus_mock" "group.sclera_CoV2" "group.sclera_mock"
group.cornea_CoV2
group.cornea_mock
group.limbus_CoV2
group.limbus_mock
group.sclera_CoV2
group.sclera_mock
resLFC_2 <- lfcShrink(dds, contrast=c(1,-1,0,0,0,0), type="ashr")
using 'ashr' for LFC shrinkage. If used in published research, please cite:
    Stephens, M. (2016) False discovery rates: a new deal. Biostatistics, 18:2.
    https://doi.org/10.1093/biostatistics/kxw041
identical(resLFC$log2FoldChange, resLFC_2$log2FoldChange)
[1] TRUE

We observe that the two approaches give us identical results.

For the next section (gene set enrichment analysis), let’s save the results from this model.

saveRDS(res, file="../results/res_Cornea_CoV2_vs_Mock.rds")

Exercises

  • Using the mean group model, how can we obtain the DEG between CoV2 and mock in limbus?
  • As a follow up to the previous question, can you try different ways to perform the contrast? For example, using a list or a numeric contrast vector?

Acknowledgements

This notebook used material adapted from the following sources:

Session info

sessionInfo()
R version 4.4.3 (2025-02-28)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 20.04.6 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale:
 [1] LC_CTYPE=en_CA.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_CA.UTF-8        LC_COLLATE=en_CA.UTF-8    
 [5] LC_MONETARY=en_CA.UTF-8    LC_MESSAGES=en_CA.UTF-8   
 [7] LC_PAPER=en_CA.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C       

time zone: America/New_York
tzcode source: system (glibc)

attached base packages:
[1] stats4    stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
 [1] knitr_1.48                  ashr_2.2-63                
 [3] ggbeeswarm_0.7.2            EnhancedVolcano_1.24.0     
 [5] ggrepel_0.9.6               ggplot2_3.5.1              
 [7] RColorBrewer_1.1-3          pheatmap_1.0.13            
 [9] hexbin_1.28.5               vsn_3.74.0                 
[11] DESeq2_1.46.0               SummarizedExperiment_1.36.0
[13] Biobase_2.66.0              MatrixGenerics_1.18.1      
[15] matrixStats_1.4.1           GenomicRanges_1.58.0       
[17] GenomeInfoDb_1.42.1         IRanges_2.40.0             
[19] S4Vectors_0.44.0            BiocGenerics_0.52.0        

loaded via a namespace (and not attached):
 [1] tidyselect_1.2.1        dplyr_1.1.4             vipor_0.4.7            
 [4] farver_2.1.2            fastmap_1.2.0           apeglm_1.28.0          
 [7] digest_0.6.37           mime_0.12               lifecycle_1.0.4        
[10] statmod_1.5.0           invgamma_1.2            magrittr_2.0.3         
[13] compiler_4.4.3          rlang_1.1.4             sass_0.4.9             
[16] tools_4.4.3             utf8_1.2.4              yaml_2.3.10            
[19] labeling_0.4.3          S4Arrays_1.6.0          DelayedArray_0.32.0    
[22] plyr_1.8.9              abind_1.4-8             BiocParallel_1.40.2    
[25] numDeriv_2016.8-1.1     withr_3.0.1             grid_4.4.3             
[28] preprocessCore_1.68.0   fansi_1.0.6             colorspace_2.1-1       
[31] MASS_7.3-64             scales_1.3.0            mvtnorm_1.3-3          
[34] bbmle_1.0.25.1          cli_3.6.3               rmarkdown_2.28         
[37] crayon_1.5.3            generics_0.1.3          httr_1.4.7             
[40] bdsmatrix_1.3-7         cachem_1.1.0            affy_1.84.0            
[43] zlibbioc_1.52.0         parallel_4.4.3          BiocManager_1.30.25    
[46] XVector_0.46.0          vctrs_0.6.5             Matrix_1.7-2           
[49] jsonlite_1.8.9          mixsqp_0.3-54           irlba_2.3.5.1          
[52] beeswarm_0.4.0          locfit_1.5-9.12         limma_3.62.2           
[55] jquerylib_0.1.4         affyio_1.76.0           glue_1.8.0             
[58] emdbook_1.3.14          codetools_0.2-19        gtable_0.3.6           
[61] UCSC.utils_1.2.0        munsell_0.5.1           tibble_3.2.1           
[64] pillar_1.9.0            htmltools_0.5.8.1       GenomeInfoDbData_1.2.13
[67] truncnorm_1.0-9         R6_2.5.1                evaluate_1.0.1         
[70] lattice_0.22-5          highr_0.11              SQUAREM_2021.1         
[73] bslib_0.8.0             Rcpp_1.0.13             coda_0.19-4.1          
[76] SparseArray_1.6.2       xfun_0.48               pkgconfig_2.0.3        
LS0tCnRpdGxlOiAiTWlDTTogMDEgLSBERVNlcTIiCmRhdGU6ICIyMDI1LTEwLTIzIgpvdXRwdXQ6CiAgaHRtbF9ub3RlYm9vazoKICAgIGRmX3ByaW50OiBwYWdlZAogICAgY29kZV9mb2xkaW5nOiBzaG93CiAgICB0b2M6IHllcwogICAgdG9jX2Zsb2F0OiAKICAgICAgY29sbGFwc2VkOiBmYWxzZQogICAgICBzbW9vdGhfc2Nyb2xsOiBmYWxzZQotLS0KCmBgYHtyLGVjaG89RkFMU0V9Cmh0bWx0b29sczo6aW1nKHNyYyA9IGtuaXRyOjppbWFnZV91cmkoImltYWdlcy9taWNtX2NvbG9yX2xvZ28ucG5nIiksIAogICAgICAgICAgICAgICBhbHQgPSAnbG9nbycsIAogICAgICAgICAgICAgICBzdHlsZSA9ICdwb3NpdGlvbjphYnNvbHV0ZTsgdG9wOjE1cHg7IHJpZ2h0OjA7IHBhZGRpbmc6MTBweDsgbWF4LXdpZHRoOjUwJTsnKQpgYGAKClRoaXMgc2Vzc2lvbiB3aWxsIGd1aWRlIHlvdSB0byB1c2UgREVTZXEyIHRvIGlkZW50aWZ5IGRpZmZlcmVudGlhbGx5IGV4cHJlc3NlZCBnZW5lcyBhbmQgYmVjb21lIGZhbWlsaWFyIHdpdGggZGVzaWduIG1hdHJpY2VzLiBUaGUgc2Vzc2lvbiB3aWxsIGFsc28gaW50cm9kdWNlIGNvbmNlcHRzIGVzc2VudGlhbCBmb3Igd29ya2luZyB3aXRoIHJlYWwtd29ybGQgZGF0YSwgc3VjaCBhcyBpbmNvcnBvcmF0aW5nIGNvdmFyaWF0ZXMgYW5kIGludGVyYWN0aW9uIHRlcm1zIGludG8gbW9kZWxzLgoKVGhlIG9iamVjdGl2ZXMgb2YgdGhpcyBzZXNzaW9uIGFyZToKCiogT2J0YWluIGZhbWlsaWFyaXR5IHdpdGggdGhlIERFU2VxMiB3b3JrZmxvdwoqIERlc2lnbiBhbmQgY29uZmlndXJlIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIG1vZGVscyBhcHByb3ByaWF0ZWx5IGZvciBSTkEtc2VxIGRhdGEKCioqTm90ZSoqOiBBbGwgdGhlIGNvZGUgaW4gdGhpcyB0dXRvcmlhbCB3aWxsIGJlIHBlcmZvcm1lZCBpbiBSIChvciBSc3R1ZGlvKS4gCgojIERhdGFzZXQKClRoaXMgd29ya3Nob3Agd2lsbCB1c2UgcHJvY2Vzc2VkIFJOQS1zZXEgY291bnRzIGZyb20gdGhpcyBbc3R1ZHldKGh0dHBzOi8vZG9pLm9yZy8xMC4xMDE2L2ouc3RlbS4yMDIxLjA0LjAyOCksIHdoZXJlIHRoZXkgc3R1ZGllZCB0aGUgdHJhbnNjcmlwdGlvbmFsIHJlc3BvbnNlIHRvIFNBUlMtQ29WLTIgaW5mZWN0aW9uIGluIHRocmVlIHRpc3N1ZXMgKGNvcm5lYSwgbGltYnVzLCBzY2xlcmEpLiBUaGUgZGF0YXNldCBjb25zaXN0cyBvZiAxOCBzYW1wbGVzLCB3aXRoIHRocmVlIHJlcGxpY2F0ZXMgcGVyIHRpc3N1ZSBhbmQgY29uZGl0aW9uIChtb2NrIHZzIENvVi0yKS4KClRoZSBkYXRhIGNhbiBiZSBmb3VuZCBoZXJlOgpgYGB7YmFzaH0KbHMgLi4vZGF0YS8KYGBgCgojIERFc2VxMiB3b3JrZmxvdwoKIyMgTG9hZCBsaWJyYXJpZXMKCmBgYHtyLCBtZXNzYWdlPUZ9CmxpYnJhcnkoREVTZXEyKQpsaWJyYXJ5KHZzbikgIyBVc2VkIGZvciBzdGFuZGFyZCBkZXZpYXRpb24gdnMgbWVhbiBwbG90cwpsaWJyYXJ5KGhleGJpbikgIyBVc2VkIGZvciBzdGFuZGFyZCBkZXZpYXRpb24gdnMgbWVhbiBwbG90cwpsaWJyYXJ5KHBoZWF0bWFwKSAKbGlicmFyeShSQ29sb3JCcmV3ZXIpCmxpYnJhcnkoRW5oYW5jZWRWb2xjYW5vKQpsaWJyYXJ5KGdncGxvdDIpCmxpYnJhcnkoZ2diZWVzd2FybSkKbGlicmFyeShhc2hyKSAjIFVzZWQgZm9yIGxvZy1mb2xkIHNocmlua2FnZQpsaWJyYXJ5KGtuaXRyKSAjIFVzZWQgZm9yIGZvcm1hdHRpbmcgb3V0cHV0CmBgYAoKIyMgTG9hZCB0aGUgZGF0YQoKTGV0J3MgbG9hZCB0aGUgZGF0YXNldCB0aGF0IGNvbnNpc3RzIG9mIGEgcmF3IGNvdW50IG1hdHJpeCBhbmQgYSBtZXRhZGF0YS4KYGBge3J9CnJhd19jb3VudHM8LSByZWFkLnRhYmxlKCIuLi9kYXRhL0dTRTE2NDA3M19FeWVfY291bnRfbWF0cml4LnRzdiIsIGhlYWRlcj1ULCByb3cubmFtZXM9MSkKbWV0YTwtIHJlYWQudGFibGUoIi4uL2RhdGEvR1NFMTY0MDczX0V5ZV9jb3VudF9tZXRhLnRzdiIsIHJvdy5uYW1lcz0xKQprYWJsZShoZWFkKHJhd19jb3VudHNbLDE6NV0pKQprYWJsZShoZWFkKG1ldGEpKQpgYGAKIyMgQ3JlYXRlIERFU2VxMiBvYmplY3QKClRoZSBmaXJzdCBzdGVwIGlzIHRvIGNyZWF0ZSBhIGRlc2lnbiBtYXRyaXguIExldCdzIHN0YXJ0IGJ5IGNyZWF0aW5nIGEgdmVyeSBzaW1wbGUgZGVzaWduIG1hdHJpeC4KYGBge3J9CiMgQ29udmVydGluZyBUaXNzdWUgaW50byBhIGZhY3RvcgptZXRhJFRpc3N1ZTwtIGZhY3RvcihtZXRhJFRpc3N1ZSkKIyBBZGp1c3RpbmcgbGV2ZWxzIGZvciBDb25kaXRpb24sIHNvIHRoYXQgbW9jayBpcyB0aGUgcmVmZXJlbmNlCm1ldGEkQ29uZGl0aW9uPC0gZmFjdG9yKG1ldGEkQ29uZGl0aW9uLCBsZXZlbHM9YygibW9jayIsICJDb1YyIikpCm1ldGFfZGVzaWduPC0gbW9kZWwubWF0cml4KCB+IENvbmRpdGlvbiwgZGF0YT1tZXRhKQprYWJsZShoZWFkKG1ldGFfZGVzaWduKSkKYGBgClRoaXMgZGVzaWduIG1hdHJpeCBpcyBhbnN3ZXJpbmcgdGhlIHF1ZXN0aW9uOiBXaGF0IGFyZSB0aGUgb3ZlcmFsbCBkaWZmZXJlbmNlcyBiZXR3ZWVuIE1vY2sgYW5kIENvVjIsIHdpdGhvdXQgY29uc2lkZXJpbmcgdGhlIHRpc3N1ZXM/CgpOb3csIGxldCdzIGNyZWF0ZSBhIERFU2VxMiBvYmplY3QuCmBgYHtyfQpkZHMgPC0gREVTZXFEYXRhU2V0RnJvbU1hdHJpeChjb3VudERhdGEgPSByYXdfY291bnRzLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb2xEYXRhID0gbWV0YSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZGVzaWduPSBtZXRhX2Rlc2lnbikKZGRzCmBgYAojIyBQcmUtZmlsdGVyaW5nCgpSZW1vdmluZyBsb3cgZXhwcmVzc2VkIGNvdW50cyBpcyBnZW5lcmFsbHkgcmVjb21tZW5kZWQgZm9yIG1lbW9yeSBhbmQgc3BlZWQgcmVhc29ucy4KYGBge3J9CnNtYWxsZXN0R3JvdXBTaXplIDwtIDMKa2VlcCA8LSByb3dTdW1zKGNvdW50cyhkZHMpID49IDEwKSA+PSBzbWFsbGVzdEdyb3VwU2l6ZQpkZHMgPC0gZGRzW2tlZXAsXQpkZHMKYGBgCgojIyBRdWFsaXR5IGNvbnRyb2wKCiMjIyBWU1QKV2UgY2FuIG9idGFpbiBhIHZhcmlhbmNlIHN0YWJpbGl6aW5nIHRyYW5zZm9ybWF0aW9uCmBgYHtyLCBmaWcud2lkdGg9NiwgZmlnLmhlaWdodD02fQojIE9idGFpbiB0aGUgdmFyaWFuY2Ugc3RhYmlsaXppbmcgdHJhbnNmb3JtYXRpb24KdnNkIDwtIHZzdChkZHMpCiMgTGV0J3MgY29tcGFyZSBpdCB0byB0aGUgbm9ybWFsIHRyYW5zZm9ybWF0aW9uCm50ZCA8LSBub3JtVHJhbnNmb3JtKGRkcykKbWVhblNkUGxvdChhc3NheShudGQpLCByYW5rcz1GKSAjIE5vcm1hbAptZWFuU2RQbG90KGFzc2F5KHZzZCksIHJhbmtzPUYpICMgVlNUCmBgYApXZSBvYnNlcnZlIHRoYXQgdGhlIHZzdCByZW1vdmVzIHRoZSBkZXBlbmRlbmNlIG9mIHRoZSB2YXJpYW5jZSBvbiB0aGUgbWVhbiwgd2hpY2ggaXMgZXZpZGVudCBmb3IgdGhlIGxvdyBleHByZXNzZWQgZ2VuZXMuCgojIyMgSGVhdG1hcCBvZiB0aGUgc2FtcGxlLXRvLXNhbXBsZSBkaXN0YW5jZXMKV2UgY2FuIG9idGFpbiBkaXN0YW5jZXMgYmV0d2VlbiBzYW1wbGVzIGFuZCB0aGVuIHBsb3QgdGhlbSBhcyBhIGhlYXRtYXAuIApgYGB7ciwgZmlnLndpZHRoPTYsIGZpZy5oZWlnaHQ9Nn0Kc2FtcGxlRGlzdHMgPC0gZGlzdCh0KGFzc2F5KHZzZCkpKQpzYW1wbGVEaXN0TWF0cml4IDwtIGFzLm1hdHJpeChzYW1wbGVEaXN0cykKcm93bmFtZXMoc2FtcGxlRGlzdE1hdHJpeCkgPC0gcGFzdGUodnNkJFRpc3N1ZSwgdnNkJENvbmRpdGlvbiwgc2VwPSItIikKY29sbmFtZXMoc2FtcGxlRGlzdE1hdHJpeCkgPC0gTlVMTApjb2xvcnMgPC0gY29sb3JSYW1wUGFsZXR0ZSggcmV2KGJyZXdlci5wYWwoOSwgIkJsdWVzIikpICkoMjU1KQpwaGVhdG1hcChzYW1wbGVEaXN0TWF0cml4LAogICAgICAgICBjbHVzdGVyaW5nX2Rpc3RhbmNlX3Jvd3M9c2FtcGxlRGlzdHMsCiAgICAgICAgIGNsdXN0ZXJpbmdfZGlzdGFuY2VfY29scz1zYW1wbGVEaXN0cywKICAgICAgICAgY29sPWNvbG9ycykKYGBgCldlIG9ic2VydmUgdGhhdCBpbiBnZW5lcmFsLCBzYW1wbGVzIGZyb20gdGhlIHNhbWUgdGlzc3VlIGNsdXN0ZXIgdG9nZXRoZXIsIGFzIGV4cGVjdGVkLgoKIyMjIFByaW5jaXBhbCBjb21wb25lbnQgb2YgdGhlIHNhbXBsZXMKV2UgY2FuIGFsc28gcGVyZm9ybSBhIFBDQSBhbmFseXNpcyBvZiB0aGUgVlNUIGNvdW50cy4KYGBge3IsIGZpZy53aWR0aD02LCBmaWcuaGVpZ2h0PTZ9CnBsb3RQQ0EodnNkLCBpbnRncm91cD1jKCJUaXNzdWUiLCAiQ29uZGl0aW9uIikpICsKICAgIGdlb21fcG9pbnQoc2l6ZT0wLjUpCmBgYApBZ2Fpbiwgd2Ugb2JzZXJ2ZSB0aGF0IHRoZSBmaXJzdCBQQ0EgaXMgbW9zdGx5IGFzc29jaWF0ZWQgd2l0aCB0aXNzdWUsIGFsdGhvdWdoIHNhbXBsZXMgY2FuIGFsc28gc3ViY2x1c3RlciBiZXR3ZWVuIHRoZSBtb2NrIGFuZCBDb1YyIGNvbmRpdGlvbnMuCgojIyBEaWZmZXJlbnRpYWwgZXhwcmVzc2lvbgpUaGUgc3RhbmRhcmQgREUgaXMgcGVyZm9ybWVkIGJ5IHRoZSBERVNlcSBmdW5jdGlvbi4KYGBge3J9CmRkcyA8LSBERVNlcShkZHMpCmRkcwpgYGAKYGBge3J9CnJlc3VsdHNOYW1lcyhkZHMpCnJlcyA8LSByZXN1bHRzKGRkcywgbmFtZT0iQ29uZGl0aW9uQ29WMiIpICMgVGhpcyBmdW5jdGlvbiBwcm92aWRlcyB0aGUgbG9mMkZvbGQgQ2hhbmdlcyBhbmQgcC12YWx1ZXMgZm9yIGEgZ2l2ZW4gY29lZmZpY2llbnQgbmFtZQpyZXMKYGBgClRoZW4sIGEgcmVzdWx0cyB0YWJsZSBjYW4gYmUgZ2VuZXJhdGVkLiBJbiB0aGlzIGNhc2UsIHdlIHByb3ZpZGUgYSB2YWx1ZSBpbiB0aGUgJ25hbWUnIGFyZ3VtZW50IGluIHRoZSByZXN1bHRzIGZ1bmN0aW9uLCB3aGljaCBjb3JyZXNwb25kZHMgdG8gdGhlIENvVjIgZWZmZWN0IGVuY29kZWQgaW4gb3VyIGRlc2lnbiBtYXRyaXguIFdlIHdpbGwgYWxzbyBleHBsb3JlIG90aGVyIHdheXMgdG8gYWNjZXNzIHJlc3VsdHMgaW4gdGhlIGZvbGxvdyB1cCBzZWN0aW9ucy4KCiMjIExvZy1Gb2xkIHNocmlua2FnZQpTaHJpbmthZ2Ugb2YgdGhlIGxvZy1mb2xkIGNoYW5nZXMgZXN0aW1hdGVzIGlzIHJlY29tbWVuZGVkIGZvciB2aXN1YWxpemF0aW9uIGFuZCByYW5raW5nIG9mIGdlbmVzLiAKYGBge3J9CnJlc0xGQyA8LSBsZmNTaHJpbmsoZGRzLCBjb2VmPSJDb25kaXRpb25Db1YyIiwgdHlwZT0iYXBlZ2xtIikKYGBgClRoZSBsZmNTaHJpbmsgaGFzIHZhcmlvdXMgc2hyaW5rYWdlIG1ldGhvZHMsIGhlcmUgd2Ugd2lsbCB1c2UgYXBlZ2xtLiBBZ2Fpbiwgd2UgbG9vayBmb3IgdGhlIENvVjIgZWZmZWN0IG1vZGVsIGJ5IHNldHRpbmcgdXAgdGhlIGNvZWYgYXJndW1lbnQgaW4gdGhlIGxmY1NocmluayBmdW5jdGlvbi4KCiMjIyBNQSBwbG90cwpMZXQncyBjb21wYXJlIHRoZSBNQSBwbG90cyBiZXR3ZWVuIHRoZSBzaHJ1bmtlbiBGQyBhbmQgdGhlIHJlZ3VsYXIgRkMuCmBgYHtyLCBmaWcud2lkdGg9NiwgZmlnLmhlaWdodD02fQpwbG90TUEocmVzKQpwbG90TUEocmVzTEZDKQpgYGAKWW91IGNhbiBzZWUgdGhhdCB0aGUgc2hydW5rZW4gbG9nMiBmb2xkIGNoYW5nZXMgcmVtb3ZlIHRoZSBub2lzZSBhc3NvY2lhdGVkIHdpdGggbG9nMiBmb2xkIGNoYW5nZXMgZnJvbSBsb3cgY291bnQgZ2VuZXMuCgojIyMgVm9sY2FubyBwbG90CmBgYHtyLCBmaWcud2lkdGg9NiwgZmlnLmhlaWdodD02fQpFbmhhbmNlZFZvbGNhbm8ocmVzTEZDLAogICAgICAgICAgICAgICAgbGFiID0gcm93bmFtZXMocmVzTEZDKSwKICAgICAgICAgICAgICAgIHggPSAnbG9nMkZvbGRDaGFuZ2UnLAogICAgICAgICAgICAgICAgeSA9ICdwdmFsdWUnKQpgYGAKV2UgY2FuIGV4cGxvcmUgYWxzbyB0aGUgcmVzdWx0cyB1c2luZyBhIFZvbGNhbm8gUGxvdC4KCiMjIyBQbG90IGNvdW50cyAKYGBge3IsIGZpZy53aWR0aD02LCBmaWcuaGVpZ2h0PTZ9CmdlbmVDb3VudHMgPC0gcGxvdENvdW50cyhkZHMsIGdlbmUgPSAiU09EMiIsIGludGdyb3VwID0gYygiQ29uZGl0aW9uIiwiVGlzc3VlIikscmV0dXJuRGF0YSA9IFQpCmdncGxvdChnZW5lQ291bnRzLCBhZXMoeCA9IENvbmRpdGlvbiwgeSA9IGxvZzIoY291bnQpLCBjb2xvciA9IFRpc3N1ZSkpICsgIGdlb21fYmVlc3dhcm0oY2V4ID0gMykKYGBgCldlIGNhbiBhbHNvIHBsb3QgdGhlIG5vcm1hbGl6ZWQgY291bnQgdmFsdWVzIGZvciBhIGdpdmVuIGdlbmUuCgojIyMgSGVhdG1hcHMKYGBge3IsIGZpZy53aWR0aD02LCBmaWcuaGVpZ2h0PTZ9CnNpZ19nZW5lczwtIHJvd25hbWVzKHJlc0xGQ1t3aXRoKHJlc0xGQywgYWJzKGxvZzJGb2xkQ2hhbmdlID4gMSkgJiBwYWRqIDwgMC4wNSksXSkKY29sb3JzIDwtIGNvbG9yUmFtcFBhbGV0dGUoYygiZGFya2JsdWUiLCJ3aGl0ZSIsImRhcmtyZWQiKSkoMjU1KQpwaGVhdG1hcChhc3NheSh2c2Rbc2lnX2dlbmVzLF0pLCBhbm5vdGF0aW9uX2NvbD1tZXRhLGNvbD1jb2xvcnMsIHNjYWxlPSJyb3ciKQpgYGAKRmluYWxseSwgd2UgY2FuIGdlbmVyYXRlIGEgaGVhdG1hcCBmb3IgYSBzdWJzZXQgb2YgZ2VuZXMgd2l0aCBzaWduaWZpY2FudCBERSBjaGFuZ2VzIChwYWRqIDwgMC4wNSBhbmQgbG9nMkZvbGRDaGFuZ2UgPjEpLgoKIyMgVGlzc3VlLXNwZWNpZmljIGVmZmVjdHMgClNvIGZhciwgd2UgaGF2ZW4ndCBjb25zaWRlcmVkIHRoZSBkaWZmZXJlbnQgdGlzc3VlcyBpbnRvIG91ciBtb2RlbC4gT25lIHdheSB0byBzdHVkeSB0aGUgdGlzc3VlLXNwZWNpZmljIGVmZmVjdHMgaXMgdGhyb3VnaCBhIG1lYW4gZ3JvdXAgbW9kZWwuCmBgYHtyLCBmaWcud2lkdGg9NywgZmlnLmhlaWdodD03fQptZXRhJGdyb3VwPC0gZmFjdG9yKHBhc3RlMCgiLiIsIG1ldGEkVGlzc3VlLCAiXyIsIG1ldGEkQ29uZGl0aW9uKSkKbWV0YV9kZXNpZ248LSBtb2RlbC5tYXRyaXgoIH4gMCArIGdyb3VwLCBkYXRhPW1ldGEpICMgV2UgcmVtb3ZlIHRoZSBpbnRlcmNlcHQgaW4gdGhpcyBtb2RlbAprYWJsZShoZWFkKG1ldGFfZGVzaWduKSkKYGBgClRoaXMgZGVzaWduIGlzIGFza2luZyB0aGUgcXVlc3Rpb246IFdoYXQgaXMgdGhlIG1lYW4gZXhwcmVzc2lvbiBsZXZlbCBpbiBlYWNoIHRpc3N1ZS1jb25kaXRpb24gZ3JvdXA/ClRoZW4sIHZpYSBjb250cmFzdHMgd2UgY2FuIHNwZWNpZnkgY29tcGFyaXNvbnMsIGxpa2UsIHdoYXQgaXMgdGhlIGRpZmZlcmVuY2UgaW4gZXhwcmVzc2lvbiBiZXR3ZWVuIENvVjIgYW5kIG1vY2sgaW4gQ29ybmVhCmBgYHtyfQpkZXNpZ24oZGRzKTwtIG1ldGFfZGVzaWduCmRkcyA8LSBERVNlcShkZHMpCmRkcwpyZXN1bHRzTmFtZXMoZGRzKQpgYGAKV2UgbmVlZCB0byBzcGVjaWZ5IHRoZSBjb250cmFzdCwgYXNraW5nIHdoYXQgaXMgdGhlIGRpZmZlcmVuY2UgYmV0d2VlbiBDb1YyIGFuZCBtb2NrIGluIGNvcm5lYS4gVGhlcmUgYXJlIG11bHRpcGxlIHdheXMgdG8gZXN0YWJsaXNoIGEgY29udHJhc3QsIG9uZSBvZiB0aGVtIGlzIHRvIHN1cHBseSB0aGUgY29udHJhc3QgYXJndW1lbnQgYSBsaXN0IG9mIHR3byBjaGFyYWN0ZXIgdmVjdG9ycywgYSBsaXN0IG9mIDIgY2hhcmFjdGVyIHZlY3RvcnM6IHRoZSBuYW1lcyBvZiB0aGUgZm9sZCBjaGFuZ2VzIGZvciB0aGUgbnVtZXJhdG9yLCBhbmQgdGhlIG5hbWVzIG9mIHRoZSBmb2xkIGNoYW5nZXMgZm9yIHRoZSBkZW5vbWluYXRvci4KYGBge3J9CnJlczwtIHJlc3VsdHMoZGRzLCBjb250cmFzdD1saXN0KCJncm91cC5jb3JuZWFfQ29WMiIsICJncm91cC5jb3JuZWFfbW9jayIpKSAjIExldCdzIG9idGFpbiB0aGUgcmVzdWx0cywgYXMgdGhlc2UgY29udGFpbiB0aGUgd2FsZCBzdApyZXNMRkMgPC0gbGZjU2hyaW5rKGRkcywgY29udHJhc3Q9bGlzdCgiZ3JvdXAuY29ybmVhX0NvVjIiLCAiZ3JvdXAuY29ybmVhX21vY2siKSwgdHlwZT0iYXNociIpCkVuaGFuY2VkVm9sY2FubyhyZXNMRkMsCiAgICAgICAgICAgICAgICBsYWIgPSByb3duYW1lcyhyZXNMRkMpLAogICAgICAgICAgICAgICAgeCA9ICdsb2cyRm9sZENoYW5nZScsCiAgICAgICAgICAgICAgICB5ID0gJ3B2YWx1ZScpCmBgYApBbm90aGVyIHdheSBpcyB0byB1c2UgYSBhIG51bWVyaWMgY29udHJhc3QgdmVjdG9yIHdpdGggb25lIGVsZW1lbnQgZm9yIGVhY2ggZWxlbWVudCBpbiDigJhyZXN1bHRzTmFtZXMob2JqZWN0KeKAmSAobW9zdCBnZW5lcmFsIGNhc2UpLgpgYGB7cn0KcmVzdWx0c05hbWVzKGRkcykKcmVzTEZDXzIgPC0gbGZjU2hyaW5rKGRkcywgY29udHJhc3Q9YygxLC0xLDAsMCwwLDApLCB0eXBlPSJhc2hyIikKaWRlbnRpY2FsKHJlc0xGQyRsb2cyRm9sZENoYW5nZSwgcmVzTEZDXzIkbG9nMkZvbGRDaGFuZ2UpCmBgYApXZSBvYnNlcnZlIHRoYXQgdGhlIHR3byBhcHByb2FjaGVzIGdpdmUgdXMgaWRlbnRpY2FsIHJlc3VsdHMuCgpGb3IgdGhlIG5leHQgc2VjdGlvbiAoZ2VuZSBzZXQgZW5yaWNobWVudCBhbmFseXNpcyksIGxldCdzIHNhdmUgdGhlIHJlc3VsdHMgZnJvbSB0aGlzIG1vZGVsLgpgYGB7ciwgZXZhbD1GfQpzYXZlUkRTKHJlcywgZmlsZT0iLi4vcmVzdWx0cy9yZXNfQ29ybmVhX0NvVjJfdnNfTW9jay5yZHMiKQpgYGAKIyBFeGVyY2lzZXMKCiogVXNpbmcgdGhlIG1lYW4gZ3JvdXAgbW9kZWwsIGhvdyBjYW4gd2Ugb2J0YWluIHRoZSBERUcgYmV0d2VlbiBDb1YyIGFuZCBtb2NrIGluIGxpbWJ1cz8KKiBBcyBhIGZvbGxvdyB1cCB0byB0aGUgcHJldmlvdXMgcXVlc3Rpb24sIGNhbiB5b3UgdHJ5IGRpZmZlcmVudCB3YXlzIHRvIHBlcmZvcm0gdGhlIGNvbnRyYXN0PyBGb3IgZXhhbXBsZSwgdXNpbmcgYSBsaXN0IG9yIGEgbnVtZXJpYyBjb250cmFzdCB2ZWN0b3I/CgojIEFja25vd2xlZGdlbWVudHMKVGhpcyBub3RlYm9vayB1c2VkIG1hdGVyaWFsIGFkYXB0ZWQgZnJvbSB0aGUgZm9sbG93aW5nIHNvdXJjZXM6CgoqIFtERVNlcTIgdmlnbmV0dGVdKGh0dHBzOi8vd3d3LmJpb2NvbmR1Y3Rvci5vcmcvcGFja2FnZXMvcmVsZWFzZS9iaW9jL3ZpZ25ldHRlcy9ERVNlcTIvaW5zdC9kb2MvREVTZXEyLmh0bWwpCiogW1JOQS1zZXEgd29ya2Zsb3ddKGh0dHBzOi8vbWFzdGVyLmJpb2NvbmR1Y3Rvci5vcmcvcGFja2FnZXMvcmVsZWFzZS93b3JrZmxvd3MvdmlnbmV0dGVzL3JuYXNlcUdlbmUvaW5zdC9kb2Mvcm5hc2VxR2VuZS5odG1sKQoKIyBTZXNzaW9uIGluZm8KYGBge3J9CnNlc3Npb25JbmZvKCkKYGBgCg==