As described in the null models section, modularity and nestedness values lack statistical significance if they are not compared with an appropiate null model. Further, an appropiate ensamble of those random networks need to be created in order to analyze those networks. Altought the user can use BiMat null models for designing his own statistical tests, BiMat comes already with an StatisticalTest that helps the user to perform such tests. BiMat comes with statistical test for both modularity and nestedness. These tests can be executed directly on an instance of the Bipartite class (where the property statistics is an instance of the StatisticalTest class, or directly in the adjacency matrix.

## Example: Working directly on a Bipartite instance

This example will show the usual flow to evaluate the significance of modularity and nestedness on an specific matrix. We will use one of the phage bacteria matrices (Zinno 2010) collected on Flores et al 2012.

%loading and creating data
bp = Bipartite(phage_bacteria_matrices.matrices{38}); %Matrices 38 corresponds to Zinno 2010
%A quick visual of how the data looks like:
pformat = PlotFormat();
pformat.use_labels = false;
pformat.back_color = [110,110,20]/255;
pformat.cell_color = 'white';
pformat.use_isocline_module = false;
bp.plotter.SetPlotFormat(pformat);

font_size = 16;

figure(1); set(gcf,'Position', [38,64,1290,435]);
subplot(1,3,1); bp.plotter.PlotMatrix; title('Original','FontSize',font_size);
subplot(1,3,2); bp.plotter.PlotNestedMatrix; title('Nested','FontSize',font_size);xlabel('Zinno 2010','FontSize',font_size+6);
subplot(1,3,3); bp.plotter.PlotModularMatrix; title('Modular','FontSize',font_size);


By looking at the last plots, we can infer that this matrix have a strong community structure pattern, while nestedness is not apparent. In order to confirm these observations, we can finally perform nestedness and modularity tests on this matrix (using default values):

bp.statistics.TestCommunityStructure();

Evaluating Modularity in random matrices: 1 - 10
Evaluating Modularity in random matrices: 11 - 20
Evaluating Modularity in random matrices: 21 - 30
Evaluating Modularity in random matrices: 31 - 40
Evaluating Modularity in random matrices: 41 - 50
Evaluating Modularity in random matrices: 51 - 60
Evaluating Modularity in random matrices: 61 - 70
Evaluating Modularity in random matrices: 71 - 80
Evaluating Modularity in random matrices: 81 - 90
Evaluating Modularity in random matrices: 91 - 100

bp.statistics.TestNestedness();

Evaluating Nestedness in random matrices: 1 - 10
Evaluating Nestedness in random matrices: 11 - 20
Evaluating Nestedness in random matrices: 21 - 30
Evaluating Nestedness in random matrices: 31 - 40
Evaluating Nestedness in random matrices: 41 - 50
Evaluating Nestedness in random matrices: 51 - 60
Evaluating Nestedness in random matrices: 61 - 70
Evaluating Nestedness in random matrices: 71 - 80
Evaluating Nestedness in random matrices: 81 - 90
Evaluating Nestedness in random matrices: 91 - 100


As we can see, BiMat shows the status of the current evaluation, which in this case was performed using 100 random matrices (the default value). In order to print the results, the user only need to call the Print method:

bp.statistics.Print;

Modularity
Null model:    	       NullModels.EQUIPROBABLE
Replicates:    	                           100
Qb value:      	                        0.6781
mean:      	                        0.5654
std:       	                        0.0257
z-score:   	                        4.3862
t-score:   	                       43.8623
percentil: 	                      100.0000
Qr value:      	                        0.6735
mean:      	                        0.4327
std:       	                        0.0762
z-score:   	                        3.1605
t-score:   	                       31.6053
percentil: 	                       99.0000
Nestedness
Used algorithm:	                NestednessNODF
Null model:    	       NullModels.EQUIPROBABLE
Replicates:    	                           100
Nestedness value:	                        0.1247
mean:      	                        0.0941
std:       	                        0.0127
z-score:   	                        2.4022
t-score:   	                       24.0221
percentil: 	                       99.0000


The output just shows the configuration of each evaluation, which in this case was performed using the default values, which may not be a strong statistical test. Let's repeat the experiment using a larger number of random matrices and a different null model:

%The null model will be the AVERAGE one with 1000 replicates:
%in the octave version you should replace @NullModels.AVERAGE by 'NullModels.AVERAGE'
bp.statistics.DoNulls(1000, @NullModels.AVERAGE);
bp.statistics.TestCommunityStructure();
bp.statistics.TestNestedness();

Evaluating Modularity in random matrices: 1 - 100
Evaluating Modularity in random matrices: 101 - 200
Evaluating Modularity in random matrices: 201 - 300
Evaluating Modularity in random matrices: 301 - 400
Evaluating Modularity in random matrices: 401 - 500
Evaluating Modularity in random matrices: 501 - 600
Evaluating Modularity in random matrices: 601 - 700
Evaluating Modularity in random matrices: 701 - 800
Evaluating Modularity in random matrices: 801 - 900
Evaluating Modularity in random matrices: 901 - 1000
Evaluating Nestedness in random matrices: 1 - 100
Evaluating Nestedness in random matrices: 101 - 200
Evaluating Nestedness in random matrices: 201 - 300
Evaluating Nestedness in random matrices: 301 - 400
Evaluating Nestedness in random matrices: 401 - 500
Evaluating Nestedness in random matrices: 501 - 600
Evaluating Nestedness in random matrices: 601 - 700
Evaluating Nestedness in random matrices: 701 - 800
Evaluating Nestedness in random matrices: 801 - 900
Evaluating Nestedness in random matrices: 901 - 1000


And finally printing the results:

bp.statistics.Print();

Modularity
Null model:    	            NullModels.AVERAGE
Replicates:    	                          1000
Qb value:      	                        0.6781
mean:      	                        0.5494
std:       	                        0.0487
z-score:   	                        2.6426
t-score:   	                       83.5656
percentil: 	                       99.2000
Qr value:      	                        0.6735
mean:      	                        0.4142
std:       	                        0.1104
z-score:   	                        2.3488
t-score:   	                       74.2749
percentil: 	                       97.8000
Nestedness
Used algorithm:	                NestednessNODF
Null model:    	            NullModels.AVERAGE
Replicates:    	                          1000
Nestedness value:	                        0.1247
mean:      	                        0.1072
std:       	                        0.0265
z-score:   	                        0.6583
t-score:   	                       20.8179
percentil: 	                       75.4000


## Example: Working using the functional approach

Sometimes the user just want to do a quick analysis, without the extra functionalities that working with a Bipartite instance provides. For those cases, the user can just execute the functional calls of the StatisticalTest class on a specific matrix. Using the previous matrix as an example:

matrix = phage_bacteria_matrices.matrices{38};
%in the octave version you should replace the handle functions by string representations

Evaluating Modularity in random matrices: 1 - 20
Evaluating Modularity in random matrices: 21 - 40
Evaluating Modularity in random matrices: 41 - 60
Evaluating Modularity in random matrices: 61 - 80
Evaluating Modularity in random matrices: 81 - 100
Evaluating Modularity in random matrices: 101 - 120
Evaluating Modularity in random matrices: 121 - 140
Evaluating Modularity in random matrices: 141 - 160
Evaluating Modularity in random matrices: 161 - 180
Evaluating Modularity in random matrices: 181 - 200
Modularity
Null model:    	            NullModels.AVERAGE
Replicates:    	                           200
Qb value:      	                        0.6822
mean:      	                        0.5529
std:       	                        0.0530
z-score:   	                        2.4424
t-score:   	                       34.5410
percentil: 	                       99.0000
Qr value:      	                        0.7143
mean:      	                        0.4949
std:       	                        0.1069
z-score:   	                        2.0517
t-score:   	                       29.0157
percentil: 	                       97.0000

stest_nest = StatisticalTest.TEST_NESTEDNESS(matrix,200,@NullModels.EQUIPROBABLE,@NestednessNODF);

Evaluating Nestedness in random matrices: 1 - 20
Evaluating Nestedness in random matrices: 21 - 40
Evaluating Nestedness in random matrices: 41 - 60
Evaluating Nestedness in random matrices: 61 - 80
Evaluating Nestedness in random matrices: 81 - 100
Evaluating Nestedness in random matrices: 101 - 120
Evaluating Nestedness in random matrices: 121 - 140
Evaluating Nestedness in random matrices: 141 - 160
Evaluating Nestedness in random matrices: 161 - 180
Evaluating Nestedness in random matrices: 181 - 200
Nestedness
Used algorithm:	                NestednessNODF
Null model:    	       NullModels.EQUIPROBABLE
Replicates:    	                           200
Nestedness value:	                        0.1247
mean:      	                        0.0944
std:       	                        0.0137
z-score:   	                        2.2088
t-score:   	                       31.2370
percentil: 	                       98.0000