Network-level Results

Overview

All network level tests use nla.net.result.NetworkTestResult as the result object. This object uses the properties of the test to name the results.

class net.result.NetworkTestResult

Network level test results Class to store all relevant results for a network level test. Each test will create an instance of this to store results

Parameters:

test_options – Options and inputs for tests to be run (also called input_struct)
number_of_networks – The number of networks in the network atlas
test_name – The name of the network test run
test_display_name – The name of the network test for display
test_specific_statistics – The statistics that a specific test produces
ranking_statistic – The statistic used for calculating p-values

test_name: Name of the network test run

test_display_name: The name of the network test for display

test_options: Options and inputs for tests to be run (also called input_struct)

ranking_statistic: The statistic used for calculating p-values

within_network_pair: Results of the within network pair test. Single sample p-values (except \(\chi^2\)and hypergeometric tests). “legacy_” results use the individual test p-values to rank and determine the final p-value.

full_connectome: Results of the full_connectome test. Two sample p-values.

permutation_results: Results of each permutation. Statistics and p-values. Note: The p-values are for each individual permutation test, not the overall p-value.

merge(other_objects)

Used to merge multiple results together into one

Parameters:: other_objects – The other result objects to merge into this result object

concatenateResult(other_object)

Concatenate results together. This is used to preserve the individual permutation results.

Parameters:: other_object – The object to append to the end of the current result

output(edge_test_options, updated_test_options, network_atlas, edge_test_result, flags)

Outputs data to be plotted using nla.net.result.plot.NetworkTestPlot

Parameters:

edge_test_options – The test_options used to instantiate the class. Contains the functional connectivity and network atlas among other options
updated_test_options – The network test options. These can also include the options for plotting.
network_atlas – The network atlas
edge_test_result – Results of the edge level test.
flags – More options that are used after the tests have run. One of them is which test method to plot.

createResultsStorage(test_options, number_of_networks, test_specific_statistics)

Creates the objects to hold results. Uses statistic names from test objects.

Parameters:

test_options – The test options
number_of_networks – The number of networks. Used to determine the size of the TriMatrix result. A property of the Network Atlas
test_specific_statistics – The statistics used in each test. A property of each test.

static editableOptions()

Static method to return options that can be adjusted afterwards.

Returns:: Options. Defaults to behavior_count, prob_max (The threshold for p-values), d_max (The threshold for Cohen’s D values)

static getPValueNames(test_method, test_name)

Static method to determine prefixes of p-values for test results

Parameters:

test_method – No permutations, full connectome, or within network pair
test_name – The name of the test run

Returns:

The full name of the p-value. (example: “single_sammple_p_value”)

Calculating p-value

p-values are calculated by calculating the cumulative distribution function (CDF) of a statistic, or by extrapolating or interpolating values from a table of pre-calculated data. In our case, we used the permutation results of our data as the CDF and then calculated the p-value from counting the number of points above or below (depending on the test used) the non-permuted (observed) value.

In NLA this is referred to as “ranking” since it is simply counting values in a sorted list. This basic ranking is referred to as the “uncorrected” p-value. There are two other options for ranking in NLA. These account for FWER. The first method is based off the “randomise” method ^17,27. The second method is called “Westfall-Young” in NLA described by an alogrithm ¹⁶ by Westfall and Young.

Result Rank

class net.ResultRank

Ranker to calculate p-values from permutation testing

Parameters:

permuted_network_results – The NetworkTestResult object from permutation test
number_of_network_pairs – The number of network pairs for the Brain Atlas used

:return

nonpermuted_network_results: The results from the network level test (NetworkTestResult object)

permuted_network_results: The network level test results for each permutation

number_of_network_pairs: The number of network pairs in the atlas being used

uncorrectedRank(test_method, permutation_results, no_permutation_results, ranking_statistic, probability, ranking)

Performs ranking of observed result among all results (all permutations plus itself)

Parameters:

test_method – The method of the test being ranked (full connectome or within network pair)
permutation_results – The test result for all permutations
no_permutation_results – The observed test result
ranking_statistic – The statistic used in ranking for each test
probability – The name of the p-value (single_sample or two_sample)
ranking – The NetworkTestResult object to place the results

Returns:

The same NetworkTestResult object with ranking results

freedmanLaneMethodRank(test_method, permutation_results, no_permutation_results, ranking_statistic, probability, ranking)

Ranks the observed result using method described by Winkler to correct for FWER

Parameters:

test_method – The method of the test being ranked (full connectome or within network pair)
permutation_results – The test result for all permutations
no_permutation_results – The observed test result
ranking_statistic – The statistic used in ranking for each test
probability – The name of the p-value (single_sample or two_sample)
ranking – The NetworkTestResult object to place the results

Returns:

The same NetworkTestResult object with ranking results

westfallYoungMethodRank(test_method, permutation_results, no_permutation_results, ranking_statistic, probability, ranking)

Ranks the observed result using method described by Westfall and Young to correct for FWER

Parameters:

test_method – The method of the test being ranked (full connectome or within network pair)
permutation_results – The test result for all permutations
no_permutation_results – The observed test result
ranking_statistic – The statistic used in ranking for each test
probability – The name of the p-value (single_sample or two_sample)
ranking – The NetworkTestResult object to place the results

Returns:

The same NetworkTestResult object with ranking results

p-value for Each Test Based on Test Method

	No Permutations	Full Connectome	Within Network Pair
\(\chi^2\)	Two Sample	Two Sample	Two Sample
Hypergeometric	Two Sample	Two Sample	Two Sample
Kolmogorov-Smirnov	Single Sample	Two Sample	Single Sample
Student’s t-test	Single Sample	Two Sample	Single Sample
Welch’s t-test	Single Sample	Two Sample	Single Sample
Wilcoxon	Single Sample (Signed-Rank)	Two Sample (Rank-Sum)	Single Sample (Signed-Rank)