Effect size calculator (with Stan)

All effect sizes based on Bayesian models run in Stan

Still in development. See Github repo for Python code and Stan models. Open to comments at uanhoro.1@osu.edu

Two sample comparison for unbounded continuous data

Description: The test allows the group means and variances to be different.

You can either:

  1. Permit outliers in the data (assuming t rather than Normal data); OR
  2. Compare the groups at different percentiles (quantile regression)

Please scale your data such that values are at most in the hundreds. For example, if working with annual incomes, you can divide by 10,000.

Please correct the following error(s):

  • {{ error }}

Paste raw data into textboxes (Separate values by a space, comma or newline)

Group 1 (treatment), n = {{ n1 }}

Group 2 (control), n = {{ n0 }}

Do you want to compare the data at a specific percentile:

Yes No

Percentile for quantile test: %

What is the largest believable difference between the two group means? An irrational response would be a value greater than the range of your data. Be skeptical of large group differences.

By default, I assume that the ratio of both group standard deviations will not exceed {{max_sr}}. You can reduce/increase this number if you expect less/more heteroskedasticity in your data.

How many iterations should Stan run? The program will return half this number of iterations across 4 chains. If you enter 2000, the program will return 1000 posterior samples for each chain.

Requested interval: %

Results (with quantile interval)

Clicking submit will print out summary results and download four files:

  1. A summary file containing summary statistics for parameters and posterior samples.
  2. A rank plot for the mean difference and SD ratio. Each plot should be uniformly distributed. See: arxiv:1903.08008
  3. A line chart showing the probability that the mean difference exceeds any value. Answers: What is the probability that the difference between the groups exceeded X?
  4. Same as last chart but for the ratio of the group standard deviations.

Two way contingency table

Description: The test allows you compare the success rate of two groups on a binary outcome.

Please correct the following error(s):

  • {{ error }}

Enter aggregate data below

Yes No Total
Group 1 (treatment) {{ tsb_t1 }}
Group 2 (control) {{ tsb_t2 }}

Is the binary event an extremely low or high rate event? Consider event rates under 5% or above 95% as extreme:

Extreme event rate Non extreme event rate

What is the largest odds ratio you would expect? Consider 2 for outcomes that are difficult to change. For relations that are obvious to the naked eye, 10 is a reasonable limit. Consider 3 when unsure, be skeptical.

How many iterations should Stan run? The program will return half this number of iterations across 4 chains. If you enter 2000, the program will return 1000 posterior samples for each chain.

Requested interval: %

Results (with quantile interval)

Clicking submit will print out summary results and download four files:

  1. A summary file containing summary statistics for parameters and posterior samples.
  2. A rank plot for the odds ratio. Each plot should be uniformly distributed. See: arxiv:1903.08008
  3. A line chart showing the probability that the odds ratio exceeds any value. Answers: What is the probability that the odds ratio of the groups exceeded X?
  4. Same as last chart but for the ratio of the group probability/risk ratio and difference.

Two sample comparison for bounded continuous data

Description: The model assumes the data are continuous and bounded.

The model works better than the continuous approach above when the data are close to the extremes. Sample data are averaged Likert responses, proportions, ...

Please correct the following error(s):

  • {{ error }}

Paste raw data into textboxes (Separate values by a space, comma or newline)

Group 1 (treatment), n = {{ n1 }}

Group 2 (control), n = {{ n0 }}

What is the theoretical minimum of these data?

What is the theoretical maximum of these data?

Are the data clustered at the extremes i.e. extremely low or high values?

Yes No

What is the largest believable difference between the two group means? An irrational response would be a value greater than the range of your data. Be skeptical of large group differences.

How many iterations should Stan run? The program will return half this number of iterations across 4 chains. If you enter 2000, the program will return 1000 posterior samples for each chain.

Requested interval: %

Results (with quantile interval)

Clicking submit will print out summary results and download four files:

  1. A summary file containing summary statistics for parameters and posterior samples.
  2. A rank plot for the mean difference and SD ratio. Each plot should be uniformly distributed. See: arxiv:1903.08008
  3. A line chart showing the probability that the mean difference exceeds any value. Answers: What is the probability that the difference between the groups exceeded X?
  4. Same as last chart but for the ratio of the group standard deviations.