When a novel, adaptive mutation arises in a population it is more likely to go extinct than go to fixation. This is because rare alleles can by chance be lost from the population through accidental deaths, chromosomal segregation, or other forces behind genetic drift.

In 1955 Motoo Kimura used what’s known as diffusion theory to find formulas for calculating the probability that a novel allele with selection coefficient goes to fixation. For example, take a haploid, Wright-Fisher population of size that consists of only **A** individuals with fitness . Assume that one of those individuals mutates to **B**, such that the . From this setup, Kimura found that the the probability that **B** will eventually become fixed in the population and **A** goes extinct is approximately

under the right assumptions. For diploids with fitnesses , , and . This is

Sella and Hirch (2005) use intuition to modify Kimura’s results and derive some better and more useful alternatives for the above equations, which we use in our work for their nice properties. Sella and Hirch give the following equation in their paper:

where if the population is haploid and if the population is diploid “with multiplicative fitness within loci. …” Unfortunately, they did not specify the diploid model that they were using because there are two different ones used in the literature.

The first one is used above, , , and .

The second one is , , and .

As you can see, is not measuring the same thing in both models. In the first approach, ; in the second . This is important because accidentally mixing up the models will lead to erroneous results.

The difference between these two diploid models came up last week when we were applying Sella and Hirch (2005) to one of our projects. For diploids, we just couldn’t get their approximation to work (), and we suspected that they were using the second model, while we had read their paper to imply the first model. We looked back at their paper and realized that they neglect to specify exactly how they calculate and for diploids; we weren’t sure what they used.

I ended up working through their unspecified math to verify that in fact Sella and Hirch (2005) used the second model of diploid fitnesses, while we wanted to work with the first model. So here is a clarification of their equation, based on our way of thinking:

where if the population is haploid with fitnesses and , and if the population is diploid with fitnesses , , and .

#### References

- Sella and Hirch (2005) The application of statistical physics to evolutionary biology. PNAS 102:27 9541–9546.

Caveat: I am not a geneticist.

You cite two fitness models that are common in the literature, and I was somewhat surprised that neither one matches the classic models that I was taught in high-school biology. Well, OK, let me rephrase that: high-school biology didn’t teach anything like an explicit fitness model, they did present a couple of standard patterns of diploid gene interaction, which would suggest the following fitness models:

1. (“Recessive mutation”) f[AA] = f[AB] = 1; f[BB] = 1 + s

2. (“Dominant mutation”) f[AA] = 1; f[AB] = f[BB] = 1 + s

Do these fitness models affect the fixation probabilities much?

You write

Unfortunately, they did specify the diploid model that they were using because there are two different ones used in the literature.Isn’t there a “not” missing?

Fixed

Full dominance (or recessivness) are used a lot in classically derived population genetic models. However, genetic systems are often not best characterized under full dominance.

The system of multiplicative fitnesses here represent incomplete dominance, in which the heterozygote is intermediate of the homozygotes. It is used a lot in molecular evolution because on can derive explicit solutions for it. If one of the alleles was dominant, then the solution is extremely complex or nonexistent.

Update