Reading 7.2 Conditional Probability

Introduction

In the last reading, we established the five basic rules of probability, which include the two restricted versions of the Addition Rule and Multiplication Rule: The Addition Rule for Disjoint Events and the Multiplication Rule for Independent Events. We have also established a General Addition Rule for which the events need not be disjoint. In order to complete our set of rules, we still require a General Multiplication Rule for which the events need not be independent. In order to establish such a rule, however, we first need to understand the important concept of conditional probability.

The rest of this week will be organized as follows: We'll first introduce the idea of conditional probability, and use it to formalize our definition of independent events, which in the first reading was presented only in an intuitive way. We will then develop the General Multiplication Rule, a rule that will tell us how to find P(A and B) in cases when the events A and B are not necessarily independent. We'll conclude with a discussion of probability trees, a method of displaying conditional probability visually that is very helpful in solving problems.

Conditional Probability

In the first part of this module, we'll introduce the concept of conditional probability. The idea here is that the probabilities of certain events may be affected by whether or not other events have occurred. Let's illustrate this idea with a simple example:

Example

All the students in a certain high school were surveyed, then classified according to gender and whether they had either of their ears pierced:

A table of the data. The column headings are "Pierced," "Not Pierced," and "Total." The Rows are "Male," "Female," and "Total." The data in the cells is given in "Row, Column: Value" format: Male, Pierced: 36; Male, Not Pierced: 144; Male, Total: 180; Female, Pierced: 288; Female, Not Pierced: 32; Female, Total: 320; Total, Pierced: 324; Total, Not Pierced: 176; Total, Total: 500;

(Note that this is a two-way table of counts that was first introduced when we talked about the relationship between two categorical variables. It is not surprising that we are using it again in this example, since we indeed have two categorical variables here: Gender: M or F (in our notation, "not M"), and Pierced: Yes or No)

Suppose a student is selected at random from the school. Let M and not M denote the events of being male and female, respectively, and E and not E denote the events of having ears pierced or not, respectively. We'll start by asking what will seem like simple questions, and we'll build our way to conditional probability:

What is the probability that the student has one or both ears pierced?

Since a student is chosen at random from the group of 500 students, out of which 324 are pierced,

P(E) = 324/500 = .648
What is the probability that the student is male?

Since a student is chosen at random from the group of 500 students, out of which 180 are male,

P(M) = 180/500 = .36
What is the probability that the student is male and has ear(s) pierced?

Since a student is chosen at random from the group of 500 students out of which 36 are male and have their ear(s) pierced, P(M and E) = 36/500 = .072

Now something new:
Given that the student that was chosen is male, what is the probability that he has one or both ears pierced?

At this point, new notation is required, to express the probability of a certain event given that another event holds. We will write "the probability of having one or both ears pierced (E) , given that a student is male (M)" as P(E | M).

A word about this new notation: The event whose probability we seek (in this case E) is written first, the vertical line stands for the word "given" or "conditioned on," and the event that is given (in this case M) is written after the "|" sign.

We call this probability the conditional probability of having one or both ears pierced, given that a student is male: it assesses the probability of having pierced ears under the condition of being male. Now to solve for the probability, we observe that choosing from only the males in the school essentially alters the sample space S from all students in the school to all male students in the school. The total number of possible outcomes is no longer 500, but has changed to 180. Out of those 180 males, 36 have ear(s) pierced, and thus:

P(E | M) = 36/180 = .20.

Comments

Note that when we evaluate the conditional probability, we always divide by the probability of the given event. The probability of both goes in the numerator.
The above formula holds as long as P(A) > 0, since we cannot divide by 0. In other words, we should not seek the probability of an event given that an impossible event has occurred.

Let's see how we can use this formula in practice:

Example

On the "Information for the Patient" label of a certain antidepressant, it is claimed that based on some clinical trials, there is a 14% chance of experiencing sleeping problems known as insomnia (denote this event by I), there is a 26% chance of experiencing headache (denote this event by H), and there is a 5% chance of experiencing both side effects (I and H).

(a) Suppose that the patient experiences insomnia; what is the probability that the patient will also experience headache?

Since we know (or it is given) that the patient experienced insomnia, we are looking for P(H | I). According to the definition of conditional probability:

P(H | I) = P(H and I) / P(I) = .05/.14 = .357.

(b) Suppose the drug induces headache in a patient; what is the probability that it also induces insomnia?

Here, we are given that the patient experienced headache, so we are looking for P(I | H).

Using the definition P(I | H) = P(I and H) / P(H) = .05/.26 = .1923.

Comment

Note that the answers to (a) and (b) above are different. In general, P(A | B) does not equal P(B | A). We'll come back and illustrate this point later in this module.

Independence

As we saw in the Exploratory Data Analysis section, whenever a situation involves more than one variable, it is generally of interest to determine whether or not the variables are related. In probability, we talk about independent events, and in the first module we said that two events A and B are independent if event A occurring does not affect the probability that event B will occur. Now that we've introduced conditional probability, we can formalize the definition of independence of events and develop four simple ways to check whether two events are independent or not. We will introduce these "independence checks" using examples, and then summarize.

Example

Consider again the two-way table for all 500 students in a particular high school, classified according to gender and whether or not they have one or both ears pierced.

The same table of the data for piercings. The column headings are "Pierced," "Not Pierced," and "Total." The Rows are "Male," "Female," and "Total." The data in the cells is given in "Row, Column: Value" format: Male, Pierced: 36; Male, Not Pierced: 144; Male, Total: 180; Female, Pierced: 288; Female, Not Pierced: 32; Female, Total: 320; Total, Pierced: 324; Total, Not Pierced: 176; Total, Total: 500;

Would you expect those two variables to be related? That is, would you expect having pierced ears to depend on whether the student is male or female? Or, to put it yet another way, would knowing a student's gender affect the probability that the student's ears are pierced? To answer this, we may compare the overall probability of having pierced ears to the conditional probability of having pierced ears, given that a student is male. Our intuition would tell us that the latter should be lower: male students tend not to have their ears pierced, whereas female students do. Indeed, for students in general, the probability of having pierced ears (event E) is P(E) = 324/500 = .648. But the probability of having pierced ears given that a student is male is only P(E | M) = 36/180 = .20.

As we anticipated, P(E | M) is lower than P(E). The probability of a student having pierced ears changes (in this case, gets lower) when we know that the student is male, and therefore the events E and M are dependent. (If E and M were independent, knowing or not knowing that the student is male would not have made a difference ... but it did.)

This example illustrates that one method for determining whether two events are independent is to compare P(B | A) and P(B).

If the two are equal (i.e., knowing or not knowing whether A has occurred has no effect on the probability of B occurring) then the two events are independent. Otherwise, if the probability changes depending on whether we know that A has occurred or not, then the two events are not independent. Similarly, using the same reasoning, we can compare P(A | B) and P(A).

Example

Recall the side effects example. On the "Information for the Patient" label of a certain antidepressant it is claimed that based on some clinical trials, there is a 14% chance of experiencing sleeping problems known as insomnia (denote this event by I), there is a 26% chance of experiencing headache (denote this event by H), and there is a 5% chance of experiencing both side effects (I and H).

Are the two side effects independent of each other?

To check whether the two side effects are independent, let's compare P(H | I) and P(H).

In the previous part of this module, we found that P(H | I) = P(H and I) / P(I) = .05/.14 = .357, while P(H) = .26. Knowing that a patient experienced insomnia increases the likelihood that he/she will also experience headache from .26 to .357. The conclusion, therefore is that the two side effects are not independent, they are dependent.

Alternatively, we could have compared P(I | H) to P(I). P(I) = .14, and previously we found that P(I | H)= P(I and H) / P(H) = .05/.26 = .1923, and again, since the two are not equal, we can conclude that the two side effects I and H are dependent.

Comment: Another method of checking independence

Recall the pierced ears example. We checked the independence of the events M (being a male) and E (having pierced ears) by comparing P(E) to P(E | M).

An alternative method of checking for dependence would be to compare P(E | M) with P(E | not M) [same as P(E | F)]. In our case, P(E | M) = 36/180 = .2, while P(E | not M) = 288/320 = .9, and since the two are very different, we can say that the events E and M are not independent.

In general, another method for checking the independence of events A and B is to compare P(B | A) and P(B | not A). In other words, two events are independent if the probability of one event does not change whether we know that the other event has occurred or we know that the other event has not occurred. It can be shown that P(B | A) and P(B | not A). would differ whenever P(B) and P(B | A) differ, so this is another perfectly legitimate way to establish dependence or independence.

Before we establish a general rule for independence, let's consider an example that will illustrate another method that we can use to check whether two events are independent:

Example

A group of 100 college students were surveyed about their gender and whether they had decided on a major.

Offhand, we wouldn't necessarily have any compelling reason to expect that deciding on a major would depend on a student's gender. We can check for independence by comparing the overall probability of being decided to the probability of being decided given that a student is female:

P(D) = 45/100 = .45 and P(D | F) = 27/60 = .45.

The fact that the two are equal tells us that, as we might expect, deciding on a major is independent of gender. Note from the comment that these must also equal P(D | M), which is 18/40 = .45.

Now let's approach the issue of independence in a different way: first, we may note that the overall probability of being decided is 45/100 = .45.

And the overall probability of being female is 60/100 = .60.

If being decided is independent of gender, then 45% of the 60% of the class who are female should have a decided major; in other words, the probability of being female and decided should equal the probability of being female multiplied by the probability of being decided. If the events F and D are independent, we should have P(F and D) = P(F) * P(D).

In fact, P(F and D) = 27/100 = .27 = P(F) * P(D) = .45 * .60. This confirms our alternate verification of independence.

In general, another method for checking the independence of events A and B is to compare P(A and B) to P(A) * P(B). If the two are equal, then A and B are independent, otherwise the two are not independent.

Let's summarize all the possible methods we've seen for checking the independence of events in one rule:

Two events A and B are independent if any one of the following hold:

P(B | A) = P(B)

P(A | B) = P(A)

P(B | A) = P(B | not A)

P(A and B) = P(A) * P(B)

Comment

These various equalities turn out to be equivalent, so that if one equality holds, all are equal, and if one equality does not hold, all are not equal. (This is the case for the same reason that knowing one of the values P(A and B), P(A and not B), P(not A and B), or P(not A and not B), along with P(A) and P(B), allows you to determine the remaining cells of a two-way probability table.)

Therefore, in order to check whether events A and B are independent or not, it is sufficient to check only whether one of the four equalities holds—whichever is easiest for you.

When you finish reading any text that's detailed and dense, like math text usually is, it makes sense to write down some notes about what you learned! Even if you've been taking notes as you read, now's a good time to jot down a summary at the end of what the most important points in this reading were.

About This Text

This text is from the Probability + Statistics text produced by Carnegie-Mellon University's Open Learning Initiative, with some modifications by Clare Heimer. It's shared under a Creative Commons Attribution-NonCommercial-ShareAlike license.