Introduction to Probability – Set Theory

Recently I have been undertaking a project to commit my accrued notes relating to statistics online. I want to do this for a few reasons:

1. Paper is perishable but the internet will persist (hopefully)
2. In rewriting these notes in a blog format perhaps they can be useful to those who are also interested in these concepts
3. To help deepen my understanding of the material – just because you pass an exam on something doesn’t necessarily mean you understand it at a deep level

I figure the beginning is as good a place as any to start. In order to understand probability theory with any level of clarity we first need to make a pitstop at set theory and collect some results and ideas we will need to help us later down the line. This post is our pitstop.

Defining Sets and Sample Space

Sets are a collection of objects, if an object belongs to a set then it is called an element or member of that set. If a is an element of the set A then this is denoted by the symbol \in – we say a \in A.

If each element of a set A_1 is also an element of a different set A_2, then we say the set A_1 is a subset of A_2. This is indicated by A_1 \subseteq A_2.

If set A has 0 elements, then A is called the null set and is denoted by A = \emptyset. Since the null set \emptyset contains no elements, it is logically correct to say that any element belonging to \emptyset also belongs to A such that \emptyset \subset A. The set which includes the totality of all elements that pertain to an experiment can be called the sample space which is denoted by S. Therefore for any event A, it is true that \emptyset \subset A \subset S.

Basic Operations and Relations of Set Theory

Unions
If A and B are any two events, the union of A and B is defined to be the event containing all outcomes that belong to A alone, to B alone, or to both A and B. The notation for the union of A and B is A \cup B. The union has the following properties:

1. A \cup B = B \cup A
2. A \cup A = A
3. A \cup \emptyset = A
4. A \cup S = S
5. If A \subset B, then A \cup B = B
6. Associativity law: A \cup B \cup C = (A \cup B) \cup C = A \cup (B \cup C)

The above properties are intuitive if you stare at them long enough. If you’re convinced initially then imagine objects in each set and see how they relate with respect to the properties above.

Intersections
If A and B are any two events, the intersection of A and B is defined to be the event containing all outcomes that belong to both A and B. The notation for the intersection of A and B is A \cap B.
The intersection has the following properties:

1. A \cap B = B \cap A
2. A \cap A = A
3. A \cap \emptyset = \emptyset
4. A \cap S = A
5. If A \subset B, then A \cap B = A
6. Associativity law: A \cap B \cap C = (A \cap B) \cap C = A \cap (B \cap C)

Complements
The complement of an event A is defined to be the event that contains all elements in the sample space S which do not belong in A. The notation for the complement of A is A^c. For any event A, the complement has the following properties:

1. (A^c)^c = A
2. \emptyset^c = S
3. S^c = \emptyset
4. A \cup A^c = S
5. A \cap A^c = \emptyset

Disjoint events
Two events A and B are disjoint or mutually exclusive if they have no elements in common. Therefore A \cap B = \emptyset.

Laws of Operations of Sets

Name of Law
Commutative Law	A \cap B = B \cap A	A \cup B = B \cup A
Associative Law	A \cap (B \cap C) = (A \cap B) \cap C	A \cup (B \cup C) = (A \cup B) \cup C
Distributive Law	A \cap (B \cap C) = (A \cap B) \cup (A \cap C)	A \cup (B \cap C) = (A \cup B) \cap (A \cup C)
Identity Law	A \cap S = A	A \cup \emptyset = A
Domination Law	A \cap \emptyset = \emptyset	A \cup S = S
Idempotent Law	A \cap A = A	A \cup A = A
Complement Law	S^c = \emptyset	\emptyset^c = S
De Morgan’s Law	(A \cap B)^c = A^c \cup B^c	(A \cup B)^c = A^c \cap B^c
Involution Law	(A^c)^c = A

De Morgan’s laws

De Morgan’s theorems have particular use in probability so it is worth writing the proof for them below. I found these proofs to be a little complicated the first time round so I will provide a running commentary as we go step by step.

Proof of De Morgan’s law: (A \cup B)^c = A^c \cap B^c

Let P = (A \cup B)^c and Q = A^c \cap B^c
and let x be an arbitrary element of P then x \in P \Rightarrow x \in (A \cup B)^c

\Rightarrow x \notin (A \cup B)
If element x is not in the union of sets A and B

\Rightarrow x \notin A \ and\ x \notin B
This implies that element x is not in set A alone and not in set B alone

\Rightarrow x \in A^c \ and\ x \in B^c
Which means that element x is in the complement of set A which is denoted as A^{c} and that element x is in the complement of set B which is denoted as B^{c}.

\Rightarrow x \in A^c \cap B^c
If element x is in both set A^c alone and in set B^c alone then it must be in the intersection of the sets A^c and B^c

\Rightarrow x \in Q
We defined the intersection of the sets A^c and B^c as the set Q

Therefore P \subset Q.

Let y be an arbitrarily element of Q then y \in Q \Rightarrow y \in A^c \cap B^c

\Rightarrow y \in A^c \ and\ y \in B^c
If element y is in the intersection of A^c and B^c then it implies element y is in set A^c alone and in set B^c alone

\Rightarrow y \notin A \ and\ y \notin B
Since element y is in the complement of sets A and B then it means that it is not in sets A and B.

\Rightarrow y \notin (A \cup B)
If element y is in neither set A or set B then it is not in the union of sets A and B.

\Rightarrow y \in (A \cup B)^c
\Rightarrow y \in P
Therefore Q \subset P

Combining both results gives P = Q which is the same as (A \cup B)^c = A^c \cap B^c.

In the same way we can work through a proof for the second law.

Proof of De Morgan’s law: (A \cap B)^c = A^c \cup B^c

Let M = (A \cap B)^c and N = A^c \cup B^c

Let x be an arbitrary element of M then x \in M \Rightarrow x \in (A \cap B)^c

\Rightarrow x \notin (A \cap B)
\Rightarrow x \notin A \ or \ x \notin B
\Rightarrow x \in A^c \ or \ x \in B^c
\Rightarrow x \in A^c \cap B^c
\Rightarrow x \in A^c \cup B^c
\Rightarrow x \in N
Therefore, M \subset N

Let y be an arbitrary element of N then y \in N \Rightarrow y \in A^c \cup B^c

\Rightarrow y \in A^c \ or\ y \in B^c
\Rightarrow y \notin A \ or\ y \notin B
\Rightarrow y \notin (A \cap B)
\Rightarrow y \in (A \cap B)^c
\Rightarrow y \in M
Therefore, N \subset M
Combining both results gives M = N which is the same as (A \cap B)^c = A^c \cup B^c

This puts an end to our pitstop and we are ready to proceed further in laying down the axioms of probability theory. I will be adding to this as I write up and summarise more of my notes – hopefully it can become somewhat of a probability fundamentals series. In the future I will also look to do the same for linear algebra (a personal favourite area of mine) and other areas related to probability and statistics so watch this space!