.. title: Engineering Probability Class 2 Mon 2018-01-22 .. slug: class02 .. date: 2018-01-22 .. tags: mathjax .. category: class .. link: .. description: .. type: text .. sectnum:: .. contents:: Table of contents:: .. Misc ---- #. Why not use LMS more? #. This static CMS is much easier to use. I can create and save material more easily. #. You don't need to login to see the content. You can deep-link into the material. People outside the course can see, and save, the material. #. A few years ago, LMS sued an open source competitor. (They lost!) #. How to find this wiki? #. `Google **RPI WRF** `_. #. Go to my home page. #. Links to my current courses are listed at the top. Chapter 1 ctd ------------- #. `Rossman-Chance coin toss applet `_ demonstrates how the observed frequencies converge (slowly) to the theoretical probability. #. Example of unreliable channel #. Want to transmit a bit: 0, 1 #. It arrives wrong with probability e, say 0.001 #. Idea: transmit each bit 3 times and vote. #. 000 -> 0 #. 001 -> 0 #. 011 -> 1 #. 3 bits arrive correct with probability :math:`(1-e)^3` = 0.997002999 #. 1 error with probability :math:`3(1-e)^2e` = 0.002994 #. 2 errors with probability :math:`3(1-e)e^2` = 0.000002997 #. 3 errors with probability :math:`e^3` = 0.000000001 #. corrected bit is correct if 0 or 1 errors, with probability :math:`(1-e)^3+3(1-e)^2e` = 0.999996999 #. We reduced probability of error by factor of 1000. #. Cost: triple the transmission plus a little logic HW. #. Example of text compression #. Simple way: Use 5 bits for each letter: A=00000, B=00001 #. In English, 'E' common, 'Q' rare #. Use fewer bits for E than Q. #. `Morse code `_ did this 170 years ago. #. E = . #. Q = _ _ . _ #. Aside: An expert Morse coder is faster than texting. #. English can be compressed to about 1 bit per letter (with difficulty); 2 bits is easy. #. Aside: there is so much structure in English text, that if you add the bit strings for 2 different texts bit-by-bit, they can usually mostly be reconstructed. #. That's how cryptoanalysis works. #. Example of reliable system design #. Nuclear power plant fails if #. water leaks #. and operator asleep (a surprising number of disasters happen in the graveyard shift). #. and backup pump fails #. or was turned off for maintenance #. What's the probability of failure? This depends on the probabilities of the various failure modes. Those might be impossible to determine accurately. #. Design a better system? Coal mining kills. #. The backup procedures themselves can cause problems (and are almost impossible to test). A failure with the recovery procedure was part of the reason for a Skype outage. Chapter 2 --------- #. A random experiment has 2 parts: #. experimental procedure #. set of measurements #. Random experiment may have subexperiments and sequences of experiments. #. Outcome or sample point :math:`\zeta`: a non-decomposable observation. #. Sample space S: set of all outcomes #. :math:`|S|`: #. finite, e.g. {H,T}, or #. discrete = countable, e.g., 1,2,3,4,... Sometimes discrete includes finite. or #. uncountable, e.g., :math:`\Re`, aka continuous. #. Types of infinity: #. Some sets have finite size, e.g., 2 or 6. #. Other sets have infinite size. #. Those are either *countable* or *uncountable*. #. A countably infinite set can be arranged in order so that its elements can be numbered 1,2,3,... #. The set of natural numbers is obviously countable. #. The set of positive rational numbers between 0 and 1 is also countable. You can order it thus: \ :math:`\frac{1}{1}, \frac{1}{2}, \frac{1}{3}, \frac{2}{3}, \frac{1}{4}, \ \frac{3}{4}, \frac{1}{5}, \frac{2}{5}, \frac{3}{5}, \ \cdots` #. The set of real numbers is not countable (aka uncountable). Proving this is beyond this course. (It uses something called `diagonalization `_. #. Uncountably infinite is a bigger infinity than countably infinite, but that's beyond this course. #. `Georg Cantor `_, who formulated this, was hospitalized in a mental health facility several times. #. Why is this relevant to probability? #. We can assign probabilities to discrete outcomes, but not to individual continuous outcomes. #. We can assign probabilities to some events, or sets of continuous outcomes. #. E.g. Consider this experiment to watch an atom of sodium-26. #. Its half-life is 1 second (`Applet: Nuclear Isotope Half-lifes `_) #. Define the outcomes to be the number of complete seconds before it decays: :math:`S=\{0, 1, 2, 3, \cdots \}` #. :math:`|S|` is countably infinite, i.e., discrete. #. :math:`p(0)=\frac{1}{2}, p(1)=\frac{1}{4}, \cdots` :math:`p(k)=2^{-(k+1)}` #. :math:`\sum_{k=0}^\infty p(k) = 1` #. We can define events like these: #. The atom decays within the 1st second. p=.5. #. The atom decays within the first 3 seconds. p=.875. #. The atom's lifetime is an even number of seconds. \ :math:`p = \frac{1}{2} + \frac{1}{8} + \frac{1}{32} + \cdots = \frac{2}{3}` #. Now consider another experiment: Watch another atom of Na-26 #. But this time the outcome is defined to be the real number, x, that is the time until it decays. #. :math:`S = \{ x | x\ge0 \}` #. :math:`|S|` is uncountably infinite. #. We cannot talk about the probability that x=1.23 exactly. (It just doesn't work out.) #. However, we can define the event that :math:`1.23 < x < 1.24`, and talk about its probability. #. :math:`P[x>x_0] = 2^{-x_0}` #. :math:`P[1.23 < x < 1.24]` :math:`= 2^{-1.23} - 2^{-1.24} = 0.003` #. Event #. collection of outcomes, subset of S #. what we're interested in. #. e.g., outcome is voltage, event is V>5. #. certain event: S #. null event: :math:`\emptyset` #. elementary event: one discrete outcome #. Set theory #. Sets: S, A, B, ... #. Universal set: U #. elements or points: a, b, c #. :math:`a\in S, a\notin S`, :math:`A\subset B` #. Venn diagram #. empty set: {} or :math:`\emptyset` #. operations on sets: equality, union, intersection, complement, relative complement #. properties (axioms): commutative, associative, distributive #. theorems: de Morgan #. Prove deMorgan 2 different ways. #. Use the fact that A equals B iff A is a subset of B and B is a subset of A. #. Look at the Venn diagram; there are only 4 cases. #. 2.1.4 Event classes #. Remember: an event is a set of outcomes of an experiment, e.g., voltage. #. In a continuous sample space, we're interested only in some possible events. #. We're interested in events that we can measure. #. E.g., we're not interested in the event that the voltage is exactly an irrational number. #. Events that we're interested in are intervals, like [.5,.6] and [.7,.8]. #. Also unions and complements of intervals. #. This matches the real world. You can't measure a voltage as 3.14159265...; you measure it in the range [3.14,3.15]. #. Define :math:`\cal F` to be the class of events of interest: those sets of intervals. #. We assign probabilities only to events in :math:`\cal F`. #. 2.2 Axioms of probability #. An axiom system is a general set of rules. The probability axioms apply to all probabilities. #. Axioms start with common sense rules, but get less obvious. #. I: 0<=P[A] #. II: P[S]=1 #. III: :math:`A\cap B=\emptyset \rightarrow` :math:`P[A\cup B] = P[A]+P[B]` #. III': For :math:`A_1, A_2, ....` if :math:`\forall_{i\ne j} A_i \cap A_j = \emptyset` then :math:`P[\bigcup_{i=1}^\infty A_i]` :math:`= \sum_{i=1}^\infty P[A_i]` #. Example: cards. Q=event that card is queen, H=event that card is heart. These events are not disjoint. Probabilities do not sum. #. :math:`Q\cap H \ne\emptyset` #. P[Q] = 1/13=4/52, P[H] = 1/4=13/52, P[Q :math:`\cup` H] = 16/52!=17/52. #. Example C=event that card is clubs. H and C are disjoint. Probabilities do sum. #. :math:`C\cap H \ne\emptyset` #. P[C] = 13/52, P[H] = 1/4=13/52, P[Q :math:`\cup` H] = 26/52. #. Example. Flip a fair coin :math:`A_i` is the event that the first time you see heads is the i-th time, for :math:`i\ge1`. #. We can assign probabilities to these countably infinite number of events. #. :math:`P[A_i] = 1/2^i` #. They are disjoint, so probabilities sum. #. Probability that the first head occurs in the 10th or later toss = :math:`\sum_{i=10}^\infty 1/2^i` #. Corollory 1 #. :math:`P[A^c] = 1-P[A]` #. E.g., P[heart] = 1/4, so P[not heart] = 3/4 #. Corollory 2: P[A] <=1 #. Corollory 3: P[:math:`\emptyset`] = 0 #. Corollory 4: #. For :math:`A_1, A_2, .... A_n` if :math:`\forall_{i\ne j} A_i \cap A_j = \emptyset` then :math:`P\left[\bigcup_{i=1}^n A_i\right] = \sum_{i=1}^n P[A_i]` #. Proof by induction from axiom III. Iclicker questions ------------------ #. What is your major? #. CSYS #. ELEC #. CSCI #. Other engineering #. Other #. What is your class? #. 2018 #. 2019 #. 2020 #. 2021 #. Other #. What one athematical tool should I use in class? #. Matlab #. SciPy #. Mathematica #. Maple #. Other Material added after class -------------------------- #. `My handwritten tablet notes <../../handwritten/122.pdf>`_.