probability - mit

 01

  1. - probability as a mathematical framework for reasoning about uncertainty

a framework for dealing with uncertainty for dealing with situations in which we have some kind of randomness.

to know how to set up what does it take to set up a probabilistic model..

   2.- probabilistic models : 

           sample space: a description of all the things that may happen during a random experiment.

            - sample space (omega)

                "List" (set) of possible outcomes

                 List must be:

                 - mutually exclusive

                 - collectively exhaustive( 하나도 빠트리는것없이 철저한)

                Art : to be at the "right" granularity

ex) let's have a experiment (하나의 experiment와 그에 따른 sample space 인지 이런거 정확히 설정:  what's the sample space for that experiment?)

1) set up the sample space :we come up with a list of all the possible things that may happen during this experiment.

2) legitimate sample space - [ H, T+raining, T+not raining] - mutual exclusive,  collectively exhaustive

whenever you have an experiment that consists of multiple stages, it might be useful at least visually, to give a diagram that shows you how those stages evolve.

** the experiment consists of stages.!! 

the more important order of business is now to look at those possible outcomes and to make some statements about their relative likelihoods.  : which outcome is more likely to occur compared to the others? the way we do this is by assigning probabilities to the outcomes

probability axioms

Event: a subset of the sample space

Probability is assigned to events

Axioms:

1. Non negativity : P(A)>=0

2.Normalization : P(omega) = 1

3.Additivity : If A 교집합 B = 공집합, then, P(A 합집합 B) = P(A)+P(B)

P({s1,s2,s3,s4,....,sk}) = P({s1}) + P({s2}) + P({s3})+....+P({sk})

                            = P(s1)+P(s2)+P(s3)+....+P(sk)

Axiom 3 needs strenthening

Do weird sets have probabilities?

어떤 개별적인 사건이 일어날 확률을 이야기하는 것은 별로 의미가 없고 subset 전체의 결과에 확률을 어싸인 하는 것이 중요하다. 

- individual outcome  이 subset 집합안에 존재하면  event가 일어났다 라고 표현하며

 individual outcome 이 subset집합 바깥에 존재하면  event가 일어나지 않았다고 한다. 

probability는 우리의 신념(our belief)을 나타내는 것이므로 여러가지 방법을 사용해서 개별적인 결과에 대응시킨다. 하지만 이렇게 하는데 규칙이 전제된다.  - within Axiom 3

then, you can repeat this argument as many times as you want for the union of any finite number of sets.

A1,A2,A3,......,An : disjoint

 수학적으로는 확률을 대응시키지 못하는 이상한 결과들도 있다. 

Discrete uniform law

let all outcomes be equally likely 

Then,

p(A) = number of elements of A / total number of sample points

Defines fair coins,  fair dice, well-shuffled decks

이 과정에서 중요한것은 counting인데 어떤 문제들은 counting 자체가 복잡해서 이것을 systemically 하게 하는 방법을 공부해야한다. 

counting is generally simple, but for some problems it gets pretty complicated,

Continuous uniform law

Two "random" numbers in [0,1].

Uniform law : Probability = Area

P(X+Y<=1/2) =?

P((X,Y)= (0.5,0.3))

   probability law: a description of our belief about which outcomes are more likely to occur compared to other outcomes. probability laws have to obey certain properties that we call the axioms of probability.

3. axioms of probability (rules of game) :

1. Non negativity : P(A)>=0

2.Normalization : P(omega) = 1

3.Additivity : If A 교집합 B = 공집합, then, P(A 합집합 B) = P(A)+P(B)

4. simple examples          

02 

conditional probability

Three important tools 

multiplication rule

Total probability theorem

Bayes' rule

information is always partial,  the question is what do we do to probabilities if we have  some partial information about the random experiments. 

- these ways basically correspond to divide and conquer methods for breaking up problems into simpler pieces, and also one more fundamental tool which allows us to use conditional probabilities to do inference if we get a little bit of information about some phenomenon, what can we infer about the things that we have not seen? 부분적인 정보를 가지고 추론할때 이러한 방법들을 사용한다. 

1.there's also element of art in how to choose your sample space, depending on how much detail you want to capture. 

2. the more interesting part is to assign probabilities to our model, that is to make some statements about what we believe to be likely and what we believe to be unlikely. the way we do that is by assigning probabilities to subset of the sample space. 

- 일반적인 이야기지만 계속 생각해야할 내용.

axiom 3,

Additive Axiom applies to the case where we have a sequence of disjoint events and we take their union.

the reason is that infinite sets are not all of the same size. it has more element than any sequence could have. 

in continuous model,

면적을 생각하면 동일하다. 모든 점은 면적이 0이지만 어느정도 합치면 면적값이 나오는것과 동일한 경우이다. 

개별 아웃컴들이 확률이 0 이더라도 그것들은 일어난것이라 할 수 있다. 

but yet those particular number did occur.

zero probability does not mean it's impossible.

it just extremely extremely unlikely by itself.

Problem solving 

- specify sample space

- define probability law

- identify event of interest

- calculate

 Conditional probability

you know something about the world. 

and based on what you know when you set up a probability model and you write down probabilities for the different outcomes. Then something happens, somebody tells you a little more about the world, gives you some new information. This new information ,in general, should change your belief about what happened or what may happen. so whenever we're given new information, some partial information about the outcome of the experiment, we should  revise our beliefs.

conditional probabilities are just the probabilities that apply after the revision of our beliefs when you are given some information.

 the initial belief about the experiment

suppose now that someone comes and tells you that event B occurred. so they don't tell you  the full outcome  with the experiment. but they just tell you the outcome is known to lie inside this set of B.

Well then, you should certainly change your beliefs in some way.

and your new beliefs about what is likely to occur or what is not 

= P(A|B)

(sample space: a description of all the things that may happen during a random experiment.)

sample space 가 조정되어야한다.  our old sample space in someways are irrelevant.  we have a new sample space , which is just the set B. we are certain that the outcome is going to be inside B.

general sample space -) partial information input -) sample space changed --)revise the probability:  P(B) = 1/2 --) P(B|B) = 1 and keep the same proportion of the probability of each event related.

how do we revise the probability that A would occur?

translate this intuitive reasoning into a definition :

we look at the total probability of B and out of that probability that was inside here what fraction of that probability is assigned to points for which the event A also occurs?

- B가 일어난 상황에서 A도 또한 일어나는 상황에 할당된 확률은 무엇인가?

Does it give us the same numbers as we got this with this heuristic argument?

정의와 리즈닝이 일치한다.

 

P(B) = 0인 경우는 conditional probability is not defined 라고 한다. 

P(A교B)= P(B)*P(A|B)  :  이것은 정의식을 변형한것이지만 훌륭한 해석을 하므로 기억.

              =P(A)*P(B|A)

Conditional probabilities are the new probabilities that apply in a new universe where event B is known to have occurred.

we had an original probability model 

- ) we are told that B occurred 

-)  we revise our model 

-)  our new model should still be legitimate probability model. 

so it should satisfy all sort of properties that ordinary  probabilities do satisfy.

 

devise된 이후에 다른 모델이 되었지만 그래도 여전히 합당한 확률 모델이다. 

 A와 B는 disjoint event 인 상황에서 C가 일어났다면, 새로운 모델로 리바이즈 해야하는 데 그 때 도 보통의 확률과 같은 이러한 법칙이 성립된다. 

Probabilities

조건부확률은 트리구조! 

Event A : Airplane is flying above

Event B: Something registers on radar screen

P(A)= 0.05

P(Ac) = 0.95

확률이 배정되어있음.

초기 샘플 스페이스

we're given probabilities about how the radar behaves.

have indirectly specified the probability law in our model by starting with conditional probabilities as opposed to starting with ordinary probabilities. 

Can we derive ordinary probabilities from conditional one?

 

think of probabilities as frequencies, if i do the experiment over and over, what fraction of the time is it going to be the case that both A and B occur? there's going to be a certain fraction of the time at which B occurs. and out of those times when B occurs, there's going to be a further fraction of the experiments in which A also occurs.

- 확률이 빈도라고 가정하면 실험이 계속될 수록 B가 일어나는 시간이 있을 것이고 A가 일어나는 시간이 있을 것이다. 

B already occurred, event A also occurs.

A branch 값P(A)* 그다음 branch 값 P(B|A) = branch multiplication 

 systemmetric calculation . - 조건부 확률을 시스테믹하게 트리구조를 이용해서 계산해낸다. 

Questions

Q: Given that your radar recorded something, how likely is it that there is an airplane up there?

계산상으로 30% 나옴

상황에 따라 성능이 좋아보이는 카메라라고 하더라도 계산을 해보면 필요한 것을 예측하는데 작은 확률이 나와버린다. the reasoning that comes in such situations is pretty subtle.

3 very important, very basic tools that you use to solve more general probability problems

Notice what we move along the branches as the tree progresses.

- Any point in the tree corresponds to certain events having happened. and then, given that this has happened, we specify conditional probabilities.

leaves 에 있는 상황의 probability를 어떻게 계산하느냐..

- all that you do is move along with the tree and multiply conditional probabilities along the way,.

일어난 순서를 간결하고 정확히 지키기.

P(A교B교C) = 첫번째발생한 사건의 확률 * 두번째 발생한 사건의 확률( 조건부확률)

the 1st 

so you write down probabilities along all those tree branches and just multiply them as you go. 

sample space is partitioned in a number of sets - ex) radar ..partitioned by 2 sets.. either a plane is there , or  a plane is not there. 여기서부터 시작되어 트리로 내려가면서 여러개의 가능한 시나리오를 제시할 수 잇다. 

the 2nd

P(B)  

맨먼저 일어난 사건 Ai를 각각의 시나리오라고 생각하면 되고, 

여러 시나리오 안에서 발생하는 event of interest , B라고 지정한다. 

and we want to calculate the overall pobability of the event B. 

 B is an event that consists of these three elements.

there are 3 ways that B can happen. either( contingency)B  happens with A1 and B happens with A2 and B happens with A3.

..

obviously, it has a generalization for the case of 4~5 more scenarios.

it gives you a way of breaking up the calculation of an event that can happen in multiple ways by considering individual probabilities for the different ways that the event can happen.

it's also true if it's a partition into an infinite sequence of events. 

how likely is it for B to happen under 1 scenario ..

these conditional probabilities tell us how likely is it for B to happen under 1 scenario or the other scenario or the other scenario,,

the overall probability of B is found by taking some combination fo the probabilities of B in the different possible worlds, in the different possible scenarios. 

under one scenario , it may be very likely. Under another scenario, it may be very unlikely.

the 3rd

P(A|B)

 -------

Cause and Effect model

So, we're starting with a causal model of our situation. it models from a given cause how likely is a certain effect to be observed. and then we do inference, which answers the question, given that the effect was observed, how likely is it that the world was in this particular situation or state or scenario.

conditional probability summary

how one can learn from experience or from experimental data and some systematic way.

how we can incorporate new knowledge to previous knowledge.

it's the foundation of almost everything that gets done when you try to do inference based on partial observations.

일부의 정보를 가지고 우리가 추론을 할때 매우 유용한 도구로서 확률모델.

given A, given B.......revise the model by definition of conditional probability.,,,,...how likely is it...

03

원래 오메가 공간속에 일부사건B이 일어났다고 정보가 들어오면 

샘플 스페이스는 오메가에서 B 로 조정되고 B에 속하지 않는 outcome들은 probability = 0 이된다. 

이렇게 조정된 샘플스페이스에서 we need to revise our probabilities, the new probabilities are called conditional probability 라고 한다. 

실제 확률 모델은 매우 복잡한 트리이지만 베이즈룰 써서 계산하는 것은 똑같다.