• Major Problems 

• • Problem 1

• • • These ML Algorithms do not allow us to combine prior knowledge (or past experience) with our Training Examples, when learning a Concept

• • Problem 2

• • • These ML Algorithms either accept or reject a Hypothesis (h) i.e., take a Binary Decision

• • • • Instead of assigning a probability to each Hypothesis (h) in the Hypothesis Space (H)

• A Possible Solution

• • Bayesian Learning Algorithms

• Naïve Bayes – Summary

• • Representation of Training Examples (D)

• • • Attribute-Value Pair

• • Representation of Hypothesis (h)

• • • Probability Distribution

• • Searching Strategy

• • • I am not clear about Search Strategy of Naïve Bayes Algorithm. Please drop me an email if you know it. جزاک اللہ خیر 

• • Training Regime 

• • • Batch Method

• • Strengths

• • • Naïve Bayes Algorithm allows us to combine prior knowledge (or past experience) with our Training Examples, when learning a Concept

• • Naïve Bayes Algorithm assigns a probability to each Hypothesis (h) in the Hypothesis Space (H)

• • • Instead of simply accepting or rejecting a Hypothesis (h)

• • Naïve Bayes Algorithm can handle noisy Data 

• • Naïve Bayes Algorithm show good Accuracy on Testing Data even if the Set of Training Examples (D) is rather small 

• • Naïve Bayes Algorithm can avoid Overfitting

• • Naïve Bayes Algorithm are used in Deep Learning Algorithms, which allows Deep Learning Algorithms to learn from small datasets / corpora 

• • Naïve Bayes Algorithm can make Probabilistic Predictions 

• • • Instead of making a Binary Prediction i.e. 0 or 1

• Weaknesses

• • Naive Bayes Algorithm assumes that Attributes / Features are independent of each other

• • • However, in Real world, Attributes / Features depend on each other 

•  Bayesian Learning Algorithms are Probabilistic Machine Learning Algorithms 

• Bayesian Learning uses Bayes’ Theorem to determine the Conditional Probability of a Hypotheses (h) given a Set of Training Examples (D)

• Bayesian Learning Algorithms allow us to combine prior knowledge (or past experience) with your Set of Training Examples (D) 

• Bayesian Learning Algorithms are suitable for following Machine Learning Problems

• • When you want to combine prior knowledge (or past experience) with your Set of Training Examples (D) 

• • When you want to assign a probability to each Hypothesis (h) in the Hypothesis Space (H)

• • • Instead of simply accepting or rejecting a Hypothesis (h)

• • When you have noisy Data 

• • When you have small Training Data

• • • In recent years, Bayesian Learning Algorithms are used in Deep Learning Algorithms, which allows Deep Learning Algorithms to learn from small datasets / corpora 

• • When you want to make Probabilistic Predictions 

• • • Instead of making a Binary Prediction i.e. 0 or 1

• A Brute Force Bayesian Learning Algorithm works as follows

• • Step 1: Compute Posterior Probability for all the Hypothesis (h) in Hypothesis Space (H)

• • Step 2: Output Most Probable Hypothesis (a.k.a. Maximum A Posteriori (MAP) Hypothesis, hMAP)

• Bayes Optimal Classifier works as follows to calculate the Most Probable Classification of an unseen instance x

• • Step 1: Calculate Posterior Probability for every Hypothesis h ∈ H

• • Step 2: Calculate the prediction of each Hypothesis (h) for each new instance

• • Step 3: Use the combination of Step 1 and 2. to classify each new instance 

• Prediction returned by hMAP(x) is (always) not Most Probable Classification? 

• To reduce the computation cost in finding the Most Probable Classification using Bayes Optimal Classifier, the Gibbs Algorithm works as follows

• • Step 1: Choose one Hypothesis (h) at random, according to P(h|D)

• • Step 2: Use this h to classify new unseen instances

• Naive Bayes Algorithm is a highly practical Bayesian Learning Algorithm

• •  Performance comparable to Artificial Neural Networks and Decision Tree Learning Algorithms on some tasks

• Naïve Bayes is very good for

         Text Classification