Skip to content

Ilm o Irfan

Technologies

  • Home
  • Courses
    • Free Courses
  • Motivational Seminars
  • Blog
  • Books
  • Contact
  • Home
  • Courses
    • Free Courses
  • Motivational Seminars
  • Blog
  • Books
  • Contact
Explore Courses

Machine Learning

  • October 31, 2022
  • Home
  • Free Book

Table of Contents

Chapter 8 -FIND-S Algorithm

Chapter Outline

  • Chapter Outline
  • Quick Recap
  • Research Cycle
  • Representing Training Examples and Hypothesis for FIND-S Machine Learning Algorithm
  • FIND-S Algorithm – Machine Learning Cycle 
  • Chapter Summary

Quick Recap

  • Quick Recap – Concept Learning and Hypothesis Representation
  • Learning is a Searching Problem
  • Research is defined as a systematic investigation of sources and materials, to develop solutions for Real-world Problems to improve the quality (peace and happiness) of human life 
  • A Researcher is a man of character and (s)he should safeguard his character 
  • The word Research is made up of
    •  Re and
    •  search 
  • Therefore, in Research, we mainly refine the solution(s) proposed for a Real-world Problem
  • The main steps of a Research Cycle are as follows
    • Step 1: Identify the Real-world Problem 
    • Step 2: Propose Solution (called Solution 1) to solve the Real-world Problem
    • Step 3: List down Strengths and Weaknesses of Solution 1
    • Step 4: Propose Solution (called Solution 2) to 
  •  further strengthen the Strengths of Solution 1
  •  overcome limitations of Solution 01
    • Step 5: List down Strengths and Weaknesses of Solution 2
    • Step 6: Propose Solution (called Solution 3) to 
  •  further strengthen the Strengths of Solution 2
  •  overcome limitations of Solution 02
    • Step 7: Continue this cycle till the Day of Judgment 😊
  • Considering FIND-S Algorithm 
    • Input to Learner (FIND-S Algorithm)
      • Set of Training Examples (D)
      • Set of Functions / Hypothesis (H)
    • Output by Learner (FIND-S Algorithm)
      • A Hypothesis (h) from H which best fits the Training Examples (D)
        • Note that h is an approximation of Target Function
  • Inductive Bias Is the set of assumptions needed in addition to Training Examples to justify Deductively Learner’s Classification
  • FIND-S Algorithm – Summary
    • Representation of Example
      • Attribute-Value Pair
    • Representation of Hypothesis (h)
      • Conjunction (AND) of Constraints on Attributes
    • Searching Strategy
      • I am not clear about this. Please drop me an email if you know. Jazak Allah Khair
    • Training Regime
      • Incremental Method
    • Inductive Bias of FIND-S Algorithm
      • Training Data is error-free 
      • Target Function / Concept is present in the Hypothesis Space (H)
    • Strengths
      • Returns a Model (h), which can be used to make predictions on unseen data
    • Weaknesses
      • Only works on error-free Data
        • However, Real-world Data is noisy 
      • Works on assumption that Target Function is present in the Hypothesis Space (H)
        • However, we may / may not find the Target Function in the Hypothesis Space (H) and this may / may not be known 
      • Only returns one hypothesis which best fits the Training Data
        • However, there can be multiple hypothesis, which best fit the Training Data

Research Cycle

  • Learning

  • Research
  • Definition
    • A systematic investigation of sources and materials, to develop solutions for Real-world Problems to improve the quality (peace and happiness) of human life 
  • Purpose
    • Develop solutions to serve the humanity for the sake of Allah and ultimately recognize Allah
  • Importance
    • In every era of human life, there will be various problems and it is important to develop efficient solutions to those problem
      • Solutions to emerging problems are not possible without Research
  • Applications 
    • Research helps us to understand
      • How to get معرفت  of الله (Who created us)?
      • How to become a balanced and characterful human being?
      • How to develop solutions to Real-world Problems to serve the humanity for the sake of Allah?
  • Researcher
  • Definition 
    • A Researcher is a man of character and (s)he should safeguard his character
  • Main Characteristics of a Researcher
    • عاجزی(Humbleness)
    • سادگی (Simplicity)
    • Only develops solutions / products which are beneficial for humanity
    • Only develops solutions / products which help human beings to become a balanced and characterful personality 
  • Understanding the Word Research
  • The word Research is made up of
    • Re and
    • search
  • Therefore, in Research, we mainly refine the solution(s) proposed for a Real-world Problem
  • Research Cycle
  • The main steps of a Research Cycle are as follows
    • Step 1: Identify the Real-world Problem 
    • Step 2: Propose Solution (called Solution 01) to solve the Real-world Problem
    • Step 3: List down Strengths and Weaknesses of Solution 01
    • Step 4: Propose Solution (called Solution 02) to 
      • further strengthen the Strengths of Solution 01
      • overcome limitations of Solution 01
    • Step 5: List down Strengths and Weaknesses of Solution 02
    • Step 6: Propose Solution (called Solution 03) to 
      • further strengthen the Strengths of Solution 02
      • overcome limitations of Solution 02
    • Step 7: Continue this cycle till the Day of Judgment 😊
  • Example 01 - Research Cycle
  • Real-world Problem
    • It gets dark after sunset and we cannot do anything
  • Proposed Solution 01
    • Produce sparks by rubbing two stones and burn woods to get fire
  • Strengths 
    • We can get light by burning woods
  • Limitations
    • It requires a lot of effort to rub stones to get sparks
    • It is not possible to move burning woods from one place to another 
  • Proposed Solution 02
    • Match sticks were made to get spark to burn woods
  • Strengths
    • It overcomes the problem of Rubbing Stones to get sparks
  • Limitations
    • The duration and quantity of light obtained by Match Sticks is very short
      • However, it is much better than Rubbing Stones Solution 😊
    • It is not possible to move burning woods from one place to another 
  • Proposed Solution 03
    • Candle
  • Strengths 
    • The duration and quantity of light obtained is much better than Match Sticks
    • We can easily take a Candle from one place to another and use its light to do tasks in the dark compared to Burning Woods
  • Limitations
    • Light generated by burning a Candle is 
      • For short time
      • In small quantity 
  • Proposed Solution 04
    • Bulb
  • Strengths 
    • The duration and quantity of light generated by a Bulb is much better than a Candle
  • Limitations
    • Bulb fuses very quickly
    • We need large power houses to generate current to enlighten a Bulb
  • Proposed Solution 05
    • Solutions to solve above problems are continuously proposed since the invention of the Bulb and will continue till the Day of Judgment In sha Allah😊

Example 02 – Research Cycle

  • Real-world Problem
    • How we can develop Intelligent Programs (Machine Learning Models) to perform various useful tasks
  • Proposed Solution 01
    • FIND-S Machine Learning Algorithm
  • Strengths 
    • We can develop an Intelligent Programs using FIND-S Machine Learning Algorithm
  • Weaknesses 
    • FIND-S Machine Learning Algorithm 
      • is a very simple algorithm and cannot handle complex Machine Learning Problems
      • cannot learn from noisy Data and Real-world Data is noisy
      • returns only one possible solution to a problem, but there can be many
  • Proposed Solution 02
    • Version Space Machine Learning Algorithms 
  • Strengths
    • returns all possible solution to a problem
  • Weaknesses 
    • Version Space Machine Learning Algorithm
      • is a very simple algorithm and cannot handle complex Machine Learning Problems
      • cannot learn from noisy Data and Real-world Data is noisy
  • Proposed Solution 03
    • Decision Tree Learning Algorithms 
  • Strengths 
    • Can handle complex Machine Learning Problems 
    • Can learn from noisy Data
  • Weaknesses
    • Cannot handle very complex Numeric Representations of Data
    • Cannot handle very complex Machine Learning Problems 
  • Proposed Solution 04
    • Artificial Neural Networks 


  • Strengths 
    • Can handle noisy Data
    • Can handle very complex Machine Learning Problems
    • Can handle very complex Numeric Representation of Data
  • Weaknesses
    • Artificial Neural Networks (particularly Deep Learning Models) require high computational resources
    • Artificial Neural Networks (particularly Deep Learning Models) require high Training Time
  • Proposed Solution 05
    • Research community is continuously doing Research to solve the above-mentioned problems and it will continue till the Day of Judgment In sha Allah😊

TODO and Your Turn​

Todo Tasks
Your Turn Tasks
Todo Tasks

TODO Task 1

  • Task 1
    • The first Light Bulb was made in 1879. A picture of how Light Bulb looked like in 1879 is given below

         

    • Note
      • Your answer should be 
        • Well Justified
    • Questions
      • Write Input and Output for a Light Bulb?
      • Write a short Biography of Thomas Edison (one Page) and list down the lessons you learned from it?
      • How many attempts Thomas Edison made before he succeeded in making a Light Bulb? What lesson you learned from his attempts?
    • Execute the Research Cycle for Light Bulb from 1879 to 2020. Also, put Pictures of Light Bulbs (with Year) whenever there was a major breakthrough in the technology?
Your Turn Tasks

Your Turn Task 1

 

  • Task 1


    • Select a Task (similar to Light Bulb) and answer the questions given below. 
    • Questions
      • Write Input and Output for the selected Task?
      • Write a short Biography of the inventor of the Task and list down the lessons you learned from it?
      • Execute the Research Cycle for selected Task. 
        • If possible, also put Pictures of selected Task (with Year) whenever there was a major breakthrough in the technology?

Representing Training Examples and Hypothesis for FIND-S Machine Learning Algorithm

  • Learning Input-Output Functions – General Settings
  • Input to Learner
    • Set of Training Examples (D)
    • Set of Functions / Hypothesis (H)
  • Output by Learner 
    • A Hypothesis (h) from H which best fits the Training Examples (D)
      • Note that h is an approxicmation of Target Functino f
  • In this Chapter
    • Learner is FIND-S Machine Learning Algorithm 😊
  • Chapter Focus
  • In sha Allah (انشاء اللہ), in this Chapter, I will take the Gender Identification Problem and try to explain three main things
    • Representation of Training Examples (D)
      • How to represent Training Examples (D) in a Format which FIND-S Algorithm can understand and learn from them?
    • Representation of Hypothesis (h)
      • How to represent Hypothesis (h) in a Format which FIND-S Algorithm can understand?
    • Searching Strategy
      • What searching strategy is used by FIND-S Algorithm to find a h from H, which best fits the Training Examples (D)?
  • Gender Identification Problem
  • Machine Lerning Problem
    • Gender Identification
  • Input 
    • Human
  • Output
    • Gender of a Human
  • Task
    • Given a Human (Input), predict the Gender of the Human (Output)
  • Treated as
    • Learning Input-Output Function 
      • i.e., Learn from Input to predict Output
  • Representation of Examples
  • Example = Input + Output
    • Input = Human
    • Output = Gender
  • Representatino of Input and Output
    • Attribute-Value Pair
  • Representation of Input
  • Input is represented as a 
    • Set of 6 Input Attributes
      • Height
      • Weight
      • HairLength
      • HeadCovered
      • WearingChain
      • ShirtSleeves

  • Representation of Output
  • Input is represented as a 
    • Set of 1 Output Attribute
      • Gender

  • Note 
    • Yes means Female and No means Male
  • Computing Size of Instance Space (X)

  • Computing Size of Instance Space (X) 
    • |X| = No. of Values of Height Input Attribute x No. of Values of Weight Input Attribute x No. of Values of HairLength Input Attribute x No. of Values of HeadCovered Input Attribute x No. of Values of WearingChain Input Attribute x No. of Values of ShirtSleeves Input Attribute 
    • |X| = 3*2*2*2*2*2 = 96
  • Sample Data
  • We obtained a Sample Data of 6 examples

  • Representation of Hypothesis (h)
  • We represent a Hypothesis (h) as
    • Conjunction (AND) of Constraints on Input Attributes
  • Each constraint can be:
    • No value allowed (null hypothesis Ø): e.g.             Height = Ø
    • A specific value : e.g.                                                  Height = Short
    • A don’t care value (any of possible values): e.g.   Height = ?
  • Representation of Hypothesis (h) Cont...
  • Most Specific Hypothesis (h) 

< Height   Weight   HairLength   HeadCovered   WearingChain   ShirtSleeves >

<     ∅             ∅               ∅                        ∅                         ∅                         ∅           >

  • Most General Hypothesis (h)

< Height   Weight   HairLength   HeadCovered   WearingChain   ShirtSleeves >

<      ?             ?                ?                          ?                          ?                          ?           >

  • Another Hypothesis (h)

< Height     Weight    HairLength   HeadCovered   WearingChain   ShirtSleeves >

< Normal     Light               ?                        ?                         No                       ?           >

  • Importance Note
    • The order of Input Attributes must be exactly same in 
      • Training Example (d) and Hypotheis (h)
  • Computing Size of Concept Spane (C) and Hypothesis Space (H)

  • Size of Instance Space (X)
    • |X| = 96
  • Size of Concept Space (C)
    • |C| = 2|X| = 2|96| = 79,228,162,514,264,337,593,543,950,336
  • Size of Hypothesis Space (H) (Syntactically Distinct Hypothesis)
    • |H| = 5*4*4*4*4*4 = 5120
  • Size of Hypothesis Space (H) (Semantically Distinct Hypothesis)
    • |H| = 1+4*3*3*3*3*3 = 973

FIND-S Algorithm - Machine Learning Cycle

  • Machine Learning Cycle
  • Four phases of a Machine Learning Cycle are
    • Training Phase
      • Build the Model using Training Data
    • Testing Phase
      • Evaluate the performance of Model using Testing Data
    • Application Phase
      • Deploy the Model in Real-world, to make prediction on Real-time Unseen Data
    • Feedback Phase
      • Take Feedback form the Users and Domain Experts to improve the Model
  • Split the Sample Data
  • We split the Sample Data using Random Split Approach into
    • Training Data – 2 / 3 of Sample Data
    • Testing Data – 1 / 3 of Sample Data
  • Sample Data

  • Training Data

  • Testing Data

  • Note
  • Sample Data is balanced
    • 3 Postivie Instances (Female)
    • 3 Negative Instances (Male)
  • After splitting Sample Data using Random Split Approach  
    • Training Data is unbalanced
      • 3 Postivie Instances (Female)
      • 1 Negative Instances (Male)
    • Testing Data is unbalanced
      • 0 Postivie Instances (Female)
      • 2 Negative Instances (Male)
  • Sample Data – Vector Representation
  • Vector Representation of Examples

               x1= <Short, Light, Short, Yes, Yes, Half> +

               x2= <Short, Light, Long, Yes, Yes, Half> +

              x3 = <Tall, Heavy, Long, Yes, Yes, Full> –

              x4 = <Short, Light, Long, Yes, No, Full> +

              x5 = <Short, Light, Short, Yes, Yes, Half> –

             x6 = <Tall, Light, Short, No, Yes, Full> –

  • Training Data – Vector Representation
  • Vector Representation of Training Examples

               x1= <Short, Light, Short, Yes, Yes, Half> +

               x2= <Short, Light, Long, Yes, Yes, Half> +

              x3 = <Tall, Heavy, Long, Yes, Yes, Full> –

              x4 = <Short, Light, Long, Yes, No, Full> +

  • Testing Data – Vector Representation
  • Vector Representation of Test Examples

               x5 = <Short, Light, Short, Yes, Yes, Half> –

              x6 = <Tall, Light, Short, No, Yes, Full> –

  • Find-S Algorithm (or Learner)
Source : Tom Mitchel Book
  • Specific to General Constraints
  • We have three constraints on our Attributes
    • No value allowed (Ø)
      • Most Specific Constraint
    • A set of specific values (for e.g. Short, Normal and Tall for Height Attribute)
      • Note that specific value is next more generic constraint then  No value allowed (Ø)
    • A don’t care value (?)
      • Note that ? is next more generic constraint then  specific value 
  • Training Phase
  •  In the Training Phase, the FIND-S Algorithm will 
    • Seearch H to find out a h, which best fits the Training Data
  • Best Fit means
    • h correctly classifies Positive and Negative instances in the Training Data
  • Correct Classification
    • Postive instance is classified as Positive
    • Negative instance is classified as Negative
  • Inorrect Classification
    • Negative instance is classified as Positive
    • Postive instance is classified as Negative
  • Initialize h to the Most Specific Hypothesis in H
      • h0 = <∅ , ∅ ,∅ ,∅ ,∅ , ∅>
  • For each positive training instance x
      • For each attribute constraint ai in h
      •   If the constraint ai in h is satisfied by x
      •         then do nothing
      •      else replace ai in h by the next more general constraint that is satisfied by x
  • First Training Example
    • x1 = <Short, Light, Short, Yes, Yes, Half> +
  • Let’s see if attribute constraints in h0 satisfy x1 or not?

  • As we can see that attribute constraints in h0 do not satisfy x1
      • Therefore, x1 is incorrectly classified as Negative
  • To satisfy x1, we will need to replace attribute constaints in h0 by the next more general constraint that is satisfied by x1
  • h0 = <∅ , ∅ ,∅ ,∅ ,∅ , ∅> will become
    • h1 = <Short, Light, Short, Yes, Yes, Half>
  • First Training Example
    • x1 = <Short, Light, Short, Yes, Yes, Half> +
  • Let’s see if attribute constraints in h1 satisfy x1 or not?

  • As we can see that attribute constraints in h1 satisfy x1
      • Therefore, x1 is correctly classified as Positive
  • Second Training Example
    • x2 = <Short, Light, Long, Yes, Yes, Half> +
  • Let’s see if attribute constraints in h1 satisfy x2 or not?

  • As we can see that attribute constraints in h1 do not satisfy x2
      • Therefore, x2 is incorrectly classified as Negative
  • To satisfy x2, we will need to replace attribute constaints in h1 by the next more general constraint that is satisfied by x2
  • h1 = <Short, Light, Short, Yes, Yes, Half> will become
    • h2 = <Short, Light, ?, Yes, Yes, Half>
  • Second Training Example
    • x2 = <Short, Light, Long, Yes, Yes, Half> +
  • Let’s see if attribute constraints in h2 satisfy x2 or not?

  • As we can see that attribute constraints in h1 satisfy x2
    • Therefore, x2 is correctly classified as Positive
  • Note
    • Learner (FIND-S Algorithm) has observed two Training examples up till now and our hypothesis is as follows
      • h2 = <Short, Light, ?, Yes, Yes, Half>
    • Let’s see if h2 must best fit the observed Trailing Example i.e. x1 and x2 or not?
      • h2 correctly classifies 
        • x1 as Positive 
        • x2 as Positive 
    • To conclude, h2 best fits’ the first two observed Trailing Example i.e. x1 and x2
  • Third Training Example
    • x3 = <Tall, Heavy, Long, Yes, Yes, Full> –
  • Note that 3rd Training Example is Negative and 
    • FIND-S only operates on Positive Training Examples
  • Therefore, there will be on change in h2 and h3
  • h2 = <Short, Light, ?, Yes, Yes, Half> will become 
    • h3 = <Short, Light, ?, Yes, Yes, Half>
  • Interesting h3 correctly classifies x3 as Negative

  • Third Training Example
    • x3 = <Tall, Heavy, Long, Yes, Yes, Full> –
  • h3 correctly classifies 
    • x1 as Positive 
    • x2 as Positive 
    • x3 as Negative 
  • Thus, h3 best fits the three Training Examples observed up till now
  • Fourth Training Example
    • x4 = <Short, Light, Long, Yes, No, Full> +
  • Let’s see if attribute constraints in h3 satisfy x4 or not?

  • As we can see that attribute constraints in h3 do not satisfy x4
      • Therefore, x4 is incorrectly classified as Negative
  • To satisfy x4, we will need to replace attribute constaints in h3 by the next more general constraint that is satisfied by x4
  • h3 = <Short, Light, ?, Yes, , Half>
    • h4 = <Short, Light, ?, Yes, ?, ?>
  • Fourth Training Example
    • x4 = <Short, Light, Long, Yes, No, Full> +
  • Let’s see if attribute constraints in h4 satisfy x4 or not?

  • As we can see that attribute constraints in h4 satisfy x4
      • Therefore, x4 is correctly classified as Positive
  • h4 correctly classifies 
    • x1 as Positive 
    • x2 as Positive 
    • x3 as Negative 
    • x4 as Positive 
  • Thus, h4 best fits the three Training Examples observed up till now
  • Note
    • There were total 4 Training Examples and we have observed all of them
  • FIND-S Algorithm

  • After observing all the Training Example, the FIND-S Algorithm will 
      • Output hypothesis h
  • The h returned by FIND-S Algorithm is
    • h = <Short, Light, ?, Yes, ?, ?>
  • Note that h is an approximation of the Target Function f
  • Recall

  • Training Data
    • x1 = <Short, Light, Short, Yes, Yes, Half> +
    • x2 = <Short, Light, Long, Yes, Yes, Half> +
    • x3 = <Tall, Heavy, Long, Yes, Yes, Full> –
    • x4 = <Short, Light, Long, Yes, No, Full>
  • Model
    • h = <Short, Light, ?, Yes, ?, ?>
  • Model – in the form of Rules

  • In the next phase i.e. Testing Phase, we will 
    • Evaluate the performance of the Model
  • Testing Phase
  • Question 
    • How good Model has learned?
  • Answer
    • Evaluate the performance of Model on unseen data (or Testing Data)
  • Evolution Measures
  • Evaluation will be carried out using 
    • Error measure
  • Error
  • Definition
    • Error is defined as the proportion of incorrectly classified Test instances
  • Formula

  • Note
    • Accuracy = 1 – Error
  • Evaluate Model
    •  
  • Applying Model on x5

  • Prediction returned by Model
    • x5 is predicted Positive (Incorrectly Classified Instance)
  • Applying Model on x6

  • Prediction returned by Model
    • x6 is predicted Negative (Correctly Classified Instance)
  • Evaluate Model, Cont…

  • Application Phase
  • We assume that our Model 
    • performed well on large Test Data and can be deployed in Real-world
  • Model is deployed in the Real-world and now we can make
    • predictions on Real-time Data
  • Steps – Making Predictions on Real-time Data
  • Step 1: Take Input from User
  • Step 2: Convert User Input into Feature Vector
  • Exactly same as Feature Vectors of Training and Testing Data
  • Step 3: Apply Model on the Feature Vector
  • Step 4: Return Prediction to the User

 

  • Example - Making Predictions on Real-time Data
  • Step 1: Take input from User

  • Step 2: Convert User Input into Feature Vector
    • Note that order of Attributes must be exactly same as that of Training and Testing Examples

 

Step 3: Apply Model on the Feaure Vector 

  • Step 4: Return Prediction to the User
    • Positive 
  • Note 
    • You can take Input from user, apply Model and return predictions as many times as you like 😊
  • Feedback Phase
  • Only Allah is Perfect 😊
  • Take Feedback on your deployed Model from
    • Domain Experts and
    • Users
  • Improve your Model based on Feedback 😊
  • Inductive Bias - FIND-S Algorithm
  • Inductive Bias 
    • Inductive Bias Is the set of assumptions needed in addition to Training Examples to justify Deductively Learner’s Classification
  • Inductive Bias of FIND-S Algorithm
    • Training Data is error free
    • Target Function / Concept is present in the Hypothesis Space (H)
  • Training Regime- FIND-S Algorithm
    • Incremental Method 
      • See Chapter 3 – Basics of Machine Learning
  • Strengths and Weakness - FIND-S Algorithm
  • Strengths
    • Returns a Model (h), which can be used to make predictions on unseen data
  • Weaknesses
      • Only works on error free Data
        • However, Real-world Data is noisy
      • Works on assumption that Target Function is present in the Hypothesis Space (H)
        • However, we may / may not find the Target Function in the Hypothesis Space (H) and this may / may not be known
      • Only returns one hypothesis which best fits the Training Data
        • However, there can be multiple hypothesis, which best fit the Training Data

TODO and Your Turn​

Todo Tasks
Your Turn Tasks
Todo Tasks

TODO Task 2

  • Task 1
    • Consider the Titanic Dataset with the following Attributes
  • Gender: Male, Female
  • Ticket Class: Upper, Middle, Lower
  • Parent/Child Abroad: Zero, One, Two, Three
  • Embarked: Cherbourg, Queenstown, Southampton
  • Survival: No, Yes
    • We obtained the following Sample Data

  • Sample Data was split into Training and Testing Data in a 
    • Train-Test Split Ratio of 67%-33%
  • Training Data

  • Testing Data

  • Note
    • Consider FIND-S Algorithm when answering questions given below
    • Your answer should be 
      • Well Justified
  • Questions
    • Write down the Input and Output for the above Machine Learning Problem?
    • How Training Example is represented?
    • How Hypothesis (h) should be represented?
    • Calculate Size of Instance Space, Concept Space, Syntactically Distinct Hypothesis and Semantically Distinct Hypothesis?
    • Executer the Machine Learning Cycle?
  • Write down your observations that you observed during the execution of Machine Learning Cycle?
Your Turn Tasks

Your Turn Task 2

 

  • Task 1
    • Select a Machine Learning Problem (similar to: Titanic – Machine Learning form Disaster) and answer the questions given below.
    • Note
      • Consider FIND-S Algorithm in answering all the questions.
    • Questions
      • Write down the Input and Output for the selected Machine Learning Problem?
      • How Training Example is represented?
      • How Hypothesis (h) should be represented?
      • Calculate Size of Instance Space, Concept Space, Syntactically Distinct Hypothesis and Semantically Distinct Hypothesis?
      • Executer the Machine Learning Cycle?
      • Write down your observations that you observed during the execution of Machine Learning Cycle? 

Chapter Summary

  • Chapter Summary
  • Learning is a Searching Problem
  • Research is defined as a systematic investigation of sources and materials , to develop solutions for Real-world Problems to improve the quality (peace and happiness) of human life 
  • A Researcher is a man of character and (s)he should safeguard his character 
  • The word Research is made up of
    •  Re and
    •  search 
  • Therefore, in Research, we mainly refine the solution(s) proposed for a Real-world Problem
  • The main steps of a Research Cycle are as follows
    • Step 1: Identify the Real-world Problem 
    • Step 2: Propose Solution (called Solution 01) to solve the Real-world Problem
    • Step 3: List down Strengths and Weaknesses of Solution 01
    • Step 4: Propose Solution (called Solution 02) to 
      •  further strengthen the Strengths of Solution 01
      •  overcome limitations of Solution 01
    • Step 5: List down Strengths and Weaknesses of Solution 02
    • Step 6: Propose Solution (called Solution 03) to 
      •  further strengthen the Strengths of Solution 02
      •  overcome limitations of Solution 02
    • Step 7: Continue this cycle till the Day of Judgment 😊
  • Considering FIND-S Algorithm 
    • Input to Learner (FIND-S Algorithm)
      • Set of Training Examples (D)
      • Set of Functions / Hypothesis (H)
    • Output by Learner (FIND-S Algorithm)
      • A Hypothesis (h) from H which best fits the Training Examples (D)
        • Note that h is an approximation of Target Function
  • Inductive Bias Is the set of assumptions needed in addition to Training Examples to justify Deductively Learner’s Classification
  • FIND-S Algorithm – Summary
    • Representation of Example
      • Attribute-Value Pair
    • Representation of Hypothesis (h)
      • Conjunction (AND) of Constraints on Attributes
    • Searching Strategy
      • I am not clear about this. Please drop me an email if you know. Jazak Allah Khair
    • Training Regime
      • Incremental Method
    • Inductive Bias of FIND-S Algorithm
      • Training Data is error-free 
      • Target Function / Concept is present in the Hypothesis Space (H)
    • Strengths
      • Returns a Model (h), which can be used to make predictions on unseen data
    • Weaknesses
      • Only works on error-free Data
        • However, Real-world Data is noisy 
      • Works on assumption that Target Function is present in the Hypothesis Space (H)
        • However, we may / may not find the Target Function in the Hypothesis Space (H) and this may / may not be known 
      • Only returns one hypothesis which best fits the Training Data
        • However, there can be multiple hypothesis , which best fit the Training Data

In Next Chapter

  • In Next Chapter
  • In Sha Allah, in next Chapter, I will present
  • Version Space Algorithm
Chapter 7 - Concept Learning and Hypothesis Representation
  • Previous
Chapter 9 - Version Space Algorithms
  • Next
Share this article!
Facebook
Twitter
LinkedIn
About Us

Ilm O Irfan Technologies believes that honest dedicated hard work with the ethical approach of commitment true to every word, and trust-building is the road to success in any field of business or life.

Quick Links
  • About Us
  • Contact
Useful Links
  • Privacy Policy
  • Terms and Conditions
  • Disclaimer
  • Support
  • FAQ
Subscribe Our Newsletter
Facebook Twitter Linkedin Instagram Youtube

© 2022 Ilmo Irfan. All Rights Reserved.