3.1 Rules vs. Learning

To understand how machine learning works, it's a good idea to start with the traditional and maybe alternative approach: rules.

In this exercise, we are going to use the popular MNIST data set of handwritten digits to work out the problems when we try to define rules for a seemingly simple task: recognize and classify handwritten digits.

a) Images in a computer

Search online for version of the MNIST data set in CSV format. If you can't find one, you can refer to this link. Download the data set, extract the file, and open it in a text editor.

• What does an image look like to a computer?

• What do all the numbers in one line mean?

b) Visualize a handwritten digit

To get a better understanding of how a computer sees and displays images, add a feature to your web app that takes a representation of a digit as an array of numbers and display it on the website pixel by pixel. Take a look at the p5.js library and especially the following functions:

• createCanvas()

• square()

• stroke()

• fill()

Additionally, if you want to add the rows and column labels:

• text()

• textSize()

You can use the following array of the number 3 shown below as your input data:

// An array of 784 numbers that represent a handwritten 3
var digit = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,12,99,91,142,155,246,182,155,155,155,155,131,52,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,138,254,254,254,254,254,254,254,254,254,254,254,252,210,122,33,0,0,0,0,0,0,0,0,0,0,0,0,220,254,254,254,235,189,189,189,189,150,189,205,254,254,254,75,0,0,0,0,0,0,0,0,0,0,0,0,35,74,35,35,25,0,0,0,0,0,0,13,224,254,254,153,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,90,254,254,247,53,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,6,152,246,254,254,49,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,158,254,254,249,103,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,54,251,254,254,254,248,74,5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,140,254,254,254,254,254,254,202,125,45,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,58,181,234,254,254,254,254,254,254,252,140,22,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,30,50,73,155,253,254,254,254,254,191,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,91,200,254,254,254,254,118,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,192,254,254,254,154,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,141,254,254,254,116,0,0,0,0,0,0,0,0,0,0,0,0,0,25,126,86,0,0,0,0,0,0,3,188,254,254,250,61,0,0,0,0,0,0,0,0,0,0,0,0,24,209,254,15,0,0,0,0,0,23,137,254,254,254,209,0,0,0,0,0,0,0,0,0,0,0,0,0,168,254,254,48,9,0,0,9,127,241,254,254,255,242,63,0,0,0,0,0,0,0,0,0,0,0,0,0,101,254,254,254,205,190,190,205,254,254,254,254,242,67,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,166,254,254,254,254,254,254,254,254,250,138,55,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,7,88,154,116,194,194,154,154,88,49,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0];

A handwritten digital 28 x 28 pixel.

c) Rules, rules, rules

Now that we have a better understanding of how a computer represents images, let's try to come up with rules that allow us to recognize the particular digit behind a long list of numbers. For simplicity, start with only the number 3 and write a program that outputs true or false. true meaning the list of numbers is a 3, and false it is not. This is called binary classification because we only distinguish between two answers.

• How can you teach this seemingly easy task (for humans) to a computer?

• Why is it so difficult to articulate the rules that determine whether it's a 3 or not?