Machine Learning - Introduction
The term “Machine Learning” learning was coined somewhere in the 1950s. However, of late, there has been a lot of exploration, analysis etc. around Machine Learning. The Google Trend graph in Figure 1 justifies this well enough.
Figure 1 : Google Trend graph showing ML is trending
Are you keeping up with new developer technologies? Advance your IT career with our Free Developer magazines covering C#, Patterns, .NET Core, MVC, Azure, Angular, React, and more. Subscribe to the DotNetCurry (DNC) Magazine for FREE and download all previous, current and upcoming editions.
So, what is Machine Learning (ML)?
Well, learning anything is challenging. Building machines is challenging too. So, making machines learn would sound very challenging as well!
Isn’t it so?
Let’s first find out why do machines need to learn in the first place.
We have so many machines or gadgets around us that are used for various purposes. They help us perform different tasks. Let’s take a simple example.
You may usually have a nicely brewed coffee every morning. So, you typically would be using a coffee maker “machine” that makes coffee after you have selected your options - like what kind of coffee you’d like to have, how much milk, how much sugar and so on.
Now, what if, the “machine” learns how to make the coffee for you based on your daily routine.
Wouldn’t that be cool? Indeed, it would be!
So, if your coffee machine is taught to study your routine behavior so that you get your daily coffee exactly how you like it, it will make your life easier. If the coffee machine can also identify you through face recognition and can make the coffee that you like (and not how your friend/companion likes), that would be really awesome!
That’s machine learning for you!
In a nutshell, machine learning can be applied to situations where:
1) A machine is used for a specific task to be completed - obviously. The machine term here is an all-encompassing entity as it would involve some app/program/appliance/device/system that is being used.
2) The behavior is repeatable and predictable so that past data can be used to predict the future actions.
3) Behavior is pattern-based or rule-driven so that it can be “taught” to the machine. That will help machines to learn and match data against a pattern to take actions or decisions.
4) Large volume of data is being processed. This will typically hold true when it is humanly impossible to look at the specific data elements in the sea of data to identify a potential issue or problem.
Why is ML so important now?
If ML was existing as a concept from 1950s, then one would wonder as to why has it become so important or has come into vogue?
Well, amongst many others, here are some key driving factors:
1) Connectivity of machines: Generally speaking the internet connectivity aspect or ability for devices to be online or simply put, Internet of Things (IoT) has made it easy for devices to be connected with each other (and servers) in real-time. So the data that was lying with machines now, is available for processing over cloud.
2) Data – lots of it: Devices/machines collect a lot of data in real time and with the connectivity available, they can now transmit all this data in real time. By leveraging all of this data, we can gain insights into user behavior. The ever-expandable elastic storage power through the cloud makes it easier to handle large volumes of data and process it on the cloud.
3) Processing power: Moore’s law still holds true. Devices have got enough processing power now to handle loads of data at the edge (i.e. on the device).
All these three aspects put together have resulted into an exciting opportunity to apply ML techniques to solve business problems.
ML is touching your lives, already!
If you are thinking that ML is some futuristic Sci-Fi stuff, then it is not!
You are already living in a digital world where ML is touching your lives on a daily basis. You don’t believe?
Here are some examples of ML being used already:
1) If you have a Gmail account, Google’s Priority Inbox features uses ML to categorize the mails as priority/spam from the mail avalanche that hits your email account every single day. It learns over time as to which emails are important to you, and takes actions accordingly. It also uses ML to validate the email messages for phishing attack related threats.
2) Google’s self-driving car or for that matter the famous Tesla cars use ML extensively while driving itself by mapping what it “sees” to what it has learnt in terms of digitized imagery of the terrain around.
3) Online recommendation based on your purchases are pushed by Online commerce giants like Amazon while you surf for products. It also uses ML to provide the most competitive price to customers during online shopping. The same technique is being used when you book online tickets, to show online ads, or if you are into online dating websites.
There are many more similar examples where ML has already “arrived” in your lives. With more connected devices coming up, it is only going to get more and more integrated (and interesting) and eventually ML would become an integral part of your day-to-day life.
In many cases, this integration will be seamless. You won’t even notice it.
The Technical ABCs of ML
Let’s delve into some technical basics of ML now. Let’s start with the generic approach followed in any machine learning process.
Any ML process would have the following key steps:
1) Data collection & preparation: You will need to identify the right data sources and prepare data that can be fed to the ML algorithm to be used.
2) Learning: This includes a key aspect of choosing the right ML algorithm to be used to get the right results. You would use the dataset as the outcome of step 1 here, to feed to the model so that it learns accordingly. The aspects considered while choosing the right algorithm could be:
- Complexity of the data
- Choice of data set
3) Prediction: In this step, you would use the model or algorithm to predict outcomes for you based on the data model supplied or provided.
The following diagram depicts the flow:
Figure 2 : the ML Learning Process
Let’s look at the various ML techniques used currently.
Supervised Learning
In supervised learning, you build a model that can be used for prediction. When you have a function f(x) that predicts the result y for the given x data, then it is termed as supervised learning.
An important aspect to note here is the use of the right approximation algorithm as the function, so that it predicts new y values based on past data used to teach.
This process of an algorithm that is used to learn from an existing dataset is synonymous to a teacher supervising the learning process. So, we train the model with a known understanding about the kind of data we are using (i.e. the labelled data).
Hence it is termed as supervised learning.
Typically, the steps involved are:
1) Provide known/labeled data and known output to algorithm to learn, so that it creates the model.
2) The model coupled with the algorithm that has “learnt” would result into predicted response for a new data set entered.
Supervised Learning is used when you need an output in the following two forms:
1) Classification: You need the ML algorithm to output a category from given categories – something like priority mail or spam mail.
2) Regression: Over here, the ML algorithm would output a real value like predicted sales values based on input product prices.
Various techniques are used in supervised learning. Some of them are as follows:
1) Decision trees: the algorithm would use some threshold values for classifying the data.
2) Support Vector Machines: This involves binary classification of data
3) ANN (Artificial Neural Network): ANN concepts are used for data classification in this case.
Unsupervised Learning
When you have a cluster of data and you would like it to be analyzed to deduce conclusions or infer from it, then it is termed as unsupervised learning.
The basic premise here is that there is no specific end goal in mind and hence there is no “teaching the model” aspect involved.
The data is also not really labelled in this case since this is more of an exploration of figuring out the pattern, from an available dataset. So you are essentially taking a bulk of data and working on finding the hidden pattern by doing the cluster analysis of the data you have.
There are various algorithms used for doing this. Some of them are mentioned below:
- k-Means Clustering: this essentially partitions data in various distinct clusters
- Hierarchical clustering: the algorithm attempts to build a hierarchy of clusters in a tree-like fashion
- Self-organizing maps: it uses the concepts of neural networks to build the clusters from data
Semi-supervised Learning
This technique essentially addresses a specific need of machine learning where we have some understanding of output to be generated from input. However, this is not true for all the data. So, this technique is essentially a combination of supervised and unsupervised learning.
You can use various combinations of supervised and unsupervised learning algorithms to get desired result.
Reinforced Learning
This is another variation of ML where the concept of reward or penalty is applied to learning process. This is similar to students taking an examination in order to evaluate how the student (in this case the algorithm) has performed in the learning process.
Learning Machine Learning techniques by Example
Let’s try and understand the different types of ML learning techniques we just learnt about, with an example.
Let’s say that you want to build a machine learning algorithm for face detection. Here is how to apply the different ML methods to make this possible:
You have a dataset of images labelled as faces (based on attributes like round face, a specific area of photo representing skin color etc.). You can use this dataset to train the model to identify a face. You then provide a new image to the algorithm for recognizing if it is an image of a face, or otherwise.
The algorithm would logically compare the attributes of the test image with the images from which it has learnt how to recognize a face. Using this logic, the algorithm would predict if the image is a face or not. This is supervised learning.
If you don’t provide a labelled data, the ML algorithm would try to cluster (group) the data into face-like shapes (round shape) based on various properties defined in the model (the roundness of a shape, proximity of the shape to some circular figure etc.). Based on this clustered data, the ML algorithm would recognize the faces. This is unsupervised learning.
The third possibility is a mixed set of data with some labelled data and some unlabeled data.
So, when you would use labelled data (i.e. images labelled as faces), the ML algorithm will first learn from labelled data. Then you can use the trained model to predict labels from unlabeled data (i.e. images that are not labelled as faces). This is called pseudo-labelled data. And then you use both labelled and pseudo-labelled data, to retrain the model. This is semi-supervised learning.
The fourth possibility is that you use an algorithm to recognize the face first and then have the algorithm decide the success of recognizing the face by coming up with some success coefficient. If the coefficient is closer to some value, then it means that the algorithm has succeeded, else it has to re-evaluate. This is at a very high level, reinforced learning process.
Deep Learning
A quick note here – Deep Learning (DL) is a term used quite often alongside Machine Learning.
Deep Learning is a subset of ML and closely tries to mimic the structures (neurons) utilized by our brains. It is essentially taking ML to the next level since DL algorithms go really deep in terms of handling data. It is a specialized way of handling data where in the learning part is more human-like in terms of execution process, as compared to ML.
Editorial Note: Deep learning is a subject of its own. To know more about what Deep Learning can really do, check out The Dark Secret at the Heart of AI.
Future of Machine Learning
Machine Learning has eventually matured as a specialization that compliments various key areas like analytics, AI and so on.
As explained earlier, the overall learning process is divided into three parts:
1) Data collection
2) Learning
3) Prediction
As things stand now, the data collection happens on the edge (i.e. device), whereas the data processing & learning happens on the cloud. This is traditionally how it has been envisioned.
However, things are supposed to get more interesting!
Once the learning from data is accomplished, the “learnt” logic can be pushed back to device/edge and then the prediction part can happen on the edge again. This kind of approach is very critical and important when:
1) The decision making based on ML needs to happen fast enough: Let’s consider a case where ML is used to predict failure of some component and based on that, some other decisions are required to be taken at that very instant. In this case, it is vital to have the learnt logic available at the edge for further decision making process.
2) Data privacy: In case of using ML in context of medical or healthcare use cases, it is prohibited to have the data move out of healthcare facility due to privacy issues or compliance restrictions. In such cases, the prediction needs to happen at the edge.
So, it is not going to be a distant future when the AI/ML combination is going to make the edge more intelligent. So, in a near future, after you drink a full cold bottle of your favorite fizzy drink from the refrigerator, the refrigerator would tell Alexa to order medicine for cough and cold for you because in some parts of the world, you always fall sick after drinking a cold drink!
Azure Machine Learning Studio
Enough of theory; let’s look at how you can actually build an Azure Machine Learning experiment yourself, using Azure Machine Learning Studio.
Editorial Note: Microsoft has long been at the forefront of Machine Learning and AI. After being successful at Hadoop-based Azure HDInsight and PowerBI for Office 365, Microsoft channeled its efforts to consumerize ML. In addition to solving existing business problems using ML, Microsoft also wanted to make available the breakthroughs of ML to those developers and engineers, who did not have a ML background.
Azure Machine Learning (also known as "Azure ML") was a step in that direction. Azure ML runs on the Azure public cloud and is built on top of the machine learning capabilities of several Microsoft products and services. ML Studio is a simple browser-based integrated development environment. It allows you to visually drag-and-drop elements where no to little coding is necessary.
Open up your favorite browser and go to http://studio.azureml.net. As goes with all Microsoft offerings, this is a DIY studio for you to get your hands dirty with ML.
You will need an Azure account to login. Get a free trial account at https://azure.microsoft.com/en-us/free/ if you don’t have one already.
Once you login, you will be presented with the following UI.
Figure 3 : Azure ML Studio Workbench
To follow a detailed documentation for ML Studio, head over to docs.microsoft.com/en-us/azure/machine-learning/studio/.
Azure ML Studio organizes things in the following manner:
1) Projects: This is like any other project in Visual Studio that will have all the required nuts and bolts for you to perform your ML tasks.
2) Experiments: A project would contain multiple experiments that you would like to carry out. You can add various datasets and modules. You need to have at least one dataset and one module in the experiment.
3) Datasets: Azure ML Studio has some sample datasets available that you can play around with.
4) Module: This is essentially an algorithm that you would use for analyzing your datasets.
So you get the drift now. Let’s create our first experiment. Please follow the step-by-step guide below:
Step 1: Create a new experiment: Click on the “New” button on the bottom left corner of the screen and then select “Blank Experiment”.
Figure 4 : Create an Experiment
You can also go through the Experiment Tutorial or if you would like to check out some sample experiments, you can select the appropriate options. Azure ML Studio has provided a lot of samples as shown in Figure 5.
Figure 5 : Azure ML Studio Samples
After clicking on Blank Experiment, you will be presented with the following workbench:
Figure 6 : Creating a Blank Experiment
Step 2: Now, for us to understand Machine Learning well enough, we need to apply the concepts of ML explained earlier, to solve some specific real-life problem.
As mentioned earlier, one of the key pre-requisites for ML is “data”. So, we need to first get some real-life data in order to apply ML and analyze a real-life problem.
There are many data sets available online for free that you can choose from and then you will have to import them into the Azure ML Studio. Alternatively, you can pick up one of the data sets from Azure in order to understand how ML works.
Step 3: Let us pick up the data set “Energy Efficiency Regression Data” from the available sample datasets in Azure ML Studio. If you want to pick up some other data set, then please go through docs.microsoft.com/en-us/azure/machine-learning/studio/use-sample-datasets that provides a brief background about the various sample data sets provided by Azure.
Let us type in “energy” in the search box. That will bring up Energy Efficiency Regression data as shown in Figure 7. Let’s drag it and drop it on to the workspace.
Figure 7 : Choosing a Data Set
The workspace should look something like this:
Figure 8 : Data Set is chosen
Please note that you can also import the data from your data source. You can refer to Azure ML Studio documentation that explains this well.
Step 4: The sample data set that we have chosen is a collection of energy profiles for twelve different buildings of various shapes.
The key objective here is to use ML technique to help us assess energy efficiency of these buildings. For this, we will need to understand the factors that affect the heating or cooling of the building. I would request you to skim through a paper (people.maths.ox.ac.uk/tsanas/Preprints/ENB2012.pdf) to get an in-depth idea of the problem we are trying to solve by using ML.
Step 5: Before we proceed, we can take a peek at the sample data set. To do so, you can right click on the data set and click on “Visualize”.
Figure 9 : Visualize Data Set
The following data gets displayed:
Figure 10 : Visualized Data Set
Step 6: Now you can view statistics and visualization for each column. Just click on any column and you will see the details about that column updated in Statistics” and “Visualization” sections as shown in Figure 11:
Figure 11 : Data Set Statistics
I have chosen another column (Grazing Area Distribution) and after scrolling down a bit, I see the data visualized in a bar graph as shown in Figure 12.
Figure 12 : Viewing Data Set Statistics for other columns
Step 7: We will select some columns from the dataset, for further processing. To do so, you can enter “select column” in the search box and then drag the “Select Column in Dataset” module onto the workspace. Your screen should appear similar to Figure 13:
Figure 13 : Using Select Column in Dataset module
After I click on the data module, it shows me the following items:
Figure 14 : Select Column in Dataset chosen
We will join the boxes and as you can observe in Figure 15, there is a red exclamation icon that appears.
Figure 15 : Select Column in Dataset with error icon
The red icon means that I need to provide additional information to proceed further. We will click on “Launch column selector” as shown in Figure 16 to select columns.
Figure 16 : Launching Column Selector
Doing so brings up a dialog box (Figure 17) with all the data columns shown towards the left. As we choose them, they will move to right. This is self-explanatory and a standard operation. Basically, we are choosing the columns that we want to use to build our model.
These are also called as “features” in ML parlance.
We are leaving out “Cooling Load” for now because we are going to predict only “Heating Load” (refer to the research paper for details).
Figure 17 : Selecting features
Step 8: Now close the dialog box after clicking on the tick mark on the bottom right. Now click on the “Run” button at the bottom. This will make Azure ML Studio save and run the model.
If everything goes well (and it will), you will see the model run successfully with a green tick as seen in Figure 18.
Figure 18 : Running the Experiment
You can double-click on the Select Columns in the Dataset box and add a note as shown in Figure 19.
Figure 19 : Adding Notes
After this, the workspace will appear as shown in Figure 20:
Figure 20: Note is added
Step 9: Now we need to choose a learning algorithm.
In this case, since we want to predict the outcome based on the available parameters or features, we will need to choose a supervised learning algorithm (since we are going to use data to train the model and predict outcome).
Typically, there are two types of supervised learning algorithms viz. classification – used for predicting an answer from a defined set of categories or regression – used for predicting a specific outcome in the form of a number based on available data.
We will use the regression algorithm here. Enter “linear regression” and then drag the linear regression module onto the workspace as shown in Figure 21.
Figure 21 : Choosing the Learning Algorithm
Step 10: Usually for any ML experiment, you would need data to train the model, and then data to test the model upon.
In this case, we can use the same data set. To do so, we need to split data. Let’s use 80% of our data to train the model and 20% of it to test the model. We can do so by dragging the “Split Data” module on the workspace as follows:
Figure 22 : Splitting Data
Step 11: Now add the “Train model” on to the workspace and connect the ‘Linear Regression’ module and ‘Split Data’ module to it. This means that we are going to use the linear regression algorithm and the data, to train the model. The outcome would be a trained model with this algorithm and data set.
This is shown in Figure 23.
Figure 23 : Training the Model
Notice the red dot again.
This means that we need to let the model know which column it needs to predict the parameter for, using the algorithm we chose. Open up “Launch column selector” and then select “Heating Load” since this is what we are planning to predict.
Figure 24 : Selecting Parameter to predict
Step 12: After this, let’s run the model again to ensure that everything is working well. Once this is done, you should see the following result:
Figure 25: Running the Model
Figure 26 shows all the steps so far:
Figure 26: Experiment Steps executed so far
Step 13: Now comes the real interesting bit to see how this has all shaped up.
We will predict the Heating Load based on our data and algorithm. For this, we need to use “Score Model” module. We will use following inputs for this:
- a) trained model from ‘Train Model’ module
- b) sample data (remember, the rest 20% data we kept aside).
Once we do that, the workspace would look like the following:
Figure 27 : Running the Experiment again
Step 14: Click on “Visualize” and it will bring up the results. See Figure 28.
Figure 28: Visualizing the output
Here is the output:
Figure 29 : Final Output
In the screenshot shown in Figure 29, the “Scored Labels” column has the predicted Heating Load.
The following screenshot shows a comparison of Heating Load value in the dataset vs. the same being predicted using the ML algorithm we used.
You can also evaluate the correctness of the results by using “Evaluate model” module. I will leave this step as a short exercise for you to perform.
Figure 30 : Comparing the output
This short step-by-step tut should have given you a glimpse of what all you can do with Azure ML Studio. Now you can play around with various modules/steps in the ML process and check out the results yourself in terms of how various changes results in corresponding changes in the final outcome. You can also try different data sets, as well as different ML techniques.
Conclusion
We are teaching machines to learn so that they can start taking certain decisions for us.
In the near future, we’ll have a self-driving car, a more advanced email client that will respond to our mails and what not. Does it sound scary or exciting?
Well, it depends on how you perceive it. One thing is certain for sure, as developers, we need to continue to learn new things to outsmart machines.
Be it humans or machines, learning never stops!