Recent developments in artificial intelligence (AI) have given rise to two main categories of algorithms: supervised learning and unsupervised learning. Supervised learning requires you to give it lots of examples of correct answers so that it can learn how to classify new examples as being part of the class or not.
Unsupervised learning does not require you to tell it which classes there are, but instead looks at data sets without labels and tries to determine what structures and patterns exist within them. More advanced forms of this kind of algorithm use concepts such as clustering or dimensionality reduction to gain insights into the space of all possible datasets.
Deep neural networks are an example of a type of algorithm that uses both types of learning together. They begin with large layers of supervised learning where the network is shown many different pictures or videos and taught to identify specific objects or actions. Then these earlier layers are stripped away and replaced by larger, more flexible layers that perform either clustering or feature extraction depending on what they are told to do!
These deeper layers then connect to other, newer layers that perform classification again, using the extracted features and the learned lessons from the first few. The whole thing is trained off of huge amounts of data so that when it is used for real applications, it will work almost automatically.
There are some who claim that deep learning is simply machine learning run very deeply, but this ignores the importance of transfer learning.
Definition of deep learning
Defining deep neural networks is not an easy task! There are several different definitions depending on who you ask. Some say it is just using lots of layers in neural network architectures, which can be stacked or parallelized, trained with backpropagation, and optimized via gradient descent.
Other’s define a deep learning model as one that uses more than 50 million parameters (the average human brain has about 100 billion neurons). This would make most humans at this level considered having “deep learning”![1]
Yet another definition says it is any artificial neural net with more than two fully-connected hidden layers, where each layer connects to all nodes in the next higher layer.[2]
We will use the third definition here for reasons discussed below.
Relationships between the two
Technically, deep learning is not machine learning. Deep neural networks are inspired by how neurons in our brain work!
Neurons are cell components of our nervous system that process information from our senses and relay this to other parts of our body. In humans, there are about one billion neurons connected together to form what we know as the mind.
Just like with computers, there are different types of neurons. Input neurons get signals from your senses, for example sight when you look at something or sound when you listen to music. Output neurons send messages to various areas of the body to do things, for instance sending a signal to your hand so it knows it should grow more quickly than normal due to an increase in blood flow.
With that analogy in mind, let’s talk about why people say “deep learning” requires machine learning.
Examples of deep learning
Recent developments in artificial intelligence (AI) are now being referred to as “deep learning” or, more commonly, just “neural networks.” At its core, neural networks are algorithms that mimic how our brains work.
Deep neural networks have three layers that interact with each other similarly to neurons in your brain interacting with other neurons. Each layer is comprised of many nodes connected to every other node in the next higher layer.
The first layer typically consists of input units, which accept information from outside sources. The second layer contains processing nodes that perform simple mathematical functions on the data coming in from the first layer. Nodes in this layer then transmit their processed results onto nodes in the third layer, where they are combined into complex patterns.
These complex patterns are then output by the network and used to make predictions about new inputs — in this case, predicting something about human behavior. For instance, a predictive algorithm for Amazon reviews could look at past review content to determine whether an article will be positive or negative about a book. Or it could check prices for the product across sites to see if there are any discount codes available.
Experts say that while classic AI was good at finding patterns in large datasets, advanced neural networks can achieve similar results using smaller sets of data. This makes them particularly useful when time is limited, space is constrained, or both, like with online shopping.
Examples of machine learning
A lot of people get stuck thinking that deep neural networks are just another kind of machine learning algorithm. They are not!
Deep neural nets with their stacking of nonlinear transformations and pooling layers are an example of what is known as representation-learning.
Representation learning uses mathematical functions or rules to organize information, in this case, knowledge about computer images.
By organizing image data into sets of numbers and then applying logic to combine those numbers together, you can create algorithms that recognize patterns in all sorts of images.
This sort of reasoning applies beyond pictures too. For instance, by looking at large groups of texts or voices, software could learn how to identify important keywords or phrases.
There are some who refer to these advanced NNs as “machine learning” because they use concepts from ML to achieve their goals.
But representations like neurons and matrix multiplication aren’t exactly intuitive so it’s best to think of them as something more abstract than plain old machine learning.
What is the difference between the two?
Over the past few years, AI has become a very popular term. Some refer to it as “machine learning” or even just “learning algorithms” because that seems easier to understand. But what does all of this really mean?
In fact, deep neural networks (DNNs) are not technically part of the larger category of machine learning algorithms. They’re actually called artificial neural networks (ANNs).
A DNN is simply an algorithm that uses layers to learn about data. Each layer learns specific concepts by using the outputs of the previous layer to help determine its own internal state.
These layers can be made up of simple computational functions like linear regression or logistic regression, but more commonly they’re some variant of feedforward ANN such as convolutional neural network (CNN), recurrent neural network (RNN), or long short-term memory (LSTM) network.
Computer scientists use terms like “deep” when referring to these architectures due to their stacking structure.
Do I need both?
A lot of people get stuck on just using deep learning because they perceive it to be some sort of magical technology that knows how to solve any problem.
That’s not true at all! Using deep neural networks doesn’t require knowing about neurons or layers, nor does it presuppose we’re giving them adequate pre-training.
What it requires is an understanding of probability theory and statistics. This applies even if you don’t use the term “probability theory” in your job, since most things done with AI these days are statistical in nature — e.g., computer vision, natural language processing (NLP), etc.
Furthermore, while there have been successful applications of ML algorithms beyond the domain of machine learning, many experts agree that advanced uses of traditional ML methods are still more efficient than non-traditional ones like DNNs when possible.
Tips for those with little experience
Even if you have no background in machine learning, there are still many ways to use deep neural networks. There are plenty of online courses that can teach you how to use them effectively. You do not need formal education in math or computer science to learn how to apply these algorithms.
Many companies now offer tools and software designed to make it easy to implement this technology. These applications typically take input data and feed it into an already-trained network, which then processes the information and outputs results.
There are also several free softwares and apps available online that contain pre-trained models for things like image recognition, natural language processing, and speech recognition. By having a pretrained model, users don’t have to spend time training the algorithm yourself first!
Experts agree that using a trained brain is usually the better option when it comes to AI.
Tips for those with experience
There is an assumption made in some literature that anyone can learn how to use deep learning tools effectively, but this isn’t always true. While there are many great resources available online, most of them focus more on using pre-trained models rather than tweaking the settings or training new layers.
In fact, it can be quite difficult figuring out what parts of the model need changing! If you’re looking to test your skills, trying to modify a pre-trained network may not be the best way to do so.
On top of that, some people assume that since every article and video about neural networks talks about how powerful they are, then everyone should have access to one immediately. This isn’t necessarily the case – developing applications using DL requires understanding certain concepts such as backpropagation first, which is often left out when diving into applications directly.
There are also several terms used interchangeably, making it hard to tell if someone means batch size, epochs, or iterations.
