Recent developments in deep learning have led to incredible results, but still require powerful graphics processing units (GPUs) or other computer hardware to run efficiently. If you do not own an expensive GPU, there are many free alternatives that work just as well if not better than paid GPUs!
This article will talk about some of these alternative gpu modes and what users can do to use them effectively. There is no need to be limited by the prices of only using NVIDIA cards!
Introduction
Deep neural networks are computational engines designed to perform complex tasks such as image recognition or natural language understanding. They are typically made up of several “layers” which are sets of computers performing simple math operations and then passing their output onto another layer. Each layer performs its task and combines it with information from the previous one until your machine is able to recognize something like a cat.
The most recent development in this field comes in the form of advanced graphical processing unit (gpu)-optimized software called TensorFlow 2.0. A gpu is simply a piece of silicon with special features designed to quickly calculate large amounts of data. Because these calculations occur rapidly, gpu-optimized software uses parallelization where different parts of a calculation are performed at the same time instead of one after the other. This creates faster overall performance because fewer things have to happen sequentially.
Next, you need to compare the differences between CPU and GPU
As mentioned before, GPUs are very efficient at running parallel computing tasks. This is why most of the state-of-the-art deep learning algorithms use them as their processing backbone.
GPUs have many more transistors than CPUs do, which allows more complex computational logic to be built into one chip instead of having to spread it out over multiple chips.
So how much gpu should I buy?
That depends on your intent. If you just want to experiment with different neural network architectures, then buying a low end GTX 1080 or Titan X will work great! You can test out all sorts of networks such as VGG, ResNet, etc.
But if you really want to push the limits by experimenting with bigger models (like ImageNet) or newer types of networks like NAS, then investing in a high end NVIDIA GPU may make sense.
Third, you need to consider the size of your model
A very common mistake is buying too much GPU memory – or gpu “lots” as some people call it. You don’t want too much RAM because that just means slower speed!
You will probably run into this problem if you are trying to increase batch sizes or use larger neural networks. The more data you have, the better the network can learn so having more RAM usually makes things faster.
However, unless you know what kind of GPU you have (and how many gigabytes it has) then doubling up on ram isn’t always helpful. Because GPUs work by storing information in registers instead of normal RAM, adding extra RAM could actually slow down your machine. This is why there is an optimal amount of RAM for deep learning!
A general rule of thumb is to only add one additional 256 GB of RAM per 1 TB of GPU memory you have. For example, if you have a 1080 Ti with 12GB VRAM, you would get around 2 years worth of training time out of it by using 32 GB RAM.
Fourth, you should consider the data set you are training with
When it comes to GPU usage, most people get very specific about how many GPUs they have or want, but they rarely talk about what kind of computer you need to run deep learning algorithms.
This is an important factor! You don’t necessarily need a high-end gaming laptop or desktop rig to experiment with neural networks. There are some great general use NVIDIA graphics cards that can function as powerful dedicated gpu devices.
For example, if you just wanted to test out basic image classification architectures then using a GTX 1050 (or even better a 1060) will do the trick. These chips work at around $100 – $150 per card depending on whether you buy them directly from NVIDIA or not.
However, if you really want to push the limits then investing in a few more expensive gpus may be your best bet.
Fifth, consider the environment you are training in
If you want to run your model on GPUs that have less memory or lower end GPU gfx, you will need to use either very large batch sizes or very many gpus to get good performance.
This is because the computer will not be able to keep up with trying to process all of the data at once.
It takes time for each GPU to calculate how much memory it has left and then to save the results into persistent storage (such as hard drives). This means that before it can start working on the next element of the batch, it must first check if there’s free space available!
If there isn’t enough, it needs to wait until there is so that it can overwrite those old results. For this reason, you will need lots and lots of gpus to train effectively on low-memory cards!
Deep learning models can become really expensive when running out of memory, especially since most require larger batches than normal NN classes do.
Sixth, compare the differences between the CPU and GPU
The final consideration in choosing which gpu to use is really about how much gpus you have!
If you do not have too many GPUs then the previous recommendations still hold true, just pick one that has more RAM per GPU as opposed to more GHz speed. This will help ensure good performance even if there are multiple applications running at once.
For most people though this tip no longer applies since almost every computer now comes with either an NVIDIA or AMD graphics card! 😉
But what kind of GPU should you get?
That depends on your purpose for using deep learning! If you want to quickly train neural networks on lots of data then go for the highest performing cards such as the Titan Xp or 1080ti. These can cost around 400-500 dollars depending on where you buy them.
However, if you only need to test out different architectures or settings on few models then lower end cards like the GTX 1050 or RX 560 are fine! They may be less powerful but they won’t cost very much either so it makes sense to spend a little bit more money upfront and save some money in the long run.
Seventh, there are many other factors that can influence your decision
Due to the growing popularity of deep learning, you will find a lot of tutorials and guides with recommendations on how much gpu money to buy. Some say one device is enough, while others tell you to spend thousands!
It’s easy to get overwhelmed by all of these numbers and trends. Luckily, we have some more solid strategies for choosing an appropriate GPU budget.
General guidelines
The first thing you should do before deciding on whether or not to invest in a specific graphics card is evaluate your workload and determine if it can be distributed across several cards.
If so, then buying only one GPU may make sense because you would still receive good performance at a lower cost. On the contrary, if every task within the computer requires high performance, then investing in just one GPU won’t cut it anymore.
Furthermore, depending on what kind of software you use for training, the settings for each program can have a significant impact on the overall speed of your machine. For example, people usually start off using low-quality image editing softwares like Photoshop and then later upgrade to better ones.
This article will go into more detail about why having multiple GPUs makes sense and which types of computers benefit most from this.
Eighth, some tasks are not suitable for GPU
There are two main reasons why you might want to avoid using GPUs for deep learning. First, as mentioned earlier, many of the networks require very large amounts of gpu memory to run. This can be due to having too much data in your images or videos, longer sequences, etc.
Second, some neural network architectures simply cannot take advantage of parallelization on graphics cards. These include things like convolutional networks, recurrent networks such as LSTMs, and generative models such as VAEs and GANs.
However, even if you have plenty of RAM and CPU cores, it is still possible to use GPUs for deep learning! Here we will discuss eight situations where that is possible. If you’re ever stuck and need help quickly, check out our beginner-level article here: Tips For Starting With Neural Networks On The Computer.
Ninth, benchmark your current GPU to be sure it is not underperforming
The final important consideration in choosing your GPU is finding one that is actually performing well!
Benchmarking your GPU can sometimes feel like a daunting task as most people are not experienced with this process. Thankfully, there are many free and paid tools that make this process easy to use.
Many of these sites have you create an account through their site where they keep track of how long it takes to complete a test using your GPU and compare it to other GPUs to give you some insights into which cards are over or under performing.
Some of the more popular benchmarks include PCI (Performance Comparison Installation) by 3DGPUsComparison, OCID, and CEDIA Pro. All three offer users a free trial before purchasing a plan that includes all features.
