# UNSUPERVISED DEEP LEARNING IN PYTHON

# Theano / Tensorflow: Autoencoders, Restricted Boltzmann Machines, Deep Neural Networks, t-SNE and PCA

Created by Lazy Programmer Inc.

Last updated 6/2017

English

What Will I Learn?

- Understand the theory behind principal components analysis (PCA)
- Know why PCA is useful for dimensionality reduction, visualization, de-correlation, and denoising
- Derive the PCA algorithm by hand
- Write the code for PCA
- Understand the theory behind t-SNE
- Use t-SNE in code
- Understand the limitations of PCA and t-SNE
- Understand the theory behind autoencoders
- Write an autoencoder in Theano and Tensorflow
- Understand how stacked autoencoders are used in deep learning
- Write a stacked denoising autoencoder in Theano and Tensorflow
- Understand the theory behind restricted Boltzmann machines (RBMs)
- Understand why RBMs are hard to train
- Understand the contrastive divergence algorithm to train RBMs
- Write your own RBM and deep belief network (DBN) in Theano and Tensorflow
- Visualize and interpret the features learned by autoencoders and RBMs

Requirements

- Knowledge of calculus and linear algebra
- Python coding skills
- Some experience with Numpy, Theano, and Tensorflow
- Know how gradient descent is used to train machine learning models
- Install Python, Numpy, and Theano
- Some probability and statistics knowledge
- Code a feedforward neural network in Theano or Tensorflow

Description

This course is the next logical step in my

**deep learning, data science,**and**machine learning**series. I’ve done a lot of courses about deep learning, and I just released a course about**unsupervised learning**, where I talked about**clustering**and**density estimation**. So what do you get when you put these 2 together? Unsupervised deep learning!
In these course we’ll start with some very basic stuff –

**principal components analysis (PCA)**, and a popular nonlinear dimensionality reduction technique known as**t-SNE (t-distributed stochastic neighbor embedding)**.
Next, we’ll look at a special type of unsupervised neural network called the

**autoencoder**. After describing how an autoencoder works, I’ll show you how you can link a bunch of them together to form a deep stack of autoencoders, that leads to better performance of a supervised**deep neural network**. Autoencoders are like a non-linear form of PCA.
Last, we’ll look at

**restricted Boltzmann machines (RBMs)**. These are yet another popular unsupervised neural network, that you can use in the same way as autoencoders to**pretrain**your supervised deep neural network. I’ll show you an interesting way of training restricted Boltzmann machines, known as**Gibbs sampling**, a special case of**Markov Chain Monte Carlo,**and I’ll demonstrate how even though this method is only a rough approximation, it still ends up reducing other cost functions, such as the one used for autoencoders. This method is also known as**Contrastive Divergence**or**CD-k**. As in physical systems, we define a concept called**free energy**and attempt to minimize this quantity.
Finally, we’ll bring all these concepts together and I’ll show you visually what happens when you use PCA and t-SNE on the features that the autoencoders and RBMs have learned, and we’ll see that even without labels the results suggest that a pattern has been found.

All the materials used in this course are FREE. Since this course is the 4th in the deep learning series, I will assume you already know calculus, linear algebra, and

**Python**coding. You’ll want to install**Numpy,****Theano, and Tensorflow**for this course. These are essential items in your**data analytics**toolbox.
If you are interested in deep learning and you want to learn about modern deep learning developments beyond just plain

**backpropagation**, including using unsupervised neural networks to interpret what features can be automatically and hierarchically learned in a deep learning system, this course is for you.
This course focuses on “

**how to build and understand**“, not just “how to use”. Anyone can learn to use an API in 15 minutes after reading some documentation. It’s not about “remembering facts”, it’s about**“seeing for yourself” via experimentation**. It will teach you how to visualize what’s happening in the model internally. If you want**more**than just a superficial look at machine learning models, this course is for you.
NOTES:

All the code for this course can be downloaded from my github: /lazyprogrammer/machine_learning_examples

In the directory: unsupervised_class2

Make sure you always “git pull” so you have the latest version!

HARD PREREQUISITES / KNOWLEDGE YOU ARE ASSUMED TO HAVE:

- calculus
- linear algebra
- probability
- Python coding: if/else, loops, lists, dicts, sets
- Numpy coding: matrix and vector operations, loading a CSV file
- can write a feedforward neural network in Theano or Tensorflow

TIPS (for getting through the course):

- Watch it at 2x.
- Take handwritten notes. This will drastically increase your ability to retain the information.
- Write down the equations. If you don’t, I guarantee it will just look like gibberish.
- Ask lots of questions on the discussion board. The more the better!
- Realize that most exercises will take you days or weeks to complete.
- Write code yourself, don’t just sit there and look at my code.

USEFUL COURSE ORDERING:

- (The Numpy Stack in Python)
- Linear Regression in Python
- Logistic Regression in Python
- (Supervised Machine Learning in Python)
- (Bayesian Machine Learning in Python: A/B Testing)
- Deep Learning in Python
- Practical Deep Learning in Theano and TensorFlow
- (Supervised Machine Learning in Python 2: Ensemble Methods)
- Convolutional Neural Networks in Python
- (Easy NLP)
- (Cluster Analysis and Unsupervised Machine Learning)
- Unsupervised Deep Learning
- (Hidden Markov Models)
- Recurrent Neural Networks in Python
- Artificial Intelligence: Reinforcement Learning in Python
- Natural Language Processing with Deep Learning in Python

Who is the target audience?

- Students and professionals looking to enhance their deep learning repertoire
- Students and professionals who want to improve the training capabilities of deep neural networks
- Students and professionals who want to learn about the more modern developments in deep learning

**Size: 556.35M**

Content retrieved from: https://www.udemy.com/unsupervised-deep-learning-in-python/.

UNSUPERVISED DEEP LEARNING IN PYTHON
Reviewed by Syasu
on
July 11, 2018
Rating: