Deep Learning (BAI701 ) unit two cover

Unit Overview

1. What this unit covers: Understanding deep learning models, their theory, training methods, architectures like CNNs and GANs, and their applications.
2. Importance in Deep Learning: Provides the foundation for building powerful AI systems in vision, NLP, and generative tasks.
3. Relevance for AKTU exams: Questions often include definitions, comparisons, diagrams, short explanations, and 5/10-mark questions.

History of Deep Learning

1. 1950s – Perceptron: First neural network model by Frank Rosenblatt; could learn simple patterns like AND/OR gates.
2. 1980s – Backpropagation: Introduced efficient method to train multi-layer networks; allowed networks to learn complex patterns.
3. 2000s – Resurgence: With more data and powerful GPUs, deep networks became practical; success in image & speech recognition.
4. Reason for recent popularity:

A Probabilistic Theory of Deep Learning

1. Probabilistic view: Neural networks can be seen as models predicting probabilities, not just fixed outputs.
2. Intuition: Helps understand uncertainty in predictions. For example, a network predicting if a photo has a cat might assign 0.9 probability to "cat" and 0.1 to "dog".
3. Why useful: Allows decision-making under uncertainty and explains network predictions statistically.

Backpropagation, Regularization & Batch Normalization

1. Backpropagation:
2. Regularization:
3. Batch Normalization:

VC Dimension and Neural Nets

1. VC Dimension: Measures the capacity of a model to classify data correctly.
2. Simple idea: Higher VC dimension = network can represent more complex functions.
3. Why it matters: Helps understand generalization—too high VC may overfit; too low VC may underfit.

Deep vs Shallow Networks

1. Shallow Networks: Few layers, simpler tasks, faster training, limited representation.
2. Deep Networks: Many layers, capture hierarchical features, complex tasks like image & language understanding.

Convolutional Networks (CNNs)

1. What CNNs are: Neural networks designed to process grid-like data (images).
2. Why good for images: Uses filters to detect edges, textures, patterns hierarchically.
3. Real-life example: Image classification—CNN can tell whether a photo contains a cat, dog, or car.

Generative Adversarial Networks (GAN) & Semi-supervised Learning

1. GAN:
2. Semi-supervised Learning:

AKTU Exam Answer Boost

Frequently Asked Questions:

1. Define deep networks and give examples.
2. Explain backpropagation with diagram.
3. Compare deep vs shallow networks.
4. Explain batch normalization and its benefits.
5. Describe GAN and semi-supervised learning.

5-Mark Answer Tips:

1. Use bold keywords like: layers, overfitting, generator, discriminator.
2. Keep 4–6 short points, each 1–2 lines.

10-Mark Answer Tips:

1. Include definition, working, diagram, example, pros/cons.
2. Compare concepts in a table wherever possible.

One-Shot Revision Summary

1. Deep Networks: Networks with multiple layers (>2) capturing hierarchical patterns.
2. Backpropagation: Method to update weights using error gradients.
3. Regularization: Prevents overfitting; common methods: L1, L2, dropout.
4. Batch Normalization: Normalizes activations; stabilizes training.
5. VC Dimension: Measures model capacity; affects generalization.
6. Shallow vs Deep: Deep = more layers, higher representation, complex tasks.
7. CNNs: Filters extract spatial patterns from images.
8. GANs: Generator vs discriminator; creates realistic synthetic data.
9. Semi-supervised Learning: Uses both labeled and unlabeled data to improve learning.

After studying this unit, the student should feel confident to attempt any AKTU question from this chapter.