Session 3: Introduction to Deep Learning and CNNs

Neural Networks and Convolutional Architectures for Earth Observation

Instructor

CoPhil Advanced Training Program

Date

November 17, 2025

Session 3: Introduction to Deep Learning and CNNs

Neural Networks and Convolutional Architectures for Earth Observation

Transitioning from feature engineering to feature learning

Session Overview

Duration: 2.5 hours | Type: Theory + Interactive Demonstrations | Difficulty: Intermediate

This pivotal session bridges traditional machine learning (Sessions 1-2) and modern deep learning approaches. You’ll understand the fundamental shift from manual feature engineering to automatic feature learning through neural networks, with specific focus on Convolutional Neural Networks (CNNs) for Earth observation applications.

Presentation Slides

ImportantPrerequisites
  • ✓ Complete Sessions 1-2 (Random Forest classification)
  • ✓ Understanding of classification concepts (accuracy, confusion matrix)
  • ✓ Basic Python and NumPy familiarity
  • ✓ Colab environment with GPU runtime enabled
  • ✓ Conceptual understanding of matrix operations

What You’ll Learn

After completing this session, you will be able to:

  1. Understand the ML → DL Transition
    • Recognize when to use traditional ML vs deep learning
    • Explain the automatic feature learning paradigm
    • Identify computational requirements and trade-offs
    • Understand data requirements for deep learning success
  2. Master Neural Network Fundamentals
    • Build simple perceptrons from scratch using NumPy
    • Implement activation functions (ReLU, sigmoid, softmax)
    • Understand forward propagation and backpropagation
    • Visualize decision boundaries and learning dynamics
  3. Comprehend Convolutional Neural Networks
    • Explain convolution operations and their purpose
    • Understand pooling, padding, and stride concepts
    • Visualize filter responses on satellite imagery
    • Compare CNN architectures (LeNet, VGG, ResNet, U-Net)
  4. Apply CNNs to Earth Observation Tasks
    • Identify appropriate CNN architectures for EO problems
    • Understand scene classification vs semantic segmentation
    • Recognize object detection and change detection approaches
    • Connect CNN capabilities to Philippine EO applications
  5. Navigate Practical Considerations
    • Address data-centric AI principles for EO
    • Handle limited training data scenarios
    • Understand transfer learning and pre-trained models
    • Recognize computational constraints and optimization strategies

Session Structure

Part A: From Machine Learning to Deep Learning (15 minutes)

Understanding the paradigm shift from manual to automatic feature learning.

🔧 Traditional ML (Sessions 1-2)

What you did: - Manually calculated NDVI, NDWI, NDBI - Engineered GLCM texture features - Extracted temporal statistics - Combined features thoughtfully

Pros: Interpretable, works with small datasets Cons: Requires domain expertise, limited by imagination

🧠 Deep Learning (Sessions 3-4)

What CNNs do: - Learn features automatically from raw pixels - Discover hidden patterns humans miss - Build hierarchical representations - Optimize end-to-end

Pros: No feature engineering, state-of-the-art accuracy Cons: Needs large datasets, computationally intensive

⚖️ When to Use Which?

Use Random Forest when: - Limited training data (<1000 samples) - Need interpretability (DENR reports) - Have domain features (indices) - Fast prototyping needed

Use CNNs when: - Large labeled datasets (>10,000 samples) - Complex spatial patterns - Maximum accuracy required - GPU resources available

🇵🇭 Philippine EO Context

PhilSA Applications: - Scene classification (land cover) - Cloud detection (Sentinel-2) - Building footprint extraction - Flood extent mapping

Why CNNs? Handle complex tropical landscapes, monsoon cloud patterns, informal settlements

Key Insight: “In Sessions 1-2, you manually engineered features. CNNs will learn these features automatically—and discover new ones you never imagined!”

Part B: Neural Network Fundamentals (25 minutes)

Building intuition from the ground up using interactive Jupyter notebooks.

B.1: The Perceptron - Simplest Neural Unit

Understanding the building block of all neural networks.

Mathematical Foundation: \[ y = f\left(\sum_{i=1}^{n} w_i x_i + b\right) \]

Where: - \(x_i\): Input features (e.g., pixel values, NDVI) - \(w_i\): Learned weights - \(b\): Bias term - \(f\): Activation function - \(y\): Output prediction

Hands-On: Build a perceptron from scratch to classify “Water vs Non-Water” using NDWI.

# Simple perceptron implementation
class Perceptron:
    def __init__(self, input_dim):
        self.weights = np.random.randn(input_dim)
        self.bias = 0

    def predict(self, X):
        linear_output = np.dot(X, self.weights) + self.bias
        return self.activation(linear_output)

    def activation(self, z):
        return 1 if z > 0 else 0  # Step function

B.2: Activation Functions - Adding Non-Linearity

Why neural networks need activation functions to solve complex problems.

Sigmoid \[\sigma(z) = \frac{1}{1 + e^{-z}}\]

Range: (0, 1) Use: Binary classification output EO Example: Cloud probability

ReLU (Rectified Linear Unit) \[\text{ReLU}(z) = \max(0, z)\]

Range: [0, ∞) Use: Hidden layers (most popular) Why: Fast, sparse activation

Softmax \[\text{softmax}(z_i) = \frac{e^{z_i}}{\sum_j e^{z_j}}\]

Range: (0, 1), sums to 1 Use: Multi-class classification EO Example: Land cover classes

Tanh \[\tanh(z) = \frac{e^z - e^{-z}}{e^z + e^{-z}}\]

Range: (-1, 1) Use: When centered data needed Note: Less common than ReLU

Interactive Demo: Visualize how each activation function transforms Sentinel-2 spectral values.

B.3: Multi-Layer Networks - Learning Complex Patterns

Stacking layers to learn hierarchical representations.

Architecture:

Input Layer (e.g., Sentinel-2 bands)
    ↓
Hidden Layer 1 (64 neurons, ReLU)
    ↓
Hidden Layer 2 (32 neurons, ReLU)
    ↓
Output Layer (8 classes, Softmax)

What Each Layer Learns: - Layer 1: Low-level features (edges, textures) - Layer 2: Mid-level features (shapes, patterns) - Layer 3: High-level features (objects, scenes) - Output: Class probabilities

Hands-On: Train a 2-layer network to classify Palawan land cover using spectral features (from Session 2).

B.4: Training Process - Learning from Data

Understanding gradient descent and backpropagation intuitively.

Training Loop: 1. Forward Pass: Input → Hidden → Output → Prediction 2. Calculate Loss: Compare prediction to true label 3. Backward Pass: Compute gradients (how much each weight contributed to error) 4. Update Weights: \(w_{new} = w_{old} - \alpha \cdot \frac{\partial L}{\partial w}\) 5. Repeat: Until loss converges

Key Hyperparameters: - Learning Rate (\(\alpha\)): Step size for weight updates (0.001 - 0.1) - Batch Size: Number of samples per gradient update (32, 64, 128) - Epochs: Complete passes through training data (10-100)

Interactive Exploration: Experiment with learning rates to see overfitting vs underfitting.

Part C: Convolutional Neural Networks (30 minutes)

Deep dive into the architecture that revolutionized computer vision and Earth observation.

C.1: Why CNNs for Images?

Problem with Regular Neural Networks: - A 10m Sentinel-2 image chip (256×256×10 bands) = 655,360 parameters just for first layer! - No spatial awareness (treats nearby pixels same as distant ones) - Computationally infeasible

CNN Solutions: - Local Connectivity: Each neuron connects to small spatial region - Parameter Sharing: Same filter applied across entire image - Translation Invariance: Detect features anywhere in image

Result: Millions fewer parameters, spatially-aware learning!

C.2: Convolution Operation - The Heart of CNNs

Understanding how convolutions extract features from satellite imagery.

Mathematical Definition: \[ (I * K)(i,j) = \sum_{m}\sum_{n} I(i+m, j+n) \cdot K(m,n) \]

Where: - \(I\): Input image (e.g., Sentinel-2 NIR band) - \(K\): Filter/kernel (e.g., 3×3 edge detector) - \(*\): Convolution operator

Visual Example:

Sentinel-2 NIR Image (5×5)    Edge Detection Filter (3×3)
┌─────────────────┐           ┌─────────┐
│ 120 115 118 122 │           │ -1  -1  -1 │
│ 118 245 242 125 │     *     │  0   0   0 │
│ 119 248 244 121 │           │  1   1   1 │
│ 121 116 119 123 │           └─────────┘
└─────────────────┘
                ↓
    Feature Map (detects water edges)

Hands-On: - Apply manual convolutions to Sentinel-2 patches - Visualize classic filters (edge detection, blur, sharpen) - See how filters respond to forests, water, urban areas

C.3: CNN Building Blocks

1. Convolutional Layer - Applies multiple filters to input - Each filter learns to detect different feature - Output: Feature maps (activations)

Parameters: - filters: Number of filters (32, 64, 128…) - kernel_size: Filter dimensions (3×3, 5×5) - stride: Step size for filter movement (usually 1) - padding: Border handling (‘same’ or ‘valid’)

2. Pooling Layer - Reduces spatial dimensions (downsampling) - Provides translation invariance - Most common: Max pooling

Max Pooling (2×2):

Input (4×4)                Output (2×2)
┌────────────┐            ┌──────┐
│ 1  3  2  4 │            │ 3  4 │
│ 2  3  1  2 │    →       │ 7  9 │
│ 5  7  8  9 │            └──────┘
│ 1  2  3  4 │
└────────────┘
Takes maximum in each 2×2 window

3. Fully Connected Layer - Traditional neural network layer - Connects all features to output classes - Usually at end of network

4. Dropout Layer - Randomly deactivates neurons during training - Prevents overfitting - Common rate: 0.3-0.5

C.4: Classic CNN Architectures

Understanding architectures used in EO applications.

LeNet-5 (1998)

Structure: - Conv → Pool → Conv → Pool → FC - 60K parameters - Original: Handwritten digits

EO Use: - Simple scene classification - Educational examples - Quick prototypes

VGG-16 (2014)

Structure: - 13 Conv layers + 3 FC - 138M parameters - Small 3×3 filters stacked

EO Use: - Scene classification - Pre-trained on ImageNet - Transfer learning baseline

ResNet-50 (2015)

Innovation: - Skip connections (residual blocks) - Solves vanishing gradient - 50 layers deep

EO Use: - High-accuracy classification - Feature extraction - PhilSA scene classifier

U-Net (2015)

Innovation: - Encoder-decoder architecture - Skip connections preserve detail - Outputs same-size segmentation

EO Use: - Semantic segmentation - Flood mapping - Building extraction - Session 4 focus!

Interactive Visualization: Explore how different architectures process Sentinel-2 imagery.

Part D: CNNs for Earth Observation (25 minutes)

Connecting CNN capabilities to Philippine EO operational needs.

D.1: Scene Classification

Task: Assign single label to entire image patch

Architecture: ResNet, VGG, EfficientNet (classification head)

Philippine Applications:

Application Classes Dataset Stakeholder
Land Cover Forest, Urban, Agriculture, Water, Bare PhilSA Sentinel-2 DENR, LGUs
Cloud Detection Clear, Thin Cloud, Thick Cloud, Shadow Sentinel-2 Level-1C PhilSA (preprocessing)
Rice Field Stage Land Prep, Transplanting, Vegetative, Harvest PlanetScope + field data DA, PhilRice
Disaster Assessment Damaged, Undamaged, Debris Drones + Sentinel-2 NDRRMC, PAGASA

Data Requirements: - Typical: 1,000-10,000 labeled image chips per class - PhilSA strategy: Start with 500/class, augment with rotations/flips

Hands-On (Session 4): Build ResNet-based classifier for Palawan land cover

D.2: Semantic Segmentation

Task: Classify every pixel in image (pixel-wise labels)

Architecture: U-Net, DeepLabv3+, SegNet

Philippine Applications:

🌊 Flood Mapping

Challenge: Rapid post-disaster assessment

CNN Solution: - Input: Sentinel-1 SAR (pre + post event) - Output: Flooded/Non-flooded pixel map - Architecture: U-Net

PhilSA Use Case: Pampanga flood 2023 - 6-hour processing time (vs 2 days manual)

🏘️ Informal Settlements

Challenge: Map slums for disaster planning

CNN Solution: - Input: High-res imagery (PlanetScope, drones) - Output: Building footprints - Architecture: U-Net + post-processing

NEDA Application: Metro Manila vulnerability assessment

🌳 Forest Degradation

Challenge: Detect selective logging

CNN Solution: - Input: Multi-temporal Sentinel-2 - Output: Degraded forest pixels - Architecture: U-Net + LSTM

DENR Use: Protected area monitoring

⛏️ Mining Activity

Challenge: Illegal mining detection

CNN Solution: - Input: Sentinel-2 + Sentinel-1 - Output: Mining site polygons - Post-process: Vectorize

MGB Application: Permit compliance checking

Data Requirements: - Annotation intensive: Need pixel-level labels - Typical: 100-500 labeled images (256×256 chips) - Tools: QGIS, LabelMe, CVAT

Hands-On (Session 4): Implement U-Net for flood mapping in Central Luzon

D.3: Object Detection

Task: Find and localize objects with bounding boxes

Architecture: Faster R-CNN, YOLO, RetinaNet

Philippine Applications: - Ship Detection: Illegal fishing monitoring (Sentinel-1) - Building Detection: Infrastructure mapping (high-res) - Tree Counting: Forest inventory (drone imagery) - Vehicle Detection: Traffic monitoring (PlanetScope)

Data Format: Bounding boxes + class labels (COCO, PASCAL VOC formats)

Computational Note: More complex than classification, requires anchor boxes and region proposals

D.4: Change Detection

Task: Identify what changed between two time points

CNN Approaches:

  1. Siamese Networks: Compare two images with shared weights
  2. Early Fusion: Stack temporal images as input channels
  3. Late Fusion: Separate encoders + change decoder

Philippine Applications: - Deforestation (Palawan, Mindanao) - Urban expansion (Metro Manila, Cebu) - Post-disaster damage (typhoon impacts) - Agricultural change (conversion detection)

Challenge: Need paired labeled change data (before + after + change mask)

Part E: Practical Considerations for EO Deep Learning (15 minutes)

Real-world challenges and solutions for Philippine EO practitioners.

E.1: The Data Challenge

How Much Data Do You Need?

Model Complexity Typical Requirement Philippine Reality
Simple CNN (5 layers) 5,000-10,000 samples ✓ Achievable
ResNet-50 (from scratch) 100,000+ samples ✗ Rarely available
ResNet-50 (fine-tuned) 1,000-5,000 samples ✓ Achievable with augmentation
U-Net (segmentation) 100-500 images ✓ Achievable but labor-intensive

Data-Centric AI Principles: 1. Quality > Quantity: 500 clean labels >> 5,000 noisy labels 2. Representative Sampling: Cover all Philippine ecosystems (lowland, upland, coastal) 3. Class Balance: Equal samples per class (or weighted loss) 4. Validation Split: Hold out 20% for unbiased evaluation

Philippine Data Sources: - PhilSA Space+ Data Dashboard (satellite imagery) - NAMRIA Geoportal (reference maps) - DOST-ASTI DATOS (disaster imagery) - LiDAR Portal (elevation + canopy) - Field campaigns (GPS + photos)

E.2: Transfer Learning - Training on Limited Data

Strategy: Start with model pre-trained on large dataset (ImageNet), fine-tune on Philippine data

Benefits: - Need 10× less data - Train 5× faster - Better accuracy with small datasets

Implementation:

# Load pre-trained ResNet
base_model = ResNet50(weights='imagenet', include_top=False)

# Freeze early layers (keep learned features)
for layer in base_model.layers[:100]:
    layer.trainable = False

# Add custom classification head
x = GlobalAveragePooling2D()(base_model.output)
x = Dense(128, activation='relu')(x)
output = Dense(8, activation='softmax')(x)  # 8 Palawan classes

model = Model(inputs=base_model.input, outputs=output)

When to Use: - Limited training data (<5,000 samples) - Similar task to pre-training (natural images) - Need quick results

Caution: ImageNet has RGB images. Sentinel-2 has 10+ bands. Adaptation strategies needed (Session 4).

E.3: Data Augmentation - Artificially Expanding Training Data

Geometric Transformations: - Rotation: 90°, 180°, 270° (satellites view from any angle) - Horizontal/Vertical Flip: Valid for overhead imagery - Zoom/Scale: Simulate different resolutions - Translation: Small shifts

Spectral Augmentations: - Brightness/Contrast: Simulate atmospheric conditions - Gaussian Noise: Simulate sensor noise - Band Dropout: Improve robustness

Philippine Context: - ✓ Use rotation/flips for land cover (no preferential orientation) - ✗ Avoid rotation for infrastructure (roads have direction) - ✓ Augment brightness for cloud variations

Implementation (Session 4):

from tensorflow.keras.preprocessing.image import ImageDataGenerator

augmentation = ImageDataGenerator(
    rotation_range=90,
    horizontal_flip=True,
    vertical_flip=True,
    brightness_range=[0.8, 1.2],
    zoom_range=0.1
)

E.4: Computational Requirements

Training Resources:

Model Training Time* GPU Memory Cost (Colab Pro)
Simple CNN 30 min 4 GB Free tier OK
ResNet-50 (fine-tune) 2-4 hours 8 GB Free tier OK
U-Net (segmentation) 4-8 hours 12 GB Pro needed
ResNet-50 (from scratch) 24+ hours 16 GB Pro+ needed

*1,000 training images, 50 epochs, V100 GPU

Philippine Context: - PhilSA uses on-premise GPU servers (8× NVIDIA A100) - Universities: Limited GPU access (submit jobs) - Practitioners: Google Colab Pro ($10/month) recommended

Optimization Strategies (Session 4): - Use mixed precision training (FP16) - Reduce batch size if memory limited - Train on smaller image chips (128×128 instead of 256×256) - Use gradient checkpointing for large models

E.5: Model Interpretability - Understanding CNN Decisions

Why It Matters: - DENR reports need explanations (“Why was this classified as deforested?”) - Debugging poor performance - Building stakeholder trust

Techniques (Session 4): 1. Activation Visualization: See what filters learned 2. Saliency Maps: Which pixels influenced decision? 3. Class Activation Maps (CAM): Highlight relevant regions 4. Filter Visualization: What patterns do filters detect?

Philippine Application Example: - Question: Why did model classify mangroves as agriculture? - CAM Analysis: Model focused on water proximity, not canopy structure - Solution: Add texture features or more mangrove training samples

Key Concepts

Automatic Feature Learning

What is it? CNNs learn optimal features directly from raw pixel data, eliminating manual feature engineering.

How it works: - Layer 1: Learns edges (horizontal, vertical, diagonal) - like Sobel filters you created manually - Layer 2: Combines edges into textures and simple shapes - Layer 3: Combines shapes into complex patterns (canopy structure, urban grid) - Layer 4+: High-level semantic features (forest type, settlement pattern)

Comparison to Session 2:

Aspect Random Forest (Session 2) CNN (Session 3-4)
Features Manual (NDVI, GLCM) Learned automatically
Spatial Context Limited (within feature) Fully exploited (receptive field)
Data Needed 500-1,000 samples 5,000-10,000 samples
Training Time Minutes Hours
Accuracy 80-85% typical 90-95% possible

Receptive Field

Definition: The region of input image that influences a particular neuron’s activation.

Example: - A neuron in Layer 1 sees a 3×3 pixel region (30m × 30m for Sentinel-2) - A neuron in Layer 3 sees a 15×15 pixel region (150m × 150m) - Output neuron “sees” entire image chip

Why it matters: - Small objects (buildings): Need fewer layers - Large objects (agricultural fields): Need deeper networks - Contextual classification: Large receptive field captures neighborhood

Philippine Example: Classifying a pixel as “mangrove” requires seeing water proximity (large receptive field) AND canopy texture (small receptive field). Multi-scale processing essential!

Translation Invariance

What is it? CNN can recognize patterns regardless of position in image.

How achieved: 1. Parameter sharing: Same filter applied everywhere 2. Pooling: Abstracts exact position

EO Benefit: Forest is forest whether in top-left or bottom-right of image. Train once, apply anywhere in Philippines!

Contrast with Position: For some tasks, position DOES matter (e.g., urban always near coasts in Philippines). Advanced architectures can encode position.

Gradient Descent and Backpropagation

Intuitive Explanation:

Imagine hiking down a foggy mountain (error surface) to reach valley (minimum loss): - Gradient: Direction of steepest descent - Learning rate: Step size - Backpropagation: Efficiently calculates gradients for all weights

Training Process: 1. Forward pass: Image → Predictions 2. Calculate loss: How wrong were predictions? 3. Backward pass: Compute ∂Loss/∂Weight for every parameter 4. Update weights: Move “downhill” toward better performance

Common Issues: - Learning rate too high: Jump over minimum (unstable) - Learning rate too low: Painfully slow convergence - Local minima: Stuck in suboptimal solution (less common with large networks)

Session 4: Implement Adam optimizer (adaptive learning rates)

Interactive Demonstrations

Demo 1: Perceptron Playground

Objective: Build intuition for how perceptrons learn decision boundaries

Activity: 1. Load 2D dataset (NDVI vs NDWI for water classification) 2. Initialize random weights 3. Visualize decision boundary 4. Update weights iteratively 5. Watch boundary align with data

Notebook: session3_theory_STUDENT.ipynb (Part 1)

Expected Outcome: Understand that neural networks find separating hyperplanes through gradient descent.

Demo 3: Manual Convolution on Sentinel-2

Objective: Understand convolution as a sliding filter operation

Activity: 1. Load Sentinel-2 NIR band (Palawan forest patch) 2. Define edge detection filter (3×3 Sobel) 3. Manually compute convolution (NumPy) 4. Visualize feature map 5. Try different filters (blur, sharpen, Gaussian)

Notebook: session3_cnn_operations_STUDENT.ipynb (Part 1)

Aha Moment: “Edge detection filter highlights forest boundaries—exactly what CNN learns automatically!”

Demo 4: Pooling Demonstration

Objective: Understand downsampling and translation invariance

Activity: 1. Load Sentinel-2 image chip (256×256) 2. Apply max pooling (2×2, stride 2) 3. Compare original vs pooled (128×128) 4. Shift image by 1 pixel, repeat 5. See that pooled output is nearly identical (translation invariance)

Notebook: session3_cnn_operations_STUDENT.ipynb (Part 3)

Takeaway: Pooling reduces dimensionality while preserving important features.

Demo 5: Architecture Exploration

Objective: Compare CNN architectures visually

Activity: 1. Visualize LeNet-5, VGG-16, ResNet-50, U-Net architectures 2. Count parameters for each 3. Trace receptive field growth 4. Discuss trade-offs (accuracy vs speed)

Notebook: session3_cnn_operations_STUDENT.ipynb (Part 4)

Connection to Session 4: Choose architecture based on task (classification → ResNet, segmentation → U-Net).

Philippine EO Applications

PhilSA Space+ Dashboard

Current CNN Applications:

  1. Automated Cloud Masking
    • Model: U-Net trained on 5,000 Sentinel-2 scenes
    • Performance: 95% accuracy, 2 min per scene
    • Impact: Enables rapid mosaic generation
  2. Land Cover Classification (National)
    • Model: ResNet-50 fine-tuned on Philippine landscape
    • Classes: 10 (following FAO LCCS)
    • Coverage: Entire Philippines, quarterly updates
    • Users: DENR, DAR, NEDA
  3. Disaster Rapid Mapping
    • Flood Detection: Sentinel-1 + U-Net → 6-hour response
    • Damage Assessment: High-res + object detection → building damage maps
    • Integration: NDRRMC operations dashboard

DENR Forest Monitoring

CNN Use Cases:

  • Protected Area Surveillance: Monthly Sentinel-2 analysis (ResNet classifier)
  • Illegal Logging Detection: Change detection CNN on multi-temporal stacks
  • Biodiversity Hotspot Mapping: Fine-grained forest type classification
  • REDD+ MRV: Automated forest cover change reporting

Data Pipeline:

Sentinel-2 (PhilSA) → Preprocessing (cloud mask) →
CNN Classification → Change Detection →
Alert Generation → Field Validation

LGU Applications (Session 4 Focus)

Accessible CNN Tools for Local Governments:

  1. ASTI SkAI-Pinas: Pre-trained models for common PH tasks
  2. Google Earth Engine: CNN inference on cloud platform
  3. Colab Notebooks: Low-cost GPU training (this training!)

Example Workflow (Session 4): - LGU staff collects 500 training labels (Palawan land cover) - Fine-tunes ResNet-50 using Session 4 notebook - Deploys model for quarterly monitoring - Integrates into local land use planning

Expected Outcomes

Conceptual Understanding

By the end of Session 3, you should be able to:

✅ Explain to a colleague why CNNs are better than Random Forest for complex spatial patterns ✅ Sketch a simple CNN architecture and label components (Conv, Pool, FC) ✅ Describe what happens during forward and backward propagation ✅ Identify appropriate architectures for classification vs segmentation ✅ Discuss data requirements and computational constraints

Technical Skills

✅ Build a simple perceptron from scratch using NumPy ✅ Implement activation functions and visualize their behavior ✅ Perform manual convolution on satellite imagery ✅ Apply classic edge detection filters (Sobel, Gaussian) ✅ Visualize feature maps and pooling operations

Practical Readiness for Session 4

✅ Understand why we’ll use TensorFlow/Keras (vs building from scratch) ✅ Anticipate challenges with Sentinel-2 multi-band data ✅ Recognize data preparation needs (chips, labels, augmentation) ✅ Set expectations for training time and resource requirements

Hands-On Notebooks

Access the Interactive Materials

Tip📓 Jupyter Notebooks (Theory + Interactive)

Two comprehensive notebooks guide you through neural network and CNN fundamentals:

Notebook 1: Neural Network Theory session3_theory_interactive.ipynb

Contents: - Build perceptron from scratch - Implement activation functions - Train 2-layer network on spectral data - Explore learning rate effects - Visualize decision boundaries

Duration: 45 minutes

Notebook 2: CNN Operations session3_theory_interactive.ipynb

Contents: - Manual convolution on Sentinel-2 imagery - Edge detection filters (Sobel, Gaussian, Laplacian) - Max pooling demonstration - Architecture comparison (LeNet, VGG, ResNet, U-Net) - Feature map visualization

Duration: 55 minutes

Google Colab: Open Theory Notebook Open CNN Notebook

Supporting Documentation

Note📚 Reference Materials

CNN Architectures Deep Dive: Cheatsheets & References

Detailed explanations of LeNet-5, VGG-16, ResNet-50, U-Net, and modern variants (EfficientNet, Vision Transformers). Includes architecture diagrams, parameter counts, and EO applications.

EO Applications Guide: Glossary & EO Resources

Comprehensive guide to CNN applications in Earth observation: - Scene classification examples - Semantic segmentation workflows - Object detection case studies - Change detection methods - Philippine-specific use cases

Session Overview: Day 2 Overview

Quick reference with session objectives, structure, and prerequisites.

Troubleshooting

Common Conceptual Questions

“Why does CNN need so much more data than Random Forest?”

Random Forest learns from features YOU engineered (NDVI, GLCM). It needs to learn relationships between ~10-20 features.

CNNs learn from raw pixels (~10,000 per image chip). It needs to learn what features to extract PLUS how to classify. Much harder optimization problem = more data needed.

Mitigation: Transfer learning (Session 4) dramatically reduces data needs.

“Can I use CNNs with small datasets (<500 samples)?”

Technically yes, but results will be poor if training from scratch.

Better approach: 1. Use pre-trained models (ImageNet weights) 2. Aggressive data augmentation 3. Use simpler architectures (fewer layers) 4. Consider traditional ML if data is truly limited

“Why are my manual convolutions so slow in the notebook?”

NumPy convolutions (for learning) are intentionally simple. Production CNNs use highly optimized libraries (cuDNN) on GPUs—1000× faster!

Session 4 will use TensorFlow/PyTorch with GPU acceleration.

“How do I know which architecture to use?”

Simple decision tree: - Classification task (one label per image) → ResNet, EfficientNet - Segmentation task (label every pixel) → U-Net, DeepLabv3+ - Object detection (find + localize) → YOLO, Faster R-CNN - Limited data → Simpler architecture + transfer learning - Real-time inference needed → MobileNet, EfficientNet-Lite

Session 4 will implement ResNet (classification) and U-Net (segmentation).

Technical Issues

“Notebook cells won’t execute in Colab”

  1. Check GPU is enabled: Runtime → Change runtime type → GPU
  2. Restart runtime: Runtime → Restart runtime
  3. Verify installations: !pip list | grep numpy

“Out of memory error when running convolutions”

  • Use smaller image chips (128×128 instead of 256×256)
  • Process one image at a time (don’t load entire dataset)
  • Restart Colab runtime to clear memory

“Visualizations aren’t displaying”

# Add to beginning of notebook
%matplotlib inline
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = (10, 10)

“Can’t access Sentinel-2 data in notebook”

Session 3 uses pre-downloaded Sentinel-2 samples (included in notebook). No GEE authentication needed.

Session 4 will integrate GEE for larger-scale data loading.

Getting Help

Additional Resources

Foundational Learning

Neural Networks: - 3Blue1Brown Neural Network Series - Best intuitive explanation (visual) - Neural Networks and Deep Learning (Free Book) - Andrew Ng’s Deep Learning Specialization (Coursera)

CNNs Specifically: - Stanford CS231n Lectures - Comprehensive (free) - CNN Explainer (Interactive) - Visualize CNNs in browser - Distill.pub Articles - Beautiful visual explanations

Earth Observation Deep Learning

Papers: - Zhu et al. (2017). “Deep Learning in Remote Sensing: A Review.” IEEE GRSM - Ma et al. (2019). “Deep learning in remote sensing applications: A meta-analysis and review.” ISPRS - Rußwurm & Körner (2020). “Self-attention for raw optical satellite time series classification.” ISPRS

Tutorials: - Deep Learning for Earth Observation (ESA) - Raster Vision - Framework for geospatial ML - TorchGeo - PyTorch library for geospatial data

Datasets: - EuroSAT - Sentinel-2 scene classification (10 classes) - UC Merced Land Use - High-res classification - DeepGlobe - Segmentation challenges - SpaceNet - Building detection

Philippine Context

Assessment

Formative Assessment (During Session)

Self-Check Questions:

  1. ✓ Can you explain the difference between a perceptron and a multi-layer network?
  2. ✓ Why does ReLU work better than sigmoid in hidden layers?
  3. ✓ What does a convolutional filter “learn” during training?
  4. ✓ How does pooling provide translation invariance?
  5. ✓ When would you use U-Net instead of ResNet?

Interactive Exercises: - ✓ Complete all TODO cells in theory notebook - ✓ Implement manual convolution from scratch - ✓ Visualize at least 3 different activation functions - ✓ Compare filter responses on forest vs urban areas

Summative Assessment (End of Session)

Knowledge Check (10 questions, multiple choice): - Neural network components - CNN architecture understanding - Application selection (classification vs segmentation) - Data requirements and constraints

Practical Demonstration: - Explain CNN decision-making process for a sample image - Sketch appropriate architecture for given EO task - Estimate data requirements for Philippine use case

Readiness for Session 4: - ✓ Understand TensorFlow/Keras workflow conceptually - ✓ Recognize data preparation needs - ✓ Anticipate training challenges

Next Steps

NoteAfter Session 3

You now understand CNN theory and operations! Session 4 puts this knowledge into practice with hands-on implementation.

Session 4: Hands-On CNN Implementation

You’ll build and train: 1. ResNet Classifier: Palawan land cover (8 classes) 2. U-Net Segmentation: Flood mapping (Central Luzon)

Using TensorFlow/Keras with real Sentinel-2 data.

Continue to Session 4 →

Extended Learning Paths

Path 1: Deep Dive into Theory - Complete CS231n Stanford course - Implement backpropagation from scratch - Study optimization algorithms (Adam, RMSprop)

Path 2: Explore Advanced Architectures - Attention mechanisms (Transformers for EO) - Vision Transformers (ViT) - Self-supervised learning (SimCLR, MoCo)

Path 3: Philippine EO Applications - Contribute training data to PhilSA Space+ - Develop CNN model for local area - Publish results in Philippine GIS conference