neural networks — from perceptrons to cnns
Implementing neural network fundamentals (perceptron, Widrow-Hoff, MLP) and applying deep learning with CNNs for image classification on the Wang database.
system_log // neural_networks
This Master 2 project had a dual objective: first, build neural networks from scratch to understand their fundamentals; second, apply deep learning to real-world image classification and compare two distinct strategies.
mission_scope
Classify 10 image categories from the Wang database (beach, dinosaurs, flowers, etc.) using two competing approaches:
model_based_approach
Use pre-computed image descriptors (JCD, PHOG, CEDD — color, texture, and shape features) fed into a Multi-Layer Perceptron (MLP). Compare against a k-NN baseline.
data_based_approach
Build a Convolutional Neural Network (CNN) that learns features directly from pixels — end-to-end classification with Conv2D, MaxPooling, and Dense layers.
fundamentals_implemented
Before jumping to Keras, we built from the ground up:
- Simple perceptron with
signandtanhactivations - Widrow-Hoff gradient descent learning rule for binary classification
- Multi-Layer Perceptron with one hidden layer
optimization_techniques
- Data augmentation to fight overfitting on small classes
- Transfer learning with pre-trained backbones
- Hyperparameter grid search (learning rate, batch size, layer count)
key_result
The CNN approach significantly outperformed the descriptor-based MLP, confirming that learned features beat hand-crafted ones for natural image classification — even on a relatively small dataset. Confusion matrix analysis revealed that visually similar categories (flowers vs. gardens) remained the hardest to separate.
Repository: [Coming soon]