Project: Developing Neural Networks for Image Classification

This academic project, part of the Master 2 program (IOI, Info, Math) at the University of Poitiers, had a twofold objective. First, to develop and master the fundamental building blocks of neural networks (Perceptron, Widrow-Hoff learning, MLP). Second, to apply this knowledge to implement and compare image classification systems (both “Full-Connected” and “Deep Learning” approaches) on the Wang database.

1. Context and Problem

The main goal was to apply Machine Learning concepts to the classification of real-world images. The project aimed to compare two distinct strategies for classifying the 10 image categories from the Wang database (Beach, Dinosaurs, Flowers, etc.):

1. “Model-Based” (Full-Connected) Approach: Use pre-calculated image descriptors (color, texture, and shape features like JCD, PHOG, CEDD) and feed them into a Multi-Layer Perceptron (MLP) for classification.
2. “Data-Based” (Deep Learning) Approach: Use a Convolutional Neural Network (CNN) to automatically learn the relevant features directly from the image pixels and perform end-to-end classification.

2. Learning Objectives

• Implement the basic components of an artificial neuron (Simple Perceptron with sign and tanh activations).
• Program a gradient descent learning algorithm (Widrow-Hoff rule).
• Build a functional Multi-Layer Perceptron (MLP).
• Master the Keras/TensorFlow framework to build “Full-Connected” (MLP) and “Deep” (CNN) architectures.
• Compare the performance of the two approaches (descriptor-based vs. data-based).
• Analyze results (confusion matrices, error rates) and implement optimization techniques (fighting overfitting, data augmentation, transfer learning).

3. Implementations Achieved

The project was divided into two main parts:

Part 1: Perceptron Development (Fundamentals)

• Created functions for the Simple Perceptron (1 neuron).
• Programmed the Widrow-Hoff learning algorithm for binary classification.
• Implemented a Multi-Layer Perceptron (MLP) with one hidden layer.

Part 2: Image Classification (“Full-Connected” vs. “Deep” Approaches)

• “Full-Connected” Approach:
  • Read pre-calculated descriptors (JCD, PHOG, CEDD, etc.).
  • Set up an MLP (Full-Connected) structure in Keras/TensorFlow to classify the 10 image classes.
  • Compared performance with a k-NN (k-Nearest Neighbors) classifier.
• “Deep” (CNN) Approach:
  • Created a sequential CNN model (Conv2D, MaxPooling2D, Flatten, Dense) to process raw images.
  • Trained the model for end-to-end feature extraction and classification.
  • Studied the influence of hyperparameters and optimized the model using Data Augmentation and Transfer Learning.

4. Technical Environment (Tech Stack)

• AI Frameworks: Keras / TensorFlow
• Python Libraries: NumPy
• Data Handling: Processing .txt and Excel (.xlsx) files
• Database: Wang Database (10 classes of real-world images)

Links

• GitHub: [Coming soon]
• Documentation: [Project Wiki]