Experiments and Results

This section documents the iterative process of developing the rabbit detection and counting system, focusing on experimental setups, dataset utilization, model training, and performance analysis. It also provides insights into why certain decisions, such as switching from YOLOv5 to TensorFlow Lite, were made during the project.

Experimental Setup

The experiments were conducted across multiple environments to optimize the development and testing processes:

1. MacBook Pro 2020 (Intel Chip Model):

   • Processor: 1.4 GHz Quad-Core Intel Core i5

   • RAM: 8 GB 2133 MHz LPDDR3

   • Used primarily for initial development and small-scale testing.

   • Limitations: Slow processing, limited memory for deep learning tasks.

2. University's Cloud Virtual Machine:

   • Specs unknown but moderately powerful compared to the MacBook.

   • Used for testing larger-scale data processing and training.

3. Google Colab (Free Version):

   • GPU: Tesla T4

   • RAM: 12 GB

   • Used for initial training runs. However, limited runtime and resource constraints hindered extended experiments.

4. Google Colab Pro:

   • GPU: Tesla V100

   • RAM: 25 GB

   • Allowed longer runtimes and more computational power, making it the optimal choice for final model training and testing.

Dataset Description

The dataset consisted of video recordings captured from a rabbit farm. The data was converted into individual frames, which were then annotated using LabelImg. Each rabbit in the frames was labeled with bounding boxes.

• Distribution of Dataset:

  • Training Set: 70%

  • Validation Set: 15%

  • Test Set: 15%

• Preprocessing Steps:

  • Frames resized to 300x300 pixels (required for TensorFlow Lite model).

  • Pixel values normalized for consistency during training.

• Training Set: 70%

• Validation Set: 15%

• Test Set: 15%

Preprocessing steps included:

• Resizing images to 640x640 pixels

• Normalizing pixel values

Model Configuration

Initially, YOLOv5 was chosen due to its reputation for accuracy and real-time performance. However, several challenges prompted a switch to TensorFlow Lite:

1. YOLOv5 Limitations:

   • Required significant computational resources, making it inefficient for low-powered devices like laptops or virtual machines.

   • Deployment challenges due to its lack of native support for edge devices like mobile phones or embedded systems.

2. TensorFlow Lite Advantages:

   • Optimized for lightweight deployment, ideal for real-time applications on limited hardware.

   • Simplified integration with edge devices and cross-platform compatibility.

Final Model:

• Framework: TensorFlow Lite

• Model File: detect.tflite

• Input Image Size: 300x300 pixels

• Confidence Threshold: 50% for rabbit detection.

Training Process

The training process involved multiple iterations to fine-tune the model for accurate rabbit detection:

1. Transfer Learning:

   • Used a pre-trained TensorFlow model (MobileNet SSD) as the base.

   • Fine-tuned on the annotated rabbit dataset to save time and computational resources.

2. Training Parameters:

   • Batch Size: 16

   • Epochs: 50

   • Optimizer: Adam

   • Learning Rate: 0.001

3. Google Colab Pro was utilized for the training process due to its GPU acceleration, allowing faster iterations and better results.

Evaluation Metrics

The following metrics were used to evaluate the model's performance:

• Precision: Ratio of true positive detections to total detections.

• Recall: Ratio of true positive detections to actual rabbits in the frame.

• mAP (Mean Average Precision): Averaged precision across all classes (used to evaluate overall performance).

• F1-Score: Harmonic mean of precision and recall.

Sample Metrics (Hypothetical Values):

• Precision: 85%

• Recall: 78%

• mAP: 81%

• F1-Score: 81%

Experiments Conducted

• Experiment 1: YOLOv5 Training (Baseline)

  • Objective: Establish a baseline using YOLOv5.

  • Results: Precision of ~70%, Recall of ~65%.

  • Issues: Long training times and inefficiency on low-powered hardware.

• Experiment 2: Transition to TensorFlow Lite

  • Objective: Address limitations of YOLOv5 by switching to TensorFlow Lite.

  • Results: Improved deployment efficiency and reduced resource usage, with minimal compromise in accuracy (~10% drop in precision).

• Experiment 3: Data Augmentation

  • Applied augmentations like random flips, rotations, and brightness adjustments to improve model robustness.

  • Results: Slight improvement in Recall (~3%) with a consistent Precision.

• Experiment 4: Real-Time Testing

  • Deployed the TensorFlow Lite model on real farm footage for testing.

  • Observed results in challenging conditions, such as varying lighting and partial occlusion of rabbits.

Results Analysis

The TensorFlow Lite model outperformed YOLOv5 in terms of deployment efficiency while maintaining competitive accuracy:

• YOLOv5: Better suited for high-resource environments but impractical for real-world rabbit farms with limited hardware.

• TensorFlow Lite: Balanced accuracy and efficiency, enabling real-time rabbit detection on standard systems.

Sample output:

• Frame-by-frame rabbit counts were logged into CSV files.

• Real-time detection visuals showed bounding boxes around rabbits with confidence scores.

Visual Evidence

real time rabbit detection with confidence scores