Warehouse Surveillance System with AI-Powered Cameras

ReactAIMLMongoDByolov8Flask

Wednesday, January 15, 2025

Overview

The AI-Powered Warehouse Surveillance System is a cutting-edge solution developed to enhance safety, security, and operational efficiency in warehouse environments. Addressing critical challenges in traditional surveillance such as manual monitoring errors, delayed incident response, and inadequate threat prediction this system integrates three specialized machine learning (ML) models for Personal Protective Equipment (PPE) compliance, fall detection, and fire detection. Built with a robust architecture combining YOLOv8 for real-time object detection, a Flask API backend, and a React-based frontend dashboard, the system delivers automated threat detection, real-time alerts, and actionable insights.

Features

  • PPE Compliance Monitoring: Detects safety equipment (helmets, vests, gloves, boots, goggles) using YOLOv8, ensuring adherence to safety protocols with high precision (0.6973).
  • Fall Detection: Combines YOLOv8 with LSTM for pose estimation and temporal analysis to identify worker falls (mAP@0.5: 0.683), enabling swift incident response.
  • Fire Detection: Identifies fire and smoke with a balanced precision (0.7392) and recall (0.7526) on testing sets, critical for early hazard mitigation.
  • Real-Time Dashboard: A React-based interface visualizes live feeds, detection alerts, and analytics, powered by a Flask API for seamless data integration.
  • Automated Alerts: Utilizes Socket.io and SendGrid for instant notifications on detected anomalies, stored in MongoDB for audit purposes.
  • Video Summarization: Employs LSTMs and SUM-GAN/VASNet for keyframe extraction, reducing cognitive load on security personnel.

Methodology

The system follows a four-phase architecture:

  1. Data Preprocessing & Video Frame Extraction: Processes live CCTV feeds at 5-15 FPS using OpenCV and FFmpeg. Resizes frames to 640x640 and applies Gaussian Blur for noise reduction.
  2. Object Detection, Tracking & Classification: Uses YOLOv8 with CSPDarknet53 backbone for detecting workers, forklifts, pallets, and safety equipment. Implements DeepSORT with Kalman Filter for consistent object tracking across frames.
  3. Activity Recognition & Anomaly Detection: Detects falls, PPE violations, forklift proximity risks, and fire/smoke events in real-time. Prioritizes alerts based on confidence scores and event severity.
  4. Automated Video Summarization & Real-Time Alerts: Generates summarized video clips using LSTM and reinforcement learning. Delivers instant alerts via Socket.io and logs events in MongoDB.

Technology Stack

  • Machine Learning: YOLOv8 (Ultralytics), PyTorch, CSPDarknet53 for feature extraction.
  • Data Processing: OpenCV, FFmpeg, Gaussian Blur for preprocessing.
  • Backend: Flask API for video processing, model inference, and alert generation.
  • Frontend: React with Tailwind CSS for a responsive dashboard.
  • Database: MongoDB for event logging and historical data.
  • Datasets: PPE Compliance: 2,700 images (helmets, vests, boots, gloves, goggles, non-compliance classes). Fall Detection: 4,000 images (down, up classes). Fire Detection: 9,000 images (fire, smoke, normal classes).

Impact

This system addresses critical warehouse safety challenges, reducing injury and fatality rates while improving operational efficiency. By automating threat detection and compliance monitoring, it minimizes human error, enhances response times, and supports scalable deployment, making it a valuable tool for modern warehouse management.

Live Demo Video: