Metro Gate Detection

Overview

AI Gatekeeper for Metro Safety

Metro gate vaulting (or crawling under) is one of the most common violations in urban rail transit, causing both fare losses and serious safety hazards. Traditional approaches rely on manual monitoring, which is inefficient and prone to missed detections. This project uses a YOLOv5 object detection model trained specifically to identify gate vaulting behavior, deployed on a Rockchip RK3588 edge device for end-to-end real-time inference from video stream input to violation alerts.

Challenge

Scene Challenges

Visual detection in metro gate scenarios faces multiple challenges:

Severe Occlusion — Narrow gate channels frequently occlude subjects via barriers and other passengers, requiring strong occlusion reasoning capability
Diverse Actions — Vaulting includes hurdle-style, hand-support, and crawl-under variants, lasting only 1-3 seconds, demanding high real-time detection performance
Complex Lighting — Underground platforms have variable lighting with backlighting, sidelighting, and fluorescent flicker
Edge Deployment — Must run on power-constrained embedded devices, requiring a sufficiently lightweight model

Model Design

Customized modifications based on the YOLOv5s architecture. First, we lightweight the Backbone by replacing standard convolutions in C3 modules with Depthwise Separable Convolutions, reducing parameters from 7.2M to 4.1M while maintaining feature extraction capability.

For the dataset, we collected 8,000+ frames from metro station surveillance videos, annotating three target classes: normal_pass, climb_over, and crawl_under. We used Mosaic + MixUp + random cropping augmentation, and introduced optical flow features from adjacent frames as auxiliary input to capture the temporal characteristics of vaulting actions.

Hardware

RK3588 Hardware Platform

Rockchip RK3588 is one of the most powerful domestic edge AI SoCs, integrating quad-core Cortex-A76 + quad-core Cortex-A55 CPU, Mali-G610 GPU, and a dedicated 6 TOPS NPU (Neural Processing Unit). Its NPU supports INT8/INT16/FP16 mixed-precision inference, making it ideal for deploying quantized YOLO models.

Our development board features 8GB LPDDR5 memory and a dedicated NPU accelerator. Compared to GPU solutions (like Jetson Orin Nano), RK3588 offers lower power consumption (typically 5-8W vs 15W) and better cost-effectiveness, making it ideal for large-scale metro station deployments. The 6 TOPS NPU is more than sufficient for YOLOv5s-level models, achieving 60+ FPS in real-world testing.

Deployment

Deployment Pipeline

Deploying from PyTorch model to RK3588 NPU requires the following key steps:

Step 1: Export ONNX — Export PyTorch model to ONNX format with fixed 640×640 input size
Step 2: RKNN Conversion — Use Rockchip's rknn-toolkit2 to convert ONNX model to RKNN format with INT8 quantization (500-image calibration set)
Step 3: NPU Deployment — Load RKNN model on RK3588 using rknn-lite2 runtime with NPU-accelerated inference
Step 4: Video Stream Integration — Capture RTSP video stream with OpenCV, feed frames to the model, and push results to the alert system via WebSocket

Optimization

Inference Optimization

When deploying on RK3588, we performed multi-dimensional inference optimization:

INT8 Quantization — Using rknn-toolkit2 quantization, FP32 model is quantized to INT8, reducing model size 4x and improving inference speed 2.5x with only 1.2% mAP drop
Multi-core NPU Parallelism — RK3588 NPU supports 3-core parallel inference via core_mask=RKNN.NPU_CORE_0_1_2 for maximum throughput
Zero-copy Optimization — DMA buffer enables zero-copy transfer from video frames to NPU, reducing memory copy overhead
Async Pipeline — Video capture, preprocessing, inference, and post-processing run as an async pipeline to maximize throughput

Code

RKNN Inference Code

rknn_infer.py

from rknnlite.api import RKNNLite
import cv2, numpy as np
rknn = RKNNLite()
rknn.load_rknn('yolov5s_gate.rknn')
rknn.init_runtime(core_mask=RKNNLite.NPU_CORE_0_1_2)
# Preprocess
frame = cv2.imread('gate_frame.jpg')
blob = cv2.resize(frame, (640, 640))
blob = blob[:, :, ::-1].transpose(2, 0, 1).copy()
# NPU inference
outputs = rknn.inference(inputs=[blob])
# Post-process: NMS + decode
detections = postprocess(outputs[0], conf_thres=0.5, iou_thres=0.45)
print(f"Detected {len(detections)} violations")

Results

Deployment Results

62 FPS

NPU Inference

91.3%

[email protected]

~6W

Power

<20ms

E2E Latency

The system achieves 62 FPS inference speed on RK3588 with end-to-end latency (including pre/post-processing) under 20ms, fully meeting real-time metro monitoring requirements. The INT8 quantized model is only 3.8MB, running stably at 6W power consumption, suitable for large-scale deployment across station gates.