AI Rover Automates Medication Delivery in Assisted...
Built an AI-enabled UGV rover that combines verified medication delivery, voice-based resident interaction, emergency response,...
View Case Study
AI Robo-Spider Powers Smart City Surveillance and Emergency Response
Built an AI-powered hexapod robot that patrols a smart-city model, detects accidents and trapped persons, verifies recipients by face for parcel delivery, and reports every event to connected police, fire, RTO, and delivery portals on web and mobile.
About the Client
The target users are smart-city operators: police and emergency services, fire departments, RTO/vehicle authorities, and delivery hubs that need faster surveillance, quicker incident response, and safer inspection of difficult environments. The solution was built by Alea and demonstrated on a physical city model with roads, junctions, a police station, fire department, delivery hub, parking, a puzzle zone, and a broken-building rescue zone.
Project Overview
Cities need surveillance, emergency response, delivery, and hazardous-area inspection that work at once, but these tasks usually run on separate systems and depend on people being in the right place. Alea built a single robot that follows one operating loop: Detect → Analyze → Decide → Act → Report.
The Robo-Spider is a Hiwonder MiniHexa 18-DOF hexapod with two computers. An ESP32 handles gait, sensors, and servos in real time. A Raspberry Pi handles vision, face recognition, plate reading, the Gemini AI assistant, and the Smart City Hub. Operators and field officers reach it through a web hub and an Expo mobile app.
Manual → Autonomous
City Patrol and Surveillance
Unverified → Face-Verified
Parcel Handover
Delayed → Real-Time
Incident Alerts with Image and Location
Separate → Unified
Police, Fire, RTO and Delivery Portals
Key Features We Built
We designed a connected workflow so city services can see what the robot sees, act on it, and track the outcome in one place.
City Surveillance
The robot patrols the city model and streams a live camera HUD with distance, battery, and obstacle readings. A Gemini Live voice assistant describes the scene and can be interrupted mid-reply.
Smart Parcel Delivery
Orders are placed from the mobile or web hub. The parcel is handed over only after the recipient’s face matches an enrolled face, then the order status updates to delivered on both portals.
Obstacle & Puzzle Navigation
Ultrasonic sensors brake the robot at 22 cm. It then commits to one turn direction until the path is clear past 32 cm, so it does not oscillate left and right.
Accident Detection
Gemini vision identifies a simulated collision, rollover, or water incident from the camera frame. The system raises an alert with the image and location on the police dashboard and stores the recording as evidence.
Search & Rescue
In the broken-building zone, the robot detects possible trapped persons and vehicles using face detection and plate matching. It creates a survivor alert with snapshot, zone, and confidence, which moves from pending to dispatched to rescued.
Vehicle Identification
Gemini reads number plates from camera frames, and an Indian state/RTO parser maps each plate to its registration details. Matches against the authorized vehicle registry trigger the notification workflow and feed the RTO portal.
Challenges and Technical Solutions
A walking robot has to balance, sense, see, and report at the same time, and each of those runs at a different speed. The design had to keep real-time motion fast while letting AI and city workflows evolve independently.
1
Real-Time Walking and High-Level AI Could Not Share One Processor
Challenge :
Gait control needs tight timing, while computer vision and Gemini calls are heavy and variable.
Solution :
Split the robot into two computers. The ESP32 runs inverse kinematics, 21 PWM servo channels, IMU, ultrasonic, and IR sensing. The Raspberry Pi runs Flask, OpenCV, and Gemini. They talk over a USB serial protocol with compact commands and telemetry every 150 ms.
2
Smooth Omnidirectional Walking on 18 Joints
Challenge :
Six legs with three joints each must walk forward, strafe, and turn without stuttering.
Solution :
Implemented a tripod-style omnidirectional crawl driven by velocity commands (vx, vy, omega). The Pi reads distance about every 40 ms and does not resend forward commands while the path is clear, which prevents gait stutter.
3
Obstacles in a Tight Puzzle Zone
Challenge :
Simple stop-and-turn logic can make a robot wobble left and right in a corridor.
Solution :
Built a puzzle navigator with 22 cm brake and 32 cm clear thresholds and a persistent turn direction. It falls back to a simulated corridor if the sensor cable is down, so the demo keeps running.
4
Turning Camera Frames into City Decisions
Challenge :
Accidents, plates, faces, and road junctions each need a different kind of vision.
Solution :
Combined on-device YuNet + SFace for face ID with Gemini vision for accident description, plate reading, and road-turn detection, with an OpenCV Haar fallback. A natural-language Gemini Robotics orchestrator turns prompts like “Patrol Sector 4 and scan for obstacles” into move, rotate, and stop steps, with an emergency abort.
5
Safe, Authorized Handover and Vehicle Lookup
Challenge :
Parcels and vehicle data must reach only the right person or authority.
Solution :
Delivery releases cargo only after a face match. Plate and owner lookups use an authorized SQLite vehicle registry rather than an open feed. Web pages sit behind a login, and API calls accept a password key.
6
One Robot, Many City Services
Challenge :
Police, fire, RTO, and delivery teams need different views of the same robot.
Solution :
Built a Flask Smart City Hub with themed police, fire, and delivery panels, plus an Expo mobile app with department-locked tiles, a robot joystick, face checkout, and RTO, Police, and Fire portals on a shared REST API.
Outcomes and Achievements
01
Autonomous Smart City Operations
Built a dual-computer AI robot capable of patrolling, navigating obstacles, detecting incidents, identifying vehicles and people, and responding to commands across a simulated smart-city environment.
02
Real-Time Emergency and Security Response
Enabled accident detection, survivor alerts, face-verified parcel delivery, and vehicle identification, with incident images, locations, evidence recordings, and status updates sent to the relevant authorities.
03
Unified Web and Mobile Control Platform
Connected the robot to a Smart City Hub and Expo mobile app, giving police, fire, RTO, and delivery teams shared access to live robot data, alerts, controls, and service-specific workflows.
AleaIT Role in Making This Happen
Robot Firmware & Hardware Integration
Configured the ESP32 firmware for gait, posture, sensors, and the serial telemetry link on the Hiwonder MiniHexa.
Vision & AI Services:
Built face ID, Gemini plate reading, accident description, road-turn detection, the voice assistant, and the robotics orchestrator.
Smart City Hub & Backend
Built the Flask web hub, SQLite data model, puzzle navigator, and the police, fire, and delivery panels.
Mobile App
Built the Expo app with robot joystick, face checkout, delivery tracking, and the Police, RTO, and Fire portals.
Harness the Power of Technologies to Drive Business Growth in No Time!
Hire dedicated developers & shape your business journey with our top-rated website & mobile app development services.
