// The Challenge

Autonomous Vehicles (AVs) are an emerging technology with many challenges along the way. The fundamental technology utilized by AVs to see and understand the world currently have many issues with correctly detecting, identifying and reacting to the objects that occupy the road. Cyclists represent a unique challenge for AVs. Light, nimble, and fast - cyclists can be hard to detect by sensors and unpredictable for the onboard analytics and machine learning algorithms.

My research goal was to find a way to bridge the gap through the development of novel, passive or active interference / communication method that would increase the visual identification of cyclist by AV sensors and AI. This solution will be most effective in situations where AV systems are least effective, at night, poor weather, and high-density urban areas. Concept validation was accomplished with thorough tests analogous to AV sensory systems, camera sensors and object recognition algorithms. In addition, user interviews and ethnographic observations were conducted to provide insights into user needs and behaviours.

How might we increase accurate detectability of cyclists by AVs as this emerging technology moves towards maturity?

// Problem Breakdown

X.1 Bicycle Low Mass + Surface Area

Bicycles represent a specific challenge for AV sensors due to their open structure - low mass and minimal surface area. The tube frame and wire construction provide little opportunity for LiDAR to receive enough laser strikes for proper detection. Radar resolution is not fine enough to distinguish bicycles from other objects. This leaves AV cameras to do most of the work in identifying cyclists.

X.2 Bicycle Quick Unpredictable Movements

Bicycles are highly agile vehicles and pose a challenge for the predictive AI systems.

X.3 LiDAR Poor Performance in Harsh Weather

Harsh weather conditions can inhibit LiDAR from working effectually. Heavy rain, fog, and snow will interact with the laser pulses from LiDAR, showing up as a large solid cloud surrounding the car, and blocking the detection of objects in the environment. LiDAR is most effective in clear weather conditions and at night with no interference from the sun.

X.4 Camera Poor Performance in Darkness

AV cameras are most effective during the day where the natural ambient light illuminates the environment and the objects within. At night or low light conditions cameras are limited to only seeing objects directly illumined by substantial artificial light sources.

X.5 A.I. Risk of Incorrect Object Identification

Objects detected by sensors will be interpreted by the AVs AI and classified as to what they are. Classifications govern how an AV predicts and reacts to an object, allowing for safe encounters on the road. If a classification is off, than the predictions and reactions will be off as well, leading to potentially dangerous situations. These misclassifications are called false positives / false negatives.

// Opportunity and solution Search

Op 1. - Retroreflective Detection

While AV camera sensors are hindered at night/low light, they are still able to detect objects that are illuminating or illuminated directly by light. Retroreflective material is an effective method to increase object visibility in low light road situations by sending the light from an AVs headlights directly back and into the cameras.

Op 2. - 2D Barcode Identification

Barcodes are a method of transmitting data from a passive surface to a computer through a scanner or camera that reads the visual information, either black and white bars or squares that represent digital information. An accurate way to passively communicate information to AVs. Making the barcode retrorefelctive would make it an effective solutions that overcomes the challenges AVs have with detection at night and harsh weather.

Op 3. - 360° 2D Barcode Visibility

Having various barcodes correspond to the relative placement on a user would allow an AV not only to understand that there is a cyclist when it detects the barcode, but also understand what part of the cyclist it is looking at. This would allow it to better orient the cyclist within the environment.

Op 4. - 2D Barcodes Mapped to Hand Signals

As a user moves their arms from a standard riding positions to give a hand signal, the positions of the barcodes on the jacket will change its orientation on the cyclist and in space. These different barcode positions can correspond to specific cycling hand signals, allowing the cyclist to communicate intend to surrounding AVs.

// Preliminary Concept Validation

To validate the concept direction, a test that would simulate the video camera system on an AV was created to determine if a retroreflective QR Code would make for an effective method to communicate with an AV in harsh environmental conditions. A 20x20 cm retroreflective QR code on black fabric was made and staged in front of a video camera at a distance of 40 meters, at night with minimal ambient light and heavy fog. In this situation, an AV would mostly be relying on its radar to see, with its LiDAR inhibited by interference from the fog, and cameras only able to see sufficiently illuminated objects.

Results from video gathered show the retroreflective QR code and retroreflective markers being the only objects visible in the video, and the detail in the QR code being reasonably clear and definable from the rest of the environment. The overall concept of a retroreflective 2D barcode as an effective communication method in harsh environmental conditions was validated.

// User Understanding

Video observations of commuter cyclists were conducted to gain insights as to what types of apparel, equipment, and habits are common with users and inform the design concept. With most commuter cyclists observed wearing technical jackets, it was conclude the cycling jacket would make an effective platform for the development of the retroreflective 2D barcode concept. In addition, the design solution will have to maintain functionality with backpacks, and utilize a dark colour scheme to maintain minimal disruption to the users regular habits.

// Jacket Surface Analysis

With the selection of the cycling jacket to carrying the design concept forward, an understanding of the garment and the dynamic factors of rider movement, body position, and how they effect fabric movement and surface distortion needed to be established. An understanding of these factors of the garment and materials informed placement and design of the retroreflective code.

A jacket mock-up was created with a checkered pattern of 1x1 inch squares on the exterior surface. This was photographed simulating cyclists riding positions and hand movements where taken to see how much jacket surface area would be covered and what fabric distortion would occur.

// Sketching Concepts

// Prototyping at Maxport

The prototype was made in collaboration with Maxport Limited in Hanoi, Vietnam. The amazing R&D team at Maxport were able to take my patterns, technical drawings, mock-ups, design intent and make it into the prototype that made the concept tangible.

// The Outcome