Imagine if shopping was as simple as snapping a photo of a shirt you liked and hitting “search.”
A group of University of Illinois Urbana-Champaign researchers have essentially done just that.
Their tech is called FashionMatch. It’s a visual search engine that analyzes every detail of a garment, then combs through a massive database to find similar items of clothing based on color, proportions, and patterns, then direct a customer to where they can purchase an item of clothing.
Here’s how it works. Say you’re intrigued by a dress in a magazine, a friend’s sweater, or you’re just dying to know where you can get a Mark Zuckerberg-style gray t-shirt. Take a photo of the item of clothing, then hit search. The program then analyzes the photo and provides a variety of search results for similar items of clothing. If you find one you like, click the image and the app shows you a Google map of retailers. If the search results don’t match what you’re looking for, you can go back, enhance the image, and search again.
“The algorithm has mastered many computer vision challenges,” said Vassilios Vonikakis, a research scientist at Advanced Sciences Digital Center (ASDC), a UIUC research outpost in Singapore. “Even with shadows and cluttered backgrounds, the algorithm can identify which pixels of the image correspond to the person, to the garment, and to the background.”
“It can then identify the skeletal structure of the person, which makes it possible to tell the fashion-related proportions of the garment, like how long the sleeves are, the proportion of the neckline to the waist, general shape, and more,” he added.
Here’s a video that shows how it works:
Currently Vonikakis said it works best with planar surfaces, whereas more complicated items, such as jewelry, might need a different algorithm. However, this technology could work beyond fashion–he added that furniture companies have expressed interest as well.
Vonikakis said the research team anticipates that visual search is going to be the next trend in search engines, and realized that there’s a huge potential for this tech in the fashion industry, where style blogs and social media drive major trends. Already many fashion bloggers’ Instagram photos include a LikeToKnow.It link, which allows followers to shop items featured in a particular outfit.
Research for the technology started at UIUC’s Coordinated Science Lab, led by PhD student Bernard Ghanem, who handed it off to ADSC for commercialization. The team received a $285,000 grant from A*STAR, an agency in Singapore that encourages research commercialization, to further develop FashionMatch. Already they’ve sold the license to five companies.
Image credit: Pixabay CC0 Public Domain