Mapping what is where on the earth’s surface is currently done primarily through overhead aerial and satellite images. This is useful in many ways but is limited in terms of portraying things like land-use classifications or public sentiment about a given location or landmark.
Professor Shawn Newsam, a computer scientist in the School of Engineering at the University of California, Merced, is aiming to enhance that mapping using georeferenced multimedia — specifically, photos and videos that have been provided voluntarily by users of social media websites like Flickr.
The idea recently earned Newsam the National Science Foundation’s Faculty Early Career Development (CAREER) Award, which will provide him research funding of $497,208 over five years to pursue the viability of using these multimedia collections as volunteered geographic information (VGI) for mapping purposes.
VGI is a field in which geographically relevant information is provided voluntarily by individuals. One example is the Audubon Christmas Bird Count, which for more than 100 years has used “citizen scientists” to gather data and create a census that aids in knowledge about bird populations and conservation efforts.
To date, though, researchers have not yet examined the possibility of using the millions of geotagged ground-level images and video provided by users of social media sites as VGI, nor have they looked into the potential benefits of using such data for purposes other than things like the automated text tagging of submitted images.
The project will only use multimedia shared under a Create Commons license, which allows them to be copied, distributed, edited, remixed and built upon, and no attempt will be made to identify individuals appearing in the image and videos.
“I think there are a lot of interesting challenges and opportunities in realizing the full worth of this data,” Newsam said. “I am glad that the NSF thinks this is an important problem and worth funding.”
Newsam has already used what he calls proximate sensing — using large numbers of images and videos taken by users on the ground in a certain geographic area — to create large-scale maps of both developed and undeveloped regions. This project will extend that work further, using social multimedia to map geographic information not observable through other sensing means.
For example, different categories of retail stores or indoor recreational spaces might look the same from above, but ground-level imagery from inside the buildings could make it possible to decide what is being sold or what sports are being played and categorize them accordingly. And mapping out public sentiment, such as how scenic a particular location is, could be useful in applications from real estate to tourism.
“This data is useful for observing geographic information that is difficult or not possible to observe from overhead imagery such as from satellite or aerial platforms,” Newsam said. “At the heart of the problem is being able to perform automated image and video understanding well enough to infer useful geographic information such as land-use classes.”
The project includes a strong educational component, as well, in that Newsam will be developing K-12 curriculum for spatial literacy — a set of abilities related to working and reasoning in the spatial world.
It’s not the first significant award for Newsam, a founding faculty member at UC Merced. He received a U.S. Department of Energy Early Career Scientist and Engineer Award and a Presidential Early Career Award for Scientists and Engineers, both in 2007. Newsam, with fellow UC Merced professors Qinghua Guo and Ruth Mostern, recently founded the campus’ Spatial Analysis and Research Center, known as SpARC.
