For the birds

Nathan Jacobs leads team that developed BirdSAT, a tool for classification and ecological mapping of global bird species

Shawn Ballard 01.11.2024

Ground-level bird images (left) alongside their four most similar satellite images (right) for six different bird species. Note that the satellite images are similar to each other while also being relevant for the corresponding bird species. (Jacobs lab)

Facebook Twitter Linkedin Email

As birdwatchers, birders and fans of the popular board game Wingspan know, feathered friends are found worldwide in a variety of different habitats, from frosty Antarctic penguins to desert-dwelling cactus wrens to Midwest favorite cardinals. BirdSAT, a new tool for classification and ecological mapping of bird species, enables the creation of species distribution maps across any geographic region to provide important information to assist ecologists who monitor and protect endangered species.

Nathan Jacobs, professor of computer science & engineering in the McKelvey School of Engineering at Washington University in St. Louis, leads the research team that developed BirdSAT. The framework combines ground-level imagery of birds, location and time metadata, and corresponding satellite imagery to classify different species of birds and map their habitation anywhere in the world. First author Srikumar Sastry, a doctoral student in Jacobs’ lab, presented BirdSAT Jan. 6 at the Winter Conference on Applications of Computer Vision in Waikoloa, Hawaii.

“The two tasks – species classification and distribution mapping – are distinct and often challenging for computer models because we need to make use of subtle differences to categorize species into fine-grained classes,” Sastry said. “With BirdSAT, we’ve developed a unified framework leveraging different machine learning techniques to solve both tasks and accurately classify and map bird species globally. Beyond birds, our models are general enough to be easily extended to any species of interest.”

The framework relies on a cutting-edge deep learning architecture to enable fine-grained visual classification (FGVC) and to retrieve accurate information across data types (e.g. between ground-level bird images and numeric metadata). Sastry and Jacobs found that BirdSAT achieved state-of-the-art performance on FGVC for a challenging birds dataset and proved to be remarkably versatile and generalizable when transferred to other datasets.

“By seamlessly blending ground-level imagery with metadata and satellite data, BirdSAT allows us to create species distribution maps for any geographic region and for any species,” Jacobs said. “This not only enhances our understanding of bird species but also paves the way for the development of exciting new tools for conservationists, ecologists and policymakers in managing and preserving biodiversity.”

Sastry S, Khanal S, Dhakal A, Huang D, and Jacobs N. BirdSAT: Cross-view contrastive masked autoencoders for bird species classification and mapping. Winter Conference on Applications of Computer Vision (WACV), Jan. 4-8, 2024. https://arxiv.org/pdf/2310.19168.pdf

The McKelvey School of Engineering at Washington University in St. Louis promotes independent inquiry and education with an emphasis on scientific excellence, innovation and collaboration without boundaries. McKelvey Engineering has top-ranked research and graduate programs across departments, particularly in biomedical engineering, environmental engineering and computing, and has one of the most selective undergraduate programs in the country. With 165 full-time faculty, 1,420 undergraduate students, 1,614 graduate students and 21,000 living alumni, we are working to solve some of society’s greatest challenges; to prepare students to become leaders and innovate throughout their careers; and to be a catalyst of economic development for the St. Louis region and beyond.