In a world of rapid technological developments, autonomous vehicles, Baited Remote Underwater Videos (BRUVs), and deployed cameras are making it easier than ever before to collect underwater images and video footage. In fact, they’re generating such huge volumes of data that it’s impossible to analyse them all manually.

But scientists are using AI to help. In a new paper published in Ecology and Evolution, researchers at the University of Gothenburg (EMBRC Sweden) have demonstrated how machine learning can manage and process video footage efficiently and effectively.

The study, which received funding from the DTO-BIOFLOW project, shows how automated marine observation can provide vital insights into our changing ecosystems.

From the late 1990s, researchers in Sweden used underwater robots to film an area of rock wall in the Koster Fjord. They gathered reams of data, returning to the study site on various occasions between 1997 and 2023. The footage that was gathered over these 26 years was put on hard drives and kept on a shelf. The data weren’t standardised – because they had been gathered through many different projects and not on reproducible transects – and there was far too much of it for anyone to analyse manually. Despite this, thankfully, no-one threw the files away.

“We are lucky that the underwater photographers at the Tjärnö Marine Laboratory chose to keep this data until now, when we can use machines to go through it,” says project lead Matthias Obst, a marine ecologist at the University of Gothenburg (EMBRC Sweden).

Christian Nilsson, a Master’s student at the University of Gothenburg trained an AI computer vision model to recognise the features of 17 different benthic species from 1,879 annotated image frames. Once able to classify the different species, the model could count their abundance.

This analysis was possible thanks to SUBSIM: the Swedish national platform for subsea image analysis. The platform has been designed to help people analyse and manage their marine data. Using the National Academic Infrastructure for Supercomputers in Sweden (NAISS), the model produced robust results from 4.4 million images in just a few hours.

Out of the 17 species they focused on, 13 of them displayed significant changes in abundance (eight positive, five negative).

The Koster Fjord is part of the Kosterhavet National Park marine protected area (MPA) on the northern west coast of Sweden. In this protected area, shrimp trawling is regulated, trawling shallower than 60 metres is banned, and there are 11 trawling-free zones below 60m.

Reviewing the footage enabled the researchers to see the long-term impact of these trawling bans. For example, coral and anemone populations were increasing. “Animals that capture nutrients through filtration, such as mussels, anemones and soft corals, do not thrive when bottom trawls stir up sediment from the seabed,” says Obst. In a promising sign, “several of these species have recovered significantly” since the bans were put in place.

The team also noticed other changes. As the water temperature increased, smaller suspension feeders that were more tolerant of warmer waters were doing well while larger, heat-sensitive species were struggling. “We see that heat-sensitive species are losing their living environment in shallow waters due to the warm temperatures there,” says Obst.

For some species, the marine protected area has helped them to recover from the impacts of trawling. “But there are some organisms that may not be saved by the protective measures,” he says.

New MPA sand restoration efforts may be needed to protect the temperature-sensitive organisms as waters continue to heat up.

The study demonstrates how technologies that enable marine observation can shed light on how ecosystems are affected by a changing climate and what can be done to protect them. It was part of the EU-funded Digital Twin of the Ocean (DTO) programme and received some funding from the DTO-BIOFLOW project.

Decoding the mysteries of the vast ocean has never been a simple task, but improvements in technology are helping us gain a clearer picture of the ocean’s depths. The success of this project – demonstrating that the protections put in place in the Kosterhavet National Park are having a positive impact on the marine ecosystem – is a great example of how we can use AI to put data to good use, instead of leaving them to gather dust.