Presenting data as sound can open new opportunities for accessibility, engagement, and discovery, but the technique still faces challenges.

There’s no sound in space. Or is there?

With a technique called sonification, astronomers can translate data — ranging from stars' brightness to the strength of gravitational waves — into sound. The technique most obviously, and vitally, makes such information accessible to those who are blind or have low vision. But sonification also opens up new experiences for the general public and researchers alike, Anita Zanella (National Institute of Astrophysics, Italy) and colleagues report in Nature Astronomy.

Humans are generally a visually oriented species, to the extent that we sometimes neglect our other senses. But those other faculties have capabilities that seeing does not. For example, while we may blink or look away from visual cues, we’re always processing audio input (even when we think we’re not). We’re also capable of listening to several things at once, and we’re especially good at filtering out noise to home in on what we want to understand.

The latter is known as the cocktail party effect, meaning you can understand what your friend is saying even when others are talking loudly nearby. It’s not just a party trick — sonification can help us get at astrophysical signals hidden amid the noise.

Cardiff University’s Black Hole Hunter shows this effect in action. Click that link and you’ll see a simulated signal from two merging black holes swinging closer together until they become one in a final “chirp.” Then, you can try your hand at finding this signal buried in noise. Be warned: The noise is loud. Despite the roar, it’s surprisingly possible to make out the quiet chirp. (At least in Level One — the levels get progressively harder!)

Gravitational-wave signal and noise
The top frame visualizes the gravitational-wave signal expected from two colliding black holes. The bottom shows that signal amid a realistic depiction of the noise that affects gravitational-wave detectors. Fishing out the signal visually is impossible. But listen to the same data, and you may hear the quiet "chirp" amid the noisy roar. Try it yourself at Black Hole Hunter.
Black Hole Hunter / Cardiff University

Note that Black Hole Hunter is an online game built for public engagement, not a research project. In fact, more sonification projects (36%) list public engagement as their primary objective than other aims. Other objectives include research (26%), art (17%), accessibility (13%), and education (8%), according to Zanella's team.

Even so, “data as sound” has been enabling scientific discovery for decades. Most notably, a radio hiss led to the discovery of the Big Bang’s afterglow, the cosmic microwave background, in 1964. The late space scientist Donald Gurnett (Iowa University) also spent decades translating space mission data into sound. A particularly eerie one, and the focus of some of Gurnett's research, is the lightning-induced plasma waves that “whistle” through Earth’s upper atmosphere. Other lightning-induced sounds recorded by plasma instruments aided in the discovery of Saturnian lightning.

Earth "whistlers" are associated with lightning.
Similar "whistlers" are heard at Jupiter.
Voyager 1 recorded the crackles of Saturnian lightning. These radio emissions are the only direct evidence of lightning at Saturn, so far.

Now, sonification is making data — and discovery —accessible to new populations. For example, a new project called Sensing the Dynamic Universe shows brightness changes over time for all sorts of celestial objects, including quasars, supernovae, and pulsating variable stars.

Cofounded by Paul Green (Center for Astrophysics, Harvard & Smithsonian), this project is designed for outreach and education, with accessibility as a primary goal. The variable sources shown currently are curated to show the best examples of each type of object. “With luck and funding, I’d ideally like to expand it to allow users to see and hear lightcurves and spectra for objects of their choice,” Green says.

Sensing the Dynamic Universe translates the brightness of celestial objects, such as the Cepheid variable star here, into sound.
Center for Astrophysics, Harvard & Smithsonian

In addition to serving the blind and those with low vision, “data as sound” also works well for those with dyslexia, autism, and really anyone who prefers to listen to data, Zanella and colleagues note in their study.

An example of a project that combines accessibility with both public engagement and research is ARISA's Eclipse Soundscapes app. Since 2017, this app has provided sonic feedback to users as they explore a “rumble map” of a solar eclipse photo.

"[The rumble map] was amazingly useful for users who are blind and low vision, because it allowed them to build a mental map of what their sighted peers were seeing," says Henry Winter, the cofounder of ARISA and creator of the Eclipse Soundscapes app. "Anecdotally, we found that it deepened the experience for sighted users as well, as it engaged more senses.

While funding for the app has been cut, ARISA still plans to update it for the 2023 and 2024 eclipses. Since access to smartphones' haptic motors was released after the 2017 eclipse, ARISA will also be enhancing the rumble map.

Finally, ARISA is planning a separate citizen-science project, launching next year. It will enable anyone to record the reaction of the natural world — such as birdsong and crickets’ whirr — to the sudden darkening of a solar eclipse.

The examples given here are just some of many ongoing sonification efforts, which have been ramping up in recent years. Zanella and her colleagues rounded up such projects across astronomy, finding that the cumulative number has been increasing exponentially since 2010. As of December 2021, the number stands at 98.

But sonification is still very much a niche thing, they find. Part of the problem, the team acknowledges, is that there’s no standard approach to sonifying data. Anyone seeking to do so basically has to reinvent the wheel. To get to standardization, Zanella's team writes, we simply need more studies on how well different sonification techniques work.

And to do that we need more funding, says Winter. “Making meaningful sonification tools takes experienced designers, programmers, and scientists as well as conducting focus groups to ensure that the tools are usable and communicate the desired information,” he explains. “Finding grants that could potentially fund this type of work has been a challenge, but ARISA Lab has been fortunate to find partners in NASA and the National Science Foundation that believe in this work and want to ensure its success.”

Another problem is how to present error bars associated with data. Sonifying data uncertainty can be vital to distinguishing, say, whether a star is really getting brighter or whether we’re mistaking instrument noise and natural fluctuations for real change. One option, Green says, is to simply use louder volume for smaller errors, fainter for larger errors. But in many cases, it's difficult to add errors without confusing the listener.

There's a lot of work to do to make sonification a standard for astronomy, but the rewards will make the efforts worthwhile.


Advertisement

Comments


Image of Anthony Barreiro

Anthony Barreiro

August 15, 2022 at 4:37 pm

Thanks for this interesting report. The gravitational wave game is hard!

Have any musicians been involved in these data sonification projects? I wonder if error bars could be sonified using overtones to change the timbre of the tone. An oboe and a violin both playing the same note, say A over middle C at 440 Hz, sound very different because the two instruments produce different overtones. A datum with a small error bar could have a pure bell-like tone, while a datum with a large error bar could have a very crunchy sounding set of overtones, like a trombone or a kettledrum.

You must be logged in to post a comment.

Image of Anthony Barreiro

Anthony Barreiro

August 16, 2022 at 1:42 pm

You could also use more or less vibrato to sonify error bars.

You must be logged in to post a comment.

Image of Bill

Bill

August 17, 2022 at 11:28 am

Interesting ideas, Anthony. As a trained musician, I think I find the vibrato idea more intuitive than overtones. This is somewhat implicit in the Cepheid light curve example where deviations from the smooth curve come out as a different pitch. Another possibility would just be to put broadband noise through a variable bandwidth filter.

You must be logged in to post a comment.

Image of Anthony Barreiro

Anthony Barreiro

August 17, 2022 at 3:54 pm

Vibrato makes more sense than overtones.

I don't understand how a variable bandwidth filter works. I did a quick online search, but everything looks pretty technical. Could you explain in simple terms?

You must be logged in to post a comment.

Image of Bill

Bill

August 19, 2022 at 4:06 pm

Yeah, most of the stuff I could see online was pretty technical.

Random noise contains a wide range of frequencies all jumbled up. If you process it with a very narrow band filter, you get a recognizable tone at the center frequency of your filter. As you increase the bandwidth of the filter you include more and more of the jumbled up frequencies, but since they are still near to the center frequency you still should be able to pick out the pitch -- it just sounds noisier, until the filter bandwidth gets so wide that it just sounds like noise again.

This is actually the basic principle behind many musical instruments -- a noise source like a bow scraping or turbulent air flow is coupled to a resonator (filter) like a stretched string or air column that selects only the desired pitch out of the noise. (There's some feedback to the noise source, too, but this comment is already too long.)

You must be logged in to post a comment.

Image of Anthony Barreiro

Anthony Barreiro

August 19, 2022 at 5:29 pm

Thanks, that makes sense! I will never listen to an oboe or a violin in quite the same way again.

You must be logged in to post a comment.

Image of Bruce Frahm

Bruce Frahm

August 20, 2022 at 4:07 pm

I am a 50-year+ trombonist and a barbershop baritone. My guess is that more valuable for this is my 55 years as an amateur radio operator. I 'won' the game with one miss - at Level 5 if I remember correctly. It wasn't trivial..... but not especially challenging either. de KØBJ

You must be logged in to post a comment.

Image of Don-Kerouac

Don-Kerouac

August 28, 2022 at 12:34 pm

I won't reach my 55th Amateur Radio anniversary till next April! Lacking any useful musical talent, I am an avid CW operator. Congrats on being a Maxim Society Member!
Digging the gravitational merger wave out of the noise was like trying to work an ATNO runnig QRP from Heard Island, during an electrical storm on 160! 73 de Don K9NR

You must be logged in to post a comment.

You must be logged in to post a comment.