Time Manipulation in Audio Fiction

Working with audio is a linear time-based artform. When listening one must experience a linear flow of time that moves from start to finish, much different from the non-time-based arts such as painting, where the eye can choose how it moves through the perceived space. While audio may be constricted to this particular perception of time, how the passage of time is perceived can be altered in subtle ways that enhance the listener’s experience.

Time is malleable, or at least our perception of time is. When producing audio, one need not work linearly, it is possible to step record, program MIDI out of sequence etc., hence our use of the term realtime when recording the actual time during which a process or event occurs, such as recording someone speaking. We don’t have to produce in linear time, but ultimately the end result must be experienced in a linear fashion. That being said, it is still possible to alter the listeners perception of the flow of time.

Listening to a recording of spoken voices is a different experience than listening to a conversation taking place in front of you. The average American English speaker tends to speak at about 150 words per minute. This speed is also typical of audiobook recordings, but if you plan on including music and sound effects the reading should be slowed down. Music has a tendency to fill in the “gaps” between words and will alter the listeners perception of the speed of the reading. For this reason, it is worth slowing down when reading, giving the listener space to digest the words and in the case of multiple voice actors, allowing an extra fraction of a second between speakers. This will help to differentiate the dialog and let the listener understand the text without missing the content, along with making clean edits easier.

Likewise with music, the speed of the rhythms will alter the listeners perspective about how fast the recording actually is. Music with transients (such as a snare hit) can also distract from the legibility of the spoken word, so should be used with a light touch. Full spectrum instruments, such as synthesizers, occupy more of the audible range at high energies and will tend to increase the perception of the spoken word moving faster. For the latter it is recommended to shave off a few decibels under 20hz to prevent bass masking and the buildup of frequencies that can interfere with (and make feel faster) the spoken word.

While music can greatly alter the perception of the flow of time, there are also many fun tricks for actually manipulating the flow of time, some with standard audio effects, others through the use of psychoacoustic effects such as Risset’s rhythmic effect (a layering of rhythms that results in the apparent endless increase in tempo).

One example of a time-based audio effect is reverb. Reverberation deals with both time and space in that the addition of “reflections” gives the impression of the sound taking place in a particular environment. The larger the space, the longer it takes for the reflection to return to the listener, so not only does an effect like reverb add environment but when using it you are also altering the time-based elements of the sound. In audio, time and space are intimately related and altering one can often alter the perception of the other. The Doppler effect is a perfect example of how time and space are interrelated to the perception of sound. Reverb can also give the impression that the sound is taking place in a fictitious “past” and is often used during flashback scenes to differentiate the apparent location in time. This can also be done using the “telephone effect” by rolling off both the high and low end of the spectrum using your equalizer.

Pitch shifting and time stretching can also be used to manipulate the flow of time. Let’s say that the characters in the story enter a time machine, or portal – stretching or compressing (with the time stretcher, not a compressor which deals with dynamics) their dialog can give the impression that their flow of time has altered. This technique should be used lightly as it can quickly make the spoken word unintelligible. Reversing the audio can also give the impression of time flowing backwards but by nature will make the speaking unintelligible as it will actually be playing backwards.

Looping, delay, and echo can also be useful effects in that repetition can be a form of time manipulation. When a sound is looped it will repeat endlessly, however our perception of the sound will still change over time. Echo and delay can also emulate spatial environments that in turn can affect the perception of the flow of time within a given space.

Jean-Paul L. Garnier

Jean-Paul L. Garnier lives and writes in Joshua Tree, CA where he is the owner of Space Cowboy Books, a science fiction bookstore, independent publisher, and producer of Simultaneous Times podcast. In 2020 his first novella Garbage In, Gospel Out was released by Space Cowboy Books and in 2018 Traveling Shoes Press released Echo of Creation, a collection of his science fiction short stories. He has also released several collections of poetry: In Iudicio (Cholla Needles Press 2017), Future Anthropology (currently being translated into Portuguese), and Odes to Scientists (audiobook - Space Cowboy Books 2019). He is a two time Elgin Nominee and also appeared in the 2020 Dwarf Stars anthology. His new collection of SF poetry, Betelgeuse Dimming has just been released and is available as a free download audiobook / ebook at spacecowboybooks.bandcamp.com. He is also a regular contributor for Canada’s Warp Speed Odyssey blog. His short stories, poetry, and essays have appeared in many anthologies and webzines.