Cutting-edge technology brings your voice into view at Canal Convergence in Scottsdale
By Chadd Scott
What does a human voice look like? What does your voice look like?
What does it look like projected as artwork?
Answers to those questions can be found at Canal Convergence | Water + Art + Light festival in Scottsdale, Arizona November 5th through the 14th.
MASARY Studios, in partnership with Scottsdale Arts and leveraging Epson Pro Series laser projectors, will install a monumental public artwork at the annual event allowing people to see their voices transformed and abstracted into animation projected on 50-foot sails suspended over the Arizona Canal.
“Say What You Will” listens to what one says and transforms the statements into expressions of video. Never the same twice, the system weighs tone and meaning as it transcodes speech into light. Participants will be invited to speak into any of the six kiosks on site and experience the transformation of their voices into light and video abstractions.
“Say What you Will” differs from other light-based projects by using spectral and sentiment analysis, machine learning, and custom-designed software.
What is “spectral analysis?”
“Spectral analysis essentially analyzes the various components that are present within a given signal; as it applies to ‘Say What You Will,’ spectral analysis means that audio signal is converted from time-domain representation to frequency-domain representation,” Sam Okerstrom-Lang, founder and principal of MASARY Studios, said. “By this, I mean that we take very short slices of the audio stream and then perform an efficient approximation of a Fourier Transform. The Fourier Transform is based on the idea that any complex sound can actually be represented by some mixture of sine waves.”
“The output of the spectral analysis shows us what kind of frequencies present within a given sound clip,” Okerstrom-Lang adds. “From there, we can perform additional analysis on the spectrum to get some high-level descriptors about the qualities of a given sound. These descriptors tell us things about the relative pitch range, bandwidth or focus of the sound, how noisy it is versus how pitched it is, and other similar things.”
Rendering of "Say What You Will" projection at Canal Convergence courtesy of MASARY Studios
It's easy to imagine artist, engineer, inventor Leonard da Vinci being fascinated by a project like this combining so many of his interest. The technology can be unapproachable to the rest of us. Fortunately, “sentiment analysis” is a little more straightforward.
Sentiment analysis performed upon text can provide estimations about how positive or negative a given statement is. “Say What You Will” makes use of several machine learning models to go from spoken voice to sentiment analysis. First, the audio is transcribed to text with that text ultimately going through a sentiment analysis model to produce a measurement of how positive or negative the statement is.
“The analysis data develops into specific parameters that are used to drive visual expressions (projections) using real-time visual animation software. We have built a procedural system that digests the specific various spectral and sentiment qualities that drive the properties of the visual expression in many various ways,” Okerstrom-Lang said. “For example, the sentiment drives the color of your expression. When a ‘positive' sentiment is submitted, the visual expression then drives towards a much more blue color palette where as if a submission sentiment is ‘negative’, the color palette will shift towards a deep orange. The spectral qualities drive speed, density, distance, and many various forces of the visual expression.”
“Say What You Will” activates more than two acres at the Scottsdale Waterfront. This year’s theme for Canal Convergence, appropriately, will be “Art and Technology” with “Say What You Will” as the centerpiece.
“MASARY Studios has found the perfect balance of using experiential technology and projection to create interactive, awe-inspiring public artwork,” Ramzi Shakra, product manager, large-venue projectors, Epson America, Inc, said.
Visiting hours for Canal Convergence are Friday, November 5th and Saturday the 6th from 6–10 p.m.; Sunday, November 7th through Thursday the 11th, 6–9 p.m.; Friday the 12th and Saturday the 13th again from 6–10 p.m. and closing night on the 14th from 6–9 p.m.