Publications‎ > ‎

A Brief Historical Survey of Audio-Visual Intermedia Composition

Andrew D. Lyons. 
Composition Unit 
The Sydney Conservatorium of Music. 
The University of Sydney 
Sydney NSW 2000 Australia 
Email: tstex at(nospam)

Alchemical Foundations

Pythagoras had considered synesthesia to be the greatest philosophical gift and spiritual achievement. The earliest experiments in colour-music instrument design occurred in the alchemical haze of antiquity. Leonardo da Vinci made sketches of colour organs in the fifteenth century. Athanasius Kircher later triggered an interest in the area as a result of experiments with his "magic lantern" and the subsequent writings in his "Musurgia Universalis. . . " of 1662. John Locke also raised the subject in his 1690 publication, "An essay concerning human understanding."

Auricular Harpsichords

In his 1725 article, Louis-Bertrand Castel wrote, "Why not make ocular as well as auricular harpsichords? It is again to our good friend [Kircher] that I owe the birth of such a delightful idea."A working example of Castels' "Clavecin Oculaire", which made use of colour/sound correspondances described by Isaac Newton in his Optik of 1730, was never actually completed although plans were set out and a model was constructed in 1734.

The Colour Organ

The incidence of colour organ design and construction accelerated during the nineteenth century culminating in Alexander Rimington's "Colour Organ" of 1893. His book describing the colour organ and the colour theories on which it was based was published in 1911 as Colour_music: The Art Of Mobile Colour. That same year the Russian composer Alexandr Scriabin completed his Promethius, The Poem Of Fire.

Scriabin's Promethius

Promethius was composed and scored for orchestra and a device to perform colour which he called the "Tastiera per luce."Scriabin had first recognised his feeling for sound/colour correspondances following conversations with Nicholai Rimsky-Korsakov who also recognised such correspondances and utilised them in his orchestration. An account of Scriabin's experience of synesthesia can be found in a 1915 article published in the British Journal of Psychology. A description of both a successful performance of Promethius and the workings of the "Tastiera per luce" used in a March 1915 performance in New York can be found in the edition ofScientific American published soon afterwards. Kenneth Peocock addresses both Scriabin's synesthesia and his Promethius comprehensively in an article published in Leonardo in 1985.

Wassily Kandinsky

Being a fusion of two modalities, the development of colour music has been pursued by both synesthetic musicians and visual artists. The theory and works of Wassily Kandinsky were concerned to a large degree with abstracted musical forms. He describes an experience of Wagner's Lohengrin during the early 1890's: "All my colours were conjured up before my eyes. Wild, almost mad lines drew themselves before me. I did not dare to tell myself in so many words that Wagner had painted `my hour' in music. But it was quite clear to me that painting was capable of developing powers of exactly the same order as those music possessed."

Kandinsky moved to Munich in 1896 in order to realise this dream in the form of the abstracted art forms he describes in his numerous and highly influential treaties on abstract art. However ". . . Western readers may be interested to learn that while he was still in Russia during the years immeadiately following the revolution, Kandinsky was co-head of the Institute for Artistic Culture where he explored the objective regularities of the synthesis of music and painting and studied colour hearing (synesthesia), the psychological basis of this synthesis."Kandinsky completed his first non-representational painting and his treatise On The Spiritual In Art in 1910. Two years later an artist working in Paris had devised and commenced work on an abstracted musical animation that he called Coloured Rhythm.

Leopold Survage

"Leopold Survage was a familiar if taciturn figure in a circle of artists that included Picasso, Braque, Modigliani, Leger, Brancusi and others."Inspired by trends in abstract and cubist painting and the burgeoning development of cinematic techniques, Survage began work on his rhythm-colour "symphonies in movement." Survage said of his work in the area that, "Coloured music is in no way an illustration or an interpretation of a musical work. It is an autonomous art, although based on the same psychological principles as music."The numerous descriptions of Survage's 200 or so completed plates are all fairly exstatic in nature. Many descriptions written at the time appear in Russett and Starr's, Experimental Animation.

These descriptions, which are too lengthy to quote in full, and the few frames presented as examples, bear an uncanny similarity to the descriptions of coloured photisms resulting from chromaesthesia. Unfortunately Survage never found an animator to complete the frames between his abstracted key frames and his work on coloured rhythm was never completed.

Wilfreds Clavilux

The prohibitive logistics of painting tens of thousands of frames to create a cinematic animation were avoided by makers of dedicated sound/colour generating instruments. Perhaps the most famous example of such instruments during the first half of this century was Thomas Wilfred's "Clavilux" of 1922. The Clavilux performed displays of prismatic colour that many compared to the shifting lights of the Aurora Borealis and the abstracted forms of Kandinsky's painting. Performances of his "Lumia" occured throughout North America, Canada and Europe during his tour of 1924-1925. Most of Wilfred's performances were silent, however orchestral collaborations such as the performances of Rimsky-Korsakov'sScheherezade directed by Leopold Stokowski that occured during 1926 in Philadelphia were perhaps exceptions.

Bentham's Light Console

Wilfreds Clavilux pre-cursored the numerous coloured-music instruments that were constructed in the decades that followed. One such notable instrument was Frederick Bentham's "Light Console." Bentham used his console to perform coloured light accompanied by phonograph records of such works as Stravinsky'sThe Firebird Suite, and Scriabin's Promethius. These instruments created works that left critical writers of the time short of words with which to describe their beauty. "For this new colour-art might very aptly be called music for the eye. . . it is colour and light and form and motion, but it is not painting, nor sculpture, nor pantomine. . . "

Walter Ruttman

Whilst such instruments were logistically amenable to real time interaction and live musical performance, animated kinetic art in cinematic forms were beginning to appear in the visual entertainment arena. In the spring of 1921, Walter Ruttman showed his short film Lichtspiel Opus 1, in Frankfurt, Germany. The film was reviewed by Bernhard Diebold in the Frankfurter Zeitung under the heading " A New Art, The Vision-Music of Films." Diebold and his young freind Oskar Fischinger were "Greatly impressed with the work", which according to William Moritz, Fischinger's biographer, was "not only the first abstract film to be shown in public, but also a film hand tinted in striking and subtle colours, with a live synchronous musical score composed especially for it." "The concepts of sound painting or tone colour seemed literally to fulfill their meaning; content and character of the musical piece express themselves, silently moving in the forms and colours of the continuous motion picture."

Opto-Acoustic Notation

By 1933 the visionary Bauhaus artist and teacher Lazlo Moholy-Nagy had already published several articles about the possibility of creating music recordings using film as a compositional media and made a short film, The Sound ABC. The utilisation of synthetic sound on film was also being explored by Oskar Fischinger in his development of an abstracted musical animation language."Fischinger's basic aim was . . . to artistically interrelate the sensory modes of sight and sound into a totally synesthetic experience."His "opto-acoustic notation"consisted of a series of geometric shapes that represented musical elements drawn onto scrolls of paper before being photographed onto the film soundtrack. The likes of Norman McLaren and Len Lye were involved in extending the field of synthetic sound before the commencement of the Second World War.

John and James Whitney

In 1940 John Whitney, a filmmaker, composer and technical innovator and his brother James Whitney, a painter, began working with synthetic sound and a film animation language in order "to develop a unified bi-sensory relationship between film and music."Stimulated by the avante garde filmmakers of France and Germany in the early nineteen twenties these American artists created numerous films during the nineteen fourties and fifties that utilised unique machinery designed and realised specifically for their purposes. One example of their creative heritage are the slit-scan photographic techniques famous for such cinematic passages as the Stargate sequence of 2001 a Space Odyssey.

John Whitney and Computer Animation

After many years developing mechanical techniques with his brother James, John Whitney found himself making use of analogue computers sourced from military surplus centres as tools with which to continue his development of a bi-sensory relationship between film and music. His animations completed using this primitive technology were of sufficient quality to secure him long time on-going support from IBM. With these tools he became a pioneer in the field of computer generated abstract musical animations.

In his book published in 1980, John Whitney states that. . . "Music, as the true model of temporal structure, is most worthy of study among prior arts. Music is the supreme example of movement become pattern. Music is time given sublime shape. If for no other reason than its universality and its status in the collective mind, music invites imitation. A visual art should give the same superior shape to the temporal order that we expect from music."Whitney goes on to describe differential motion patterns as one means by which, "would-be music/image innovators" may attempt to play color "musically" on a two dimensional field such as a monitor with time placed on the cartesian z-axisas the principal axis of construction.

The Multi-Media Age

By the time that John Whitney's three sons were beginning to move into the field of experimental animation in the late 1960's they were not alone. The multi-media age had dawned. The different arts were being presented together, at the same time, in a variety of settings on an unprecedented scale. Some members of this popular artistic movement took to describing their art as intermedia. It would seem that such terms as audio-visual intermedia, colour-music and kinetic art are all representative of the philosophy and technology of the age in which which they were realised.

As Bulat M. Galayev explains: "The techniques of music-kinetic art . . . are not susceptible to unification; each music-kinetic art instrument is unique, as is any work of art."This suggests that with each new advance in technology a new name will be developed to take its place amongst the "colour harpsichords" of history. This problem is exacerbated be the lack of cohesive tradition within the art. Musicians and painters, for example, acknowledge and revel in their tradition. Coloured-music/kinetic-artists however are traditionally characterised by an ignorance of their predecessors matched seemingly only by their keenness to claim the art as their own and give it a new name. The defiance of both a static terminology and technique may however be seen to be part of the integral nature of the art and perhaps also the source of its attractiveness to artists.

Selected Bibliography.

  • Aristotle. "De Anima." In The works of Aristotle, edited by W. D. Ross. Oxford: Oxford University Press, 1928.
  • Dube W.D. The Expressionists. London: Thames nad Hudson,1972.
  • Galayev, Bulat M. "The Fire of Promethius: Music-Kinetic Art Experiments in the USSR." Leonardo 21.4 (1988),383-396.
  • Goethe J.W. von. "Zur Farbenlahre." In Theory of Colors, translated by C.L. Eastlake. London: Frank Cass and Co, 1967 (1810).
  • Iamblichus. De Vita Pythagorus. tr. Thomas Taylor. London: J.M. Watkins, 1965.
  • Kandinsky, W. "On the Spiritual in Art" in Kandinsky: Complete Writings on Art, edited and translated by K.C. Lindsay and P. Vergo. London: Faber and Faber, 1982.
  • Kircher, Athanasius. Musurgia universalis: . . . New York: Barenreiter, 1988 (1662).
  • Kuenzli, Rudolph E. ed., Dada and Surrealist Film. NewYork: Willis Locker and Owens, 1987.
  • Moritz, William. "Abstract Film and Color Music." in The Spiritual in Art: Abstract Painting 1890-1985, edited by Maurice Tuchman, Judi Freeman and Carel Blotkamp. New York: Abbeville Press, 1986.
  • Peacock, K. "Synesthetic perception: Alexander Scriabin's color hearing."Music Perception 2 (1985) : 498.
  • Peacock, Kenneth. "Instruments to Perform Color-Music: Two Centuries of Technological Experimentation." Leonardo 21.4 (1988), 399.
  • Plummer, H.C. "Colour music - a new art created with the aid of science."Scientific American 112 (1915) : 343, 350-1.
  • Russett, Robert., and Cecile Starr. Experimental Animation : origins of a new art . London: DeCapo, 1988.
  • Scriabin, Aleksandr Nikolayevich. Prometheus The poem of fire ; Piano concerto in F sharp minor. London: Phonodisc, 1972.
  • Whitney, J. Digital Harmony: On the Complementarity of Music and Visual Art. (Peterborough, N.H: Byte Books, 1980), 44.
  • Youngblood, Gene. Expanded Cinema. London: Studio Vista, 1970.