Image to Music

[Terry King]

Dennis Miller posted a very interesting article some time ago discussing programs that convert images to music (Say It With Pictures). I got the impression from this article that the programs work only with static images, not with videos or animations. Can someone recommend software for converting video files to music? thanks, Terry

[Andrew Hill]

Jitter can be used in this way in association with its siblings MAX and MSP from Cycling74.com. However unless you can find ready made patches on the net you could spend a fair while creating the effect and associations that you desire.

[Rick]

Hi Dennis,
I too am interested in finding a fairly easy and ready made solution to this problem of conversion from video/movie into audio/music. I've been interested in doing this in some of my work exploration since 1967 when I saw my first video synth at the Art Institute in Kansas City. Lately, I too have figured on probably having to set up my own MAX/Jitter conversion myself on my own; later this year; but I'm getting old and worn out and if you find any ready made MAX/JItter patches for this kind of thing available out on the web for download please let me know; or even a not too expensive ready made software program that can do this would be fine. There are at least 2 nice software programs out there that I know of now and I own one of them that will do picts to audio conversion but I still have not been able to google to find one that can do the same with video/movie formats.

It is surprising since the move format is perfect for conversion into music where as the 2 dimensional photo/pict is really the most awkward and worst one in terms of composing within time. Moving Imagery already has the time parameter as a part of that visual experience and translating that into audio is perceptually a small step. Think of all the applications that do the inverse of this like Music Color organs or Visualizers using audio input to create visuals that are in synch to the inputed audio via various audio file types, but the incredibly useful tool for composers of being able to do the reverse of those visualizers and taking the flowing imagery in time and converting or translating it in synch into audio seems not to be.

Plus being able to set all kinds of parameters for making the conversion from the imagery flowing in time (movie) into the resulting audio would give the composer nice parameters to really get a useful result in audio too.

I have not yet started working in MAX/Jitter hardly at all yet; but I guess I'll have too if i want to do this; since surprisingly I can find nothing ready made yet that can do this. So if you hear of anything let me know please.
Good Luck

[Rick]

PS I guess being a composer is part of my bias in this. Being able to translate/convert music with true or merely correlated controls into imaging is all over the place these days in software at an affordable price. However, being a composer who wants to create both the imagery and the music and who really has no desire for collage or eclectic or combining non-organic development I find myself in using both mediums simultaneously on a one way street. I can compose organic musical compositions and via today's software translate/convert them into flowing images that are in true and correlated sync with the interrelationships and parameters within my musical composition; but can not do the reverse. So without being able to have a 2 way street here in audio/imagery conversion. the only way I can see to maintain organic unity of the work is to always write the music and then portray it visually and never being able to take my visual compositions and in a reverse process portray them in music. Plus, if one can do both then one has the opportunity for a infinite amount of development of new materials too just from re inputting the visual into audio and then reversing the process on and on and then all of the resulting materials having underlying organic unity yet with a tremendous diversity in many various manifestation. It would be wonderful. However for now all one can do unless one writes up a Jitter/MAX set up is to write music and portray it visually which is so very very limiting uncreative with only that one way street available right now.
If I did not care about an 'underlying organic unity' or had much greater tech know-how to accomplish this I guess this would not be such a major hurdle in this work for me.

[Dennis Miller]

Hi. Arts Sync can do this:
http://www.hkbu.edu.hk/~lamer/download.htm

The author recently sent me a patch that allows for writing video in non-real time so you can get a better resolution when going the other way. He's at [e-mail] if you want to contact him.
Also, U and I software's Artmatic has some capabilities in this direction:
http://www.uisoftware.com/artmatic/index.html
but again, the main purpose is the other way round.

Also check out Isadora
http://www.troikatronix.com/isadora.html

Any form of data out can control any type of parameter, be it audio or video (a very nice patching network UI).
Hope some of those get you going in the right direction.
Best.
D