I’ve been working on a few MIDI-based programs in Processing recently, a couple of which are MIDI visualisers. These are programs that create a visual display based on the output from a midi file (similar to those seen at: http://www.musanim.com/). The visualiser will use the pitch, velocity and duration of midi notes to create the display. I’ve uploaded a video to youtube, but due to some problems getting videos out of processing, the quality is rather low – I hope you can discern what’s going on (edit: I’ve invested in fraps and improved the video).
[youtube http://www.youtube.com/watch?v=pe8oU4JPNeU&w=420&h=315]
So what have I used the MIDI pitch and velocity for?
Firstly, I’ve used colour to highlight harmonic relationships between notes (those unfamiliar with music theory may want to skip this bit). I’ve worked on the principle that the perfect 5th is the most consonant interval, and assigned colours to notes being played based on the relationships in the diagram below.
Circle of Fifths, with RGB colour wheel
So, we can see that the notes most consonant with C are F and G, which will have the most similar colours. Also, the least consonant interval (i.e. a tritone, or the “devil’s interval”) between C and F# also has the most jarring colour difference.
The velocity of each midi note was then assigned to the speed at which the arcs move, as well as the transparency of each arc (i.e. quiet notes are slow and barely visible, and loud notes are fast and bright) .The final length of each arc is decided by a mix of how loudly a note was played combined with how long it was held on for.
Practically, I used a piece of DAW software to play the midi file, and routed this through the excellent midi yoke (http://www.midiox.com/) to a processing sketch using the rwmidi library (http://ruinwesen.com/support-files/rwmidi/documentation/RWMidi.html). I’ve used the processing video library to capture frames, but as mentioned, there are some significant problems if you wish to sync this output with an audio file (especially if your processing sketch is even remotely complex). I’m looking into using some screen capture software instead, such as fraps.
If you’re interested in this sort of thing, feel free to contact me for the original processing sketch – the video doesn’t really do it justice.
Edit: I’ve included a new midi visualisation below.
[youtube http://www.youtube.com/watch?v=YyevhOfo6pA&w=420&h=315]
Luca 28th March 2014
Hi!
It’s a beautiful project!!
I want to ask you how did you choose the relationship between colours and notes.
Did you referred to some particular studies?
chrisballprojects 28th March 2014
Hi Luca,
I’m glad to hear you think it’s beautiful. I didn’t refer to any studies – but it may have been done this way before, I haven’t checked! It simply seemed the most logical way to do it – to overlay the two circles, one of colour (RGB) and one of consonance (circle of fifths). I was looking at processing an audio signal for similar effect, but I struggled with aliasing at lower frequencies and haven’t returned to it for a few months. I have some solutions, but they are tricky to implement in processing.
jordancald 6th May 2015
Hey Chris,
I am doing a project in Processing that is hacking the Philips Hue wi-fi enabled LED lights to interpret music frequencies as color/light frequencies. I came across your gorgeous MIDI visualization and it inspired me to investigate the means to have a corresponding sound-to-color visualization built alongside my hack in Processing. The project will eventually turn into a 2-screen installation. If I could see your Processing file, that would be great. I’m super curious about your algorithm.
Amazing work.
Thank you,
Jordan
chrisballprojects 6th May 2015
Hi Jordan, thanks for the comments – they’re appreciated. In my processing code I work with MIDI, so I have an exact reference for what frequencies I’m working with. I then set up some arrays:
int[] r = new int[128]; //setup arrays for R,G & B values - could be a single array of COLOR objects insteadint[] g = new int[128];
int[] b = new int[128];
Then in the setup void:
for (int i=0;i<128;i++){r[i]=int(255*(0.5+0.5*cos(10*PI*float(i)/12))); //precalculate R, G, B values
g[i]=int(255*(0.5+0.5*cos(10*PI*float(i)/12+2*PI/3)));
b[i]=int(255*(0.5+0.5*cos(10*PI*float(i)/12+4*PI/3)));
}
When I receive MIDI notes 1-128 I can just look up the colour values I need.
If you’re looking at working with an audio file/input (as opposed to MIDI), or want to see the full code, get in touch on my email address (on the About page). I should be able to help.