I had a few days ago a couple of minutes and I wanted to take a look into three.js.
I had the idea to use Audio Data API to get the sound spectrum of a song and combine this with three.js particles.
Just for an experiment…

As I needed a song that I could use without get fucked by those labels/artists I started a CC search and found Galdson’s Denilio – awesome.

A few minutes later the Experiment was ready and started. And it was so beautiful in my Eyes! But my computer is really crappy so I couldn’t enjoy and my time was over too…

Yesterday I started the experiment and I was unhappy again because of performance issues. But this was the original purpose of the Experiment, get a feeling for performance…..

Anyway I hacked a quick FPS keeper and voilá.

I promised to share but first you will need a Mozilla with the Audio Data API you can get it here:

Linux:
firefox-3.7a1pre.en-US.linux-i686
Mac builds:
10.5
10.6
Windows:
firefox-3.7a1pre.en-US.win32
The Experiment is here
cheers

Just for a short overview,

I sometimes play drum and bass on parties. To improve the audience experience, I use flash to visualize the digital culture I’m living. For this, I wrote some applications in flash and air. These apps are more experiments to find out how my sound can influence visual media to generate a special atmosphere.

Sometimes some people notice what I am doing or know what is happening. Therefore, questions came up and it is difficult to have a productive conversation at a party….

So why flash?

I think the main reason was that I wanted to experiment with Flash and later on with the new features of the player 10.

So let us get started.

The first step to reach the stuff I had in mind was to grab the audio signal. I have a line-in, so let us take a look what possibilities I have in flash to use this. I found the flash.media package providing some classes to access the microphone and the sound spectrum. I found out that it was not really easy to get apply the computeSpectrum method on the microphone. So I thought. O.K. I’ll code a Java app that catches the audio signal, computes spectrum and send this data as bytearray to flash. However, this was not very cool, even having some java code, I could re use and I wanted to stay with flash.

I explored the possibilities and none of them were easy and latency efficient for my purposes. I really needed some time to find the solution and it was sooo simple !

Setup a streaming Server on your computer. My choice after some tests was Icecast2 here is a sample of my config:

 xml | 
 copy |
?
 
01
02
<icecast>
03
	<limits>
04
        	<sources>2</sources>
05
 <queue-size>0</queue-size>
06
 <burst-on-connect>1</burst-on-connect>
07
 <burst-size>0</burst-size> 
08
    </limits>
09
    <authentication>
10
        <source-password>hackme</source-password>
11
        <relay-password>hackme</relay-password>
12
        <admin-user>admin</admin-user>
13
        <admin-password>hackme</admin-password>
14
    </authentication>
15
 
16
    <hostname>localhost</hostname>
17
	<listen-socket>
18
<port>6666</port>
19
    </listen-socket>
20
    <fileserve>1</fileserve>
21
<paths>
22
        <logdir>./logs</logdir>
23
        <webroot>./web</webroot>
24
        <adminroot>./admin</adminroot>
25
        <alias source="/" dest="/status.xsl">
26
    </alias>
27
    <logging>
28
        <accesslog>access.log</accesslog>
29
        <errorlog>error.log</errorlog>
30
      	<loglevel>3</loglevel> <!-- 4 Debug, 3 Info, 2 Warn, 1 Error -->
31
</logging></paths></icecast>

Setup a streaming client on your computer like Edcast and choose the right audio in.

When you get this programs running you will have a streaming server running under the following URL:

http://localhost:6666/stream

or if you are using shoutcast :

….. /;stream.nsv

So now, you can catch this audio stream from your flash app like this:

 actionscript | 
 copy |
?
 
1
2
this._sfx = new Sound();
3
var req:URLRequest = new URLRequest("http://localhost:6666/stream");// /;stream.nsv
4
var context:SoundLoaderContext = new SoundLoaderContext(0, false);
5
this._sfx.load(req, context);
6
this._channel = this._sfx.play();
7
SoundMixer.bufferTime=0;  // works for me locally
8

Now you can get the spectrum over an onEnterFrame Event with SoundMixer.computeSpectrum. I am going to explain this in part II