Sound spectrum visualizer from a NetStream in AS3

We recently had to work on some sound spec­trum in AS3 for DJ ANDY’s new web­site and the idea was to use the RSS feed of his exist­ing iTunes pod­cast on the infa­mous m4a for­mat. Here I found out a silly thing. You can’t load any­thing else than mp3s in a Sound instance. I know the sound geeks will cru­cify me but it’s like the old Flash who could only load some jpeg images and was unable to load pngs. Every­thing looks com­pli­cated when it comes to the Sound.

Any­way, I fig­ured I’d use a Net­Stream to load the m4a song. My google pet told me there were other help­less peo­ple ask­ing how to pull a Sound­Chan­nel out of a Net­Stream but it doesn’t seem to be the way to do it.

The answer is rather sim­ple but its imple­men­ta­tion is a bit weird :

var ba:ByteArray;
//computeSpectrum(output:ByteArray, FFTMode:Boolean=false, stretchFactor:int=0)
SoundMixer.computeSpectrum(ba);

Basi­cally you’ll need to start the usual Net­Con­nec­tion and Net­Stream stuff, grab your song and lis­ten on ENTER_FRAME events to ana­lyze the sound spec­trum with this method.

Here’s what the doc says about it :

Takes a snap­shot of the cur­rent sound wave and places it into the spec­i­fied ByteAr­ray object. The val­ues are for­mat­ted as nor­mal­ized floating-point val­ues, in the range –1.0 to 1.0. The ByteAr­ray object passed to the out­putAr­ray para­me­ter is over­writ­ten with the new val­ues. The size of the ByteAr­ray object cre­ated is fixed to 512 floating-point val­ues, where the first 256 val­ues rep­re­sent the left chan­nel, and the sec­ond 256 val­ues rep­re­sent the right channel.

If you want to use this bytear­ray as a source for a spec­trum visu­al­izer, you’ll have to exper­i­ment doing some maths and bit­wise cal­cu­la­tions, all that in some loops. Using ba.readFloat() will give you the next value in the byte array. So your loop will likely per­form 512 (or 256 if you only want one chan­nel) iter­a­tions to use all the val­ues of the spectrum.

This method of ana­lyz­ing the sound spec­trum on the fly has two major weaknesses :

1. if you got mul­ti­ple sounds play­ing, you can’t tell them apart with this method as it gives you a spec­trum for all the sounds together.
2. if you intend to use it for seri­ous sound appli­ca­tions, you’ll want to enable FFT (using a Fourier trans­for­ma­tion) and the cal­cu­la­tion will gen­er­ate a notice­able lag, mak­ing your ana­lyzer appear unsyn­chro­nized, late.

Luck­ily for us and in most appli­ca­tions, the sound spec­trum is meant as a visual enter­tain­ment, noth­ing more.

There’s a very good sam­ple on todaycreate.com with insight­ful com­ments in case you don’t really know where to start.

Addi­tional and detailed expla­na­tions are avail­able on the Flex 3 lan­guage ref­er­ence.

A great thing how­ever, is the way you can get a lot of infor­ma­tion on an m4a input. There are three events cur­rently supported :

• onText­Data gives you the cur­rent text of the play­ing m4a. It could be any­thing and in our case it’s the name of the cur­rent track.
• onIm­age­Data gives you the cur­rent image of the pod­cast. Just like the texts, the image can change if your pod­cast got chapters.
• onMeta­Data is impor­tant to get the dura­tion of the stream. Addi­tion­ally, the event object tells you of a lot of things about the stream like the sam­ple rate, the chan­nels, the tags, etc.

Depend­ing on how the m4a is doc­u­mented by the pod­cast cre­ator and the soft­ware one’s using, you could even chap­ter­ize a pod­cast in order to get some­thing sim­i­lar to cue points in the chap­ters prop­erty in the meta­data. You won’t get every­thing you want though. I would have appre­ci­ated to get a list of all the text data and their seek time but you can’t get them before­hand. You must play the stream in order to get the events one by one so if you really need a list of the “text points” (they aren’t cue points), you’ll have to make one manually…