Fixing the Mango Microphone issue – part 2

In the first part of this short series I have demonstrated that Mango has somewhat broken XNA Microphone object, at least when it is used in Silverlight. Even the standard Microsoft’s own Silverlight Microphone Sample generates a broken sound record with some data missing, if you change the buffer length from default 500 ms to 100 ms.

Look at the sound sample recorded with 500 ms buffer on Mango, or with 100 ms buffer on NoDo:

Debug Wave recorded by NoDo 7.0 Emulator in SDK 7.1

Joined 100 ms NoDo or 500 ms Mango Debug Wave records on both phone and emulator look similar to this graph (click on it for a bigger picture).

The same sound sample recorded with 100 ms buffer on Mango:

Debug Wave recorded by Mango 7.1 Samsung Omnia 7 with 100 ms buffer

Joined 100 ms Mango Debug Wave records on both phone and emulator look similar to this one. The wider (louder) marker waves are unevenly spaced, so some data is missing.

My next step was to measure the real rhytm of the buffers generated by the Microphone. Note that you cannot use System.DateTime.Now property, because of its ca 50 ms resolution. Instead, we will use System.Diagnostics.Stopwatch object, which provides better than 1 ms accuracy.

In theory, the Microphone should fire the BufferReady event every 100 ms. But the reality is quite different.

This is the list of the intervals – measured in milliseconds – between the subsequent 100 ms BufferReady events in the (emulated) NoDo:

101
51
151
101
51
150
51
152
101
51
151
51
151
51
101
151
51
152
51
101
152
51

As you can see, there is a regular pattern of the three 100+, 50+ and 150+ ms intervals, with some swaps. Ten intervals usually make ca 1011 ms. The length of the buffer is always 3200 B and the buffer contains 1600 x 16bit samples, which make 100 ms of sound. Buffers are perfectly connected to each other and no sample is missing or superfluous. (I haven’t yet cracked the interesting question where are going these surplus 11 milliseconds.)

Let’s switch to Mango:

97
139
69
105
68
135
70
104
138
69
139
35
138
138
35
139
69
105
137
70
138
69

This is recorded on Samsung Omnia 7. The emulator produces different values, but with a similar pattern: This time there are four, not three typical lengths – ca 100, ca 140, ca 70, ca 35. The sum of ten intervals is usually ca 1005 ms now, so it’s better, but the output data is incomplete. It seems that the intervals longer than 100 ms overflow the buffer.

The Microphone object is part of Microsoft.Xna.Framework.Audio namespace. It is dependent on the XNA Framework game loop. You can use it in the Silverlight Windows Phone project, but you have to simulate the game loop with a code like this:

DispatcherTimer dt = new DispatcherTimer();
dt.Interval = TimeSpan.FromMilliseconds(33);
dt.Tick += new EventHandler(dt_Tick);
dt.Start();

void dt_Tick(object sender, EventArgs e)
{
     try { FrameworkDispatcher.Update(); }
     catch { }
}

33 milliseconds is a timer interval used in the Microsoft’s Silverlight Microphone Sample. I have used the same value, but my next step was to experiment with different timer intervals. Indeed, the resulting Microphone tick lengths were somewhat different, but there was always some data missing, overflowing the buffer – the fundamental issue hasn’t disappeared.

Perhaps you already suspect the solution: We just need a bigger buffer! So, let’s record it with a 6400 B buffer.

Debug Wave recorded with a 6.400 B buffer

Folks, this looks promising! At first look, when we erase the gaps, and join the signal segments, the resulting record may be perfect… Let’s try it in Excel…

6400 B buffer segments compacted in Excel

Oops!😦 What could have gone wrong???

Why are some segments too long, but no segment is too short? These 35 ms segments should have been short, I guess? Wait! Do I clear the buffer every time? No, it wasn’t necessary in NoDo, because the buffer was always filled up to its fullness in every Microphone tick. In Mango, the Microphone buffer must be cleared before every BufferReady event.😦 Let’s go on!

Cleared, recorded and compacted 6400 B buffers.

Bingo!

So the winning process is:

  1. Allocate bigger than nominal buffers.
  2. Clear them.
  3. Fill them.
  4. Join only the filled parts of them.

This seems weird. The Microphone object has GetSampleSizeInBytes method. So we should just invoke this method after (or before) every tick and it should return the actual buffer size… but no, it doesn’t work this way. I have played with it for some time, but it always returns just the BufferDuration x 3200 value. It doesn’t reflect the real tick record size, just its ideal size, which is always matched in NoDo, but never in Mango…

The issue with the XNA Microphone object in Mango with 100 ms buffer is that the real buffer length is variable and there is no working API to get its length!

The good news is that no data is really lost. We just don’t know we actual size of the one tick record, so we have to figure it with some workaround. The way to go is to count zeros from the end of the buffer backwards. It may be slightly wrong in some cases, when the sound samples are actually zero at the end of the partial records, for example when the wave crosses the X axis and has exactly zero value at the end of the record. But these cases are quite rare and we can neglect them – at least if we are building a tuner.

In the next and the last part of this mini-series I will publish the source code of the Carousel object, which uses the above described methods to deliver solid data from the capricious Mango Microphone. The living proof of its reliability is my app Accurate Tuner Pro, probably the best pitch tuner for Windows Phone.

INSTALL ACCURATE TUNER PRO UNLIMITED TRIAL FOR FREE

What should Microsoft do now? Change GetSampleSizeInBytes of the Microphone object to return the actual size of the just recorded buffer.

Let’s hope it’s fixed in Tango. But still, Tango rollout is going to take some time, so the Carousel object may help you.

To be continued.

5 Comments

Filed under WP developement

5 responses to “Fixing the Mango Microphone issue – part 2

  1. Pingback: Why the Microphone doesn't work in Mango and how to fix it – Part 2

  2. According to MS documentation http://msdn.microsoft.com/en-us/library/microsoft.xna.framework.audio.microphone.getsamplesizeinbytes.aspx Return value of GetSampleSizeInBytes: Size (10 ms block aligned), in bytes, of the audio buffer. So here is first problem of Microphone class, but you are true about checking for zeros – that isn’t too good idea when is silence on mic. Anyway no one found any better🙂

    • oslik

      See the Axel’s comment below.🙂 The buffer can be filled with some other value, the best one is probably 0xffff. Such value can happen in the buffer too in theory, but it will be much, much less common than zero.

  3. Zero is a valid value and occurs twice per cycle so there is a high probability to find it in the sampled data. Why not fill the buffer with something less common, like 0xffff ?

    • oslik

      It’s a good idea!

      (Although zero is not so common as you may think. The wave usually crosses the X axis without a zero sample, the samples are for example …6, 3, -2, -7…)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s