The SpeechRecognitionEngine class can load one or many Grammar objects. The MSDN page describes these Grammar objects in the following way:

The System.Speech.Recognition..::.Grammar class provides run time objects that allow an application to specify a specific combination of words, choices of words, and other speech elements that the Speech platform uses to identify meaningful phrases.

The Grammar object fully supports the W3C Speech Recognition Grammar Specification (SRGS) and Context Free Grammar (CFG) specifications. For more information, see SpeechRecognitionGrammar Specification.

Grammars may be precise word phrases, such as “Turn the computer off,” or provide choices, semantic lookup tables or wildcards, such as “Change color to” and a look up table of acceptable values.

An application’s recognition engine, as managed by instances of SpeechRecognizer or [T:System.Speech.Recognition.SpeechRecognitionEngine,] may create and load one or more instances Grammar, independently enabling or disabling particular grammars instance, and set Grammar properties such as priorities (Priority and weight (Weight)

So you can see that we can construct Grammars that will recognize certain types of input and then load these into the SpeechRecognitionEngine. These can be prioritized to help with conflict resolution as well as toggled on and off if you know that only certain grammars are appropriate for a certain context.

So how do we construct these grammars? Grammars can be constructed in a variety of ways, but we are going to examine the GrammarBuilder Class as this is the easiest way to programmatically construct a Grammar.

The simplest example is simple string matching. If you have a discrete atomic command (e.g. shutdown) that takes no parameters, you can initialize a GrammarBuilder with a string and it will fire its recognize event when that string is heard.

GrammarBuilder stringGb = new GrammarBuilder("shutdown");

If this was added to a Grammar and loaded in the Recognizer it would fire a recognized event for this Grammar whenever the word “shutdown” was heard. This is easy enough, but somewhat limited, because what if you wanted to also allow them to say “turn off” or “quit”? At this point you would have to add 3 Grammars, one with each of those strings. Enter the Choices class.

The Choices class allows us to create a phrase where one part of the phrase may have many different acceptable values. In our above example, we want the verbs “shutdown”, “turn off” and “quit” to be interchangeable. To show how this can be used with a phrase, lets make the user ask nicely by preceding either of those with “please”.

GrammarBuilder choicesGB = new GrammarBuilder("please");
Choices verbChoices = new Choices("shutdown", "turn off", "quit");
choicesGB.Append(verbChoices);

Now our grammar will trigger an event for all of the following phrases:

• please shutdown
• please turn off
• please quit

You can probably now imagine how you could combine strings and choices to come up with a variety of different recognizable phrases. However, there is a lot more that this engine can handle, but before we get into that lets review what happens when a phrase is recognized by the SpeechRecognitionEngine.

Remember that when a phrase is recognized, all that the engine will trigger is a SpeechRecognizedEvent to which you can assign a handler. This same event is fired for all phrases in all grammars that are loaded, so the problem becomes figuring out which phrase triggered the event. There are numerous strategies for dealing with this so I won’t go into detail here, but note that the SpeechRecognizedEventArgs object has a RecognitionResult that contains a pointer to the Grammar as well as the Text of the phrase in addition to any Semantic Values that you have assigned (more on that later). For an example see here.

Why take that little detour? Because once you’ve tried to actually do something with the result of a recognition event you’ll realize the utility of SemanticValues. Semantic Values allows you to assign values to certain responses so that you do not have to do annoying string parsing in the event handler.

Our first basic example will show how we can simply get the appropriate text in the event handler without having to do string parsing. Lets say that we know that the user is either going to say ‘yes’ or ‘no’ to a request, but we want to recognize synonyms of those phrases. We can do something like the following:

GrammarBuilder yesChoices = new Choices("yes", "yeah", "ok").ToGrammarBuilder();
yesChoices.Append(new SemanticResultValue(true));
GrammarBuilder noChoices = new Choices("no", "nope").ToGrammarBuilder();
noChoices.Append(new SemanticResultValue(false));
 
Choices allChoices = new Choices(yesChoices, noChoices);
SemanticResultKey choiceKey = new SemanticResultKey("YesNoBool", allChoices);
 
GrammarBuilder choiceGB = new GrammarBuilder(choiceKey);

What does this do for us? It creates a GrammarBuilder that has 2 choices. Each of these Choices has a SemanticValue that can be obtained in the recognition handler. Semantically there is a Yes choice (which is actually itself a choice of “yes”, “yeah” and “ok”) and a No choice (”no” and “nope”). The Recognition engine, since we have assigned semantic values to these, will set the “YesNoBool” variable to either true or false depending on which Semantic choice the user makes. This is great for us. No string parsing, and thus we can easily build up grammars with many synonyms and never worry about adding more complexity in the event handler.

So far we have looked at inputs that are just combinations of predefined words. This is great for many scenarios, but what if we want to allow more free form input? Lets say that our grammar is built to allow users to get a current stock price. You would probably want to match on any of the following phrases:

• Show me the stock price of foo
• Let me see the stock price of foo
• What is the price of foo

Here foo may be any stock ticker and lets say we’re lazy so we don’t want to hard code all of them. The solution lies in the DictationGrammar class. This class is exactly what it sounds like–a class for dealing with understand user dictation. Now, extracting the ‘foo’ from any of those in the event handler could be error prone, especially as you add more and more recognized ways of asking for a stock price. This is where semantic values come into play again. So lets see how this might work:

GrammarBuilder stockGB = new GrammarBuilder();
GrammarBuilder dictation = new GrammarBuilder();
dictation.AppendDictation("spelling");
 
stockGB.Append(new Choices("Show me the stock price of", "Let me see the stock price of", "What is the stock price of"));
stockGB.Append(new SemanticResultKey("DictationInput", dictation));

This code creates a choice of antecendents and then appends a SemanticResultKey whose name is DictationInput. Note that since we assigned no semantic value, the value that is assigned to DictationInput will be the actual text of the dictation. Finally, note that when we created the Dictation we passed the parameter “spelling”. This was to let the Recognizer know that we wanted the user to spell out the ticker name and that it should not interpret the input as words. If we had not passed that parameter it would have tried to match the input to words. That type of dictation might be useful if you were trying to build a voice IM client for example.

That’s all for now. Good luck and let me know if you build anything cool

We will start by creating a NotifyIcon object in our Window’s Constructor. Note that it is in the System.Windows.Forms namespace, and you may need to add a reference to your project in order to access it (it is not there by default in a new WPF project). You may also need to add a System.Drawing reference to your project as well. Once your references are in order, lets write some code to create the icon for us in the Window Constructor:

//Set up the system tray icon
_ni = new NotifyIcon();
_ni.BalloonTipTitle = "Qurbit Home Automation Engine";
_ni.BalloonTipText = "The engine is still running, it has just been minimized.  Double click to restore to normal size";
_ni.Text = "Qurbit Home Automation Engine";
//Add a double click handler
_ni.MouseDoubleClick += new System.Windows.Forms.MouseEventHandler(_ni_MouseDoubleClick);
try
{
    _ni.Icon = new System.Drawing.Icon("Images\\flag.ico");
}
catch (Exception e)
{
    Debug.Print(e.StackTrace);
}

The above code will create the necessary NotifyIcon object and give it an icon, text, etc. Customize it as is appropriate.

Next we want to make sure that when a user minimizes the window we show the icon and do not show our application on the task bar. To do so, we need to trap the Window’s OnStateChangedEvent. For more info on how window events work start here. In our xaml we need to add this attribute to the window element:

StateChanged=”Window_StateChanged”

Then in C#, we can add a handler for this. First however, let’s write a helper function for toggling the visibility of our window. It will take a boolean that indicates whether the window should live in the system tray or not.

private void windowInSysTray(bool inTray)
{
    if (inTray)
    {
        //Hide the window
        this.ShowInTaskbar = false;
        this.WindowState = WindowState.Minimized;
 
        if (_ni != null)
        {
            //Show the icon
            _ni.Visible = true;
            //Show the balloon tip
            //_ni.ShowBalloonTip(2000);
        }
    }
    else
    {
        //Show the window
        this.ShowInTaskbar = true;
        //Restore it
        this.WindowState = WindowState.Normal;
        //Bring it to the front
        this.Activate();
        //Remove the sys tray icon
        if (_ni != null)
        {
            _ni.Visible = false;
        }
    }
}

Finally, the event handler looks like the following:

private void Window_StateChanged(object sender, EventArgs e)
{
    //If the state is minimized then change its display type
    if (this.WindowState == WindowState.Minimized)
    {
        windowInSysTray(true);
    }
    else
    {
        windowInSysTray(false);
    }
}

Next we need to implement the double click handler for the notify icon to restore the application:

private void _ni_MouseDoubleClick(object sender, System.Windows.Forms.MouseEventArgs e)
{
    windowInSysTray(false);
}

At this point you should be able to minimize and restore your application as you would expect. There is on final bit of cleanup that we are going to want to include, and that is to handle what happens when the window is closed. If we allow the user to close the window with the close button, then we need to explicitly remove the notify icon or it will stay down there until the area is redrawn (the user mouses over it). For my app however I am going to make the close button also minimize the app to the sys tray and then you really close via a sys tray menu. So lets grab onto the OnClosing Method by adding the following attribute to our window object in the xaml code:

Closing=”Window_Closing”

Then lets write an event handler:

private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
{
    //Trap the event
    e.Cancel = true;
    //Put it in the system tray
    windowInSysTray(true);
}

Now, when we close the application, it minimizes itself to the system tray as well. Unfortunately, if you run it at this point there is no way to close the application smoothly, so lets add a context menu to the system tray icon that has a close option.

In order to get started, lets add a click handler to the NotifyIcon in the window constructor:

_ni.MouseClick += new System.Windows.Forms.MouseEventHandler(_ni_MouseClick);

Then we need to implement the click handler. It will be very simple, and just check for a right mouse click. If it detects one, then it will show the context menu:

void _ni_MouseClick(object sender, System.Windows.Forms.MouseEventArgs e)
{
    if (e.Button == MouseButtons.Right)
    {
        //Show the context menu
        System.Windows.Controls.ContextMenu menu = (System.Windows.Controls.ContextMenu)this.FindResource("SysTrayContextMenu");
        menu.IsOpen = true;
    }
}

In order for that to work we now need to add a context menu. There a couple of ways we could do this, but lets build ours in XAML so that down the road we can have some fun with it. To do this, lets add a context menu to our XAML file, as the first element within the window:

<Window.Resources>
    <ContextMenu x:Key="SysTrayContextMenu">
        <MenuItem Header="Open" Click="MenuOpen_Click"></MenuItem>
        <MenuItem Header="Exit" Click="MenuExit_Click"></MenuItem>
    </ContextMenu>
</Window.Resources>

Now we have a context menu, with two options. All that remains is to implement their click handlers. The open option is will just open up the window by calling our windowInSysTray method, and the exit one will remove our NotifyIcon (cleanup so that it doesn’t linger around) and then shutdown the application.

private void MenuOpen_Click(object sender, RoutedEventArgs e)
{
    windowInSysTray(false);
}
 
private void MenuExit_Click(object sender, RoutedEventArgs e)
{
    //Remove the system tray icon
    _ni.Dispose();
    //Shut it down
    System.Windows.Application.Current.Shutdown();
}

There you have it. Pretty simple stuff, and if you want to add more options or add fancier effects just edit the xaml file.

First lets start by looking at the Windows Clipboard in general. The clipboard, as we all know, stores blobs that the user has copied or cut from windows applications for future use. It stores this data in a variety of formats (if writing a windows application you can use the formats specified by the DataFormats class.) It is a convenient way to transport data between applications in windows. A simple example of writing to the clipboard in c# would then look like:

// For this example, the data to be placed on the clipboard is a simple
// string.
string textData = "I want to put this string on the clipboard.";
 
// After this call, the data (string) is placed on the clipboard and tagged
// with a data format of "Text".
Clipboard.SetData(DataFormats.Text, (Object)textData);

Doing this on in a windows app is great, and easy, because that is really what the api’s were designed for… but what if we want to do this in a web app? Why would we do this you ask, well one argument is that as web apps and client apps converge, users are starting to expect more and more of the familiar windows UI concepts to be available to them on the web. Yeah, we’ll say that is the reason.

In a web app, there are primarily 3 ways that we are going to run code that will interact with our user: javascript, flash, and silverlight. We’ll take a look at each of these and see what is possible.

Javascript

So what can we do with javascript and the clipboard? It depends on the browser and on the interaction that you want. Lets start with the simplest, and potentially cross browser action: execCommand. The execCommand method can perform a variety of operations, but the ones we care about are ‘copy’, ‘cut’, and ‘paste’. Because execCommand was built with enabling javascript based wysiwyg editors in mind, it operates on the selected areas of a web page. That is, text that has been highlighted or where there is an active cursor (e.g. in a text box). So how can we use execCommand? If you want to simply copy the selection to the clipboard, then it is trivial and you can just:

document.execCommand("Copy");

This copies the user’s selection to the clipboard.

What is good about the execCommand copy approach? It is potentially cross browser (not with default security settings in FF) and simple to implement. So long as they have javascript enabled you can use it. What is bad? In modern browsers the user will be prompted the first time that you attempt to do this and warned that you are trying to access their clipboard, which is a bad UX but necessary for security reasons.

Furthermore, you may only copy text that the user has selected in the browser.

For more examples of execCommand clipboard stuff, see this Geekpedia tutorial or this excellent JS WYSIWYG Editor tutorial.

The IE only alternative

An option, if for some strange reason you would like to build an application that only works in IE, is to take advantage of the clipboardData object. Using this, you can write and read directly from the clipboard and the user prompt is ignored when using the default security settings. Using this you can only take advantage of two ‘types’ on the clipboard: ‘text’ and ‘url’

To write text to the clipboard, you could then do the following:

var textToCopy = "This is some text for the clipboard";
window.clipboardData.setData("Text", textToCopy);

One nice thing about this, is that you can write arbitrary text to the clipboard (no user selection needed). But remember, this is Internet Explorer only. Mozilla does have a solution, but it is FF only. So if you want to use the clipboard in all browsers and not change security settings what can we do?

Flash

The best answer so far is flash. The idea (originally credited to Mark O’Sullivan) is to use the Flash setClipboard method to write to the clipboard. Why is this better than the javascript solutions? Because, the good people at Adobe did all the cross browser work for us. Now the only requirement is that the browser has a flash plugin, which given the penetration of flash is a fair bet. What we can then do is, when we want to copy something to the clipboard, we pass (via javascript) the the text to our flash object and let it do the clipboard dirty work.

Jeffothy Keyings has a good example of how to do this, and links to the open source flash object (_clipboard.swf) by O’Sullivan.

Silverlight

I looked and looked, and to the best of my knowledge Silverlight does not yet expose a method for accessing the clipboard. Alas.

Conclusions

So where does that leave us?

1. If you want cross browser clipboard access you will need to use flash.
2. If you do not want a dependency on flash, then you only have a good solution for IE

One last thing…

One more thing that is worth noting is that you can only write to and read from the clipboard from within the browser using the TEXT format. Why is that important? Because if you want to put images, or rich text on the clipboard you can’t. If you go into your browser and highlight some text with styles and then paste it into Microsoft Word, you see that the styles are preserved. That is because this is written to the Clipboard and read from it as RTF. Not only that, but if you paste it into Microsoft OneNote, you’ll find that you can see the original source of the clip. That is because as rich client apps, IE (or FF), Word and OneNote are not sandboxed in the same way that flash and javascript are. To see this more clearly try downloading Clipboard Inspector and looking at the format of items that you copy to the clipboard.

When clipping from a web browser you get the HTML option and there is header information showing source, etc:

When you write via javascript or flash you are forced to write out text, so when applications access the clipboard, even if you have valid HTML on there they will html encode the characters thinking they they are text, and thus not render them correctly. There are numerous reasons why this makes sense from a security perspective, but it is a pain and something that web developers need to be aware of nonetheless.

Lets look at a simple example.

Say you have 2 cars, a Honda Sedan which gets 30mpg and a Ford SUV which gets 15mpg. You put an equal number of miles on both cars and can’t get rid of the SUV because you need it for hauling SUV sized things.

Now you are proud of your eco-friendliness, you recycle your cans and buy only organic, free-range, local food, so you want to figure out what you can do to use less gas. You realize that you have enough money to upgrade one of your cars. You can replace the Sedan with a Hybrid that gets 55mpg or you can replace the SUV with a Hybrid that gets 22mpg. Which would you choose?

According to Larrick most people would replace the Sedan. Why? Because 50mpg sounds like a great thing! A gain of 25 mpg sounds far more Green Peacey than a gain of 7mpg.

Now, lets look at the math.

Lets say that you drive 1000 miles a month, evenly split between the two vehicles. That means that pre-upgrade we are burning:

500/30 + 500/15 = ~50.0 gallons of gas (16.6 for the Sedan, 33.3 for the SUV)

If we upgrade the Sedan, we get:

500/55 + 500/15 = ~42.4 gallons of gas (9.1 for the Sedan, 33.3 for the SUV)

If we upgrade the SUV, we get:

500/30 + 500/22 = ~39.3 gallons of gas (16.6 for the Sedan, 22.7 for the SUV)

So by upgrading the SUV, even though you only boost it by 7mpg you are actually burning less gas overall! Why is this? Because we are holding the number of miles driven fixed, thus the more important statistic is how many gallons per mile your car gets. Miles per gallon is useful when figuring out how far you can go on a tank of gas but less so, when you have a fixed distance to travel and are looking to figure out how much gas you’ll need.

Larrick’s paper concludes that people looking to save gas (or trees) should figure out the gallons per mile for their vehicles when considering upgrades. For our above examples we can see that the cars got (in units of gallons per 10 miles):

Sedan + SUV = 0.333 + 0.666 = 1 gallons/10 miles
Hybrid Sedan + SUV = 0.182 + 0.666 = 0.848 gallons/10 miles
Sedan + Hybrid SUV = 0.333 + 0.456 = 0.789 gallons/10 miles

Something to consider next time you get a car, and yet another feather in the behavioral decision theorists’ caps.

We begin by creating a new project in our Speech Engine solution, and setting it up as described in the previous tutorial. One this is setup, we can begin by creating a WeatherPlugin class that implements the ISpeechPlugin interface:

using Speech_Project.Core;
using System.Speech.Recognition;
using System.Xml.XPath;
using System.Xml;
 
namespace Speech_Project.Plugins
{
    [SpeechPluginAttribute("Weather",
        "This retrieves weather data from the Yahoo weather services API",
        "1.0")]
    public class WeatherPlugin : ISpeechPlugin
    {
        private string _grammarName = "Weather Plugin";
 
        #region ISpeechPlugin Members
        public Grammar getGrammar()
        {
           ...
        }
 
        public void handleSpeechInput(string input)
        {
            ...
        }
 
        public string getGrammarName()
        {
            return _grammarName;
        }
 
        #endregion
    }
}

Now we just need to build our grammar and implement the handlers. In order to make it easier to standardize the language that we’ll be using, lets create a helper class called WeatherSpeechChoices that will contain the strings that we recognize in our grammar. We will give it two commands to start; one will get today’s forecast and the other, tomorrow’s.

public static class WeatherSpeechChoices
    {
        //Inputs
        public const string forecastTomorrow = "What is tomorrow's forecast";
        public const string forecastToday = "What is today's forecast";
    }

Now that we have this defined we can implement the grammar building method, as before we just need to add these choices to the grammar and the engine will compile all of the various plugin grammars.

public Grammar getGrammar()
        {
            //Create a set of choices
            Choices thisChoices = new Choices(
                WeatherSpeechChoices.forecastTomorrow,
                WeatherSpeechChoices.forecastToday
                );
 
            //Create a grammar based on these choices
            Grammar thisGrammar = new Grammar(thisChoices.ToGrammarBuilder());
            // Set the Grammar name
            thisGrammar.Name = _grammarName;
 
            return thisGrammar;
        }

Halfway there, now we just need to implement the event handler. This is the fun part, where we call the Weather service. Lets implement this by having the event handler call a getForecastString method that takes the number of days out as an integer parameter. Once we know we will build that we can write our event handler:

        public void handleSpeechInput(string input)
        {
            switch (input)
            {
                case WeatherSpeechChoices.forecastToday:
                    string forecast = this.getForecastString(0);
                    Talker.Say(forecast);
                    break;
 
                case WeatherSpeechChoices.forecastTomorrow:
                    string forecast1 = this.getForecastString(1);
                    Talker.Say(forecast1);
                    break;
            }
        }

Now to implement the getForecastString method. Take a look at the API and you will see that we need to be issuing requests to http://weather.yahooapis.com/forecastrss. This URL takes two types of parameters, US zipcodes OR Yahoo location IDs. In order to get the Yahoo location ID, their page states that:

The location parameter can be a US Zip code or a location ID. To find your location ID, browse or search for your city from the Yahoo! Weather home page. The weather ID is in the URL for the forecast page for that city. You can also get the location ID by entering your zip code on the home page. For example, if you search for Los Angeles on the Yahoo! Weather home page, the forecast page for that city is http://weather.yahoo.com/forecast/USCA0638.html. The location ID is USCA0638.

However in this tutorial we will be using a zipcode (I live in the US). You may additionally change the units (celsius or farenheit) by appending a “u=c” or “u=f” to the URL. For this example lets, add two private members to our WeatherPlugin class that will help us create the URL:

private string _zipCode = "55407";
private const string _serviceURL = "http://xml.weather.yahoo.com/forecastrss?p=";

Now that we know what the URL is, we need to know how to parse the response. This service will respond with an XML document that is RSS 2.0 compliant. We will make use the XPathNavigator .NET class to parse the response. So lets see what our method looks like:

        public string getForecastString(int daysFromNow)
        {
             // Create a new XmlDocument  
            XPathDocument doc = new XPathDocument(_serviceURL + _zipCode);
 
            // Create navigator  
            XPathNavigator navigator = doc.CreateNavigator();
 
            // Set up namespace manager for XPath  
            XmlNamespaceManager ns = new XmlNamespaceManager(navigator.NameTable);
            ns.AddNamespace("yweather", "http://xml.weather.yahoo.com/ns/rss/1.0");
 
            //Get the forecast with XPath
            XPathNodeIterator nodes = navigator.Select("//rss/channel/item/yweather:forecast", ns);
 
            string day = "";
            string text = "";
            string low = "";
            string high = "";
 
            nodes.MoveNext();
            XPathNavigator node = nodes.Current;
 
            if (daysFromNow == 1)
            {
                nodes.MoveNext();
                node = nodes.Current;
            }
 
            day = node.GetAttribute("day", ns.DefaultNamespace);
            text = node.GetAttribute("text", ns.DefaultNamespace);
            low = node.GetAttribute("low", ns.DefaultNamespace);
            high = node.GetAttribute("high", ns.DefaultNamespace);
 
            string forecast = "It will be " + text + " with a high of " + high + ", and a low of " + low;
            return forecast;
        }

Now lets break this method down a little further. In order to parse the document correctly we need to load the yweather namespace (http://xml.weather.yahoo.com/ns/rss/1.0). We select our first node for parsing by jumping to the first yweather:forecast item. This particular service only returns a 1 day forecast, so we know that if the daysFromNow variable is either 1 or 0 (you should implement something to enforce this). Once we have retrieved the correct node we get the attributes as described by the API, construct an English sounding sentence out of them, and return that to our Engine class. Note that if we are worried about lag or connection overhead we could cache the navigator object and use that for subsequent calls. In this example though, we hit the webpage every time you ask for the weather.

This service returns much more than the temperature in its response including windchill, speed and direction, atmospheric conditions, sunrise, sunset and others. It also returns the current conditions in the yweather:condition node. For an example rss feed, here is a link to the output for Minneapolis. As you can see its pretty easy to extend the plugin and get it to tell you this other information if you want. Until next time, have at it and let me know what you come up with.

Could your shower curtain be harming your health?

An environmental group claims in a new study of vinyl shower curtains that some of them may release into the air toxic chemicals which could cause asthma, eye irritation or even cancer.

“We have a clear-cut case that these products release elevated levels of harmful chemicals,” said report co-author Mike Schade, PVC campaign coordinator for the Center for Health, Environment and Justice. “Our research builds on a growing body of evidence that shows that the government fails to protect us from the growing number of toxic chemicals in products.”

But some health experts are paying scant attention to those behind the curtain study. And perhaps with good reason.

Skeptics pointed out what they call a glaring error in the study’s methodology. The group tested a total of five shower curtains, of which only one shower curtain — not one brand; one curtain — was subjected to complete testing for chemicals in its composition, as well as those it released into the air — a phenomenon known as “off-gassing.”

Further testing was not performed “to avoid potential instrument damage,” according to the report.

The study found the one curtain which was tested for off-gassing may have released, over the course of the first few hours after it was opened, chemicals that could be toxic if swallowed or inhaled only in quantities thousands of times greater than those found.

Critics said that the testing was not verified by an independent lab and didn’t account for real-world conditions such as temperature or humidity in a shower stall.

The study also included estimated results for some of the chemical amounts it reported, and most of the off-gassed chemicals found initially were not detectable after a few days.

In short, it is a piece of shower curtain research that some experts said just doesn’t hold water.

They tested FIVE CURTAINS? Thats it?! Not only that but only one was fully tested and it emmitted for a short times toxic gasses that are 1/1000 of the toxic dose? While the rest of the article continues to backtrack and talk slightly more rationally, if you acknowledge these facts in your article, then how in the world can you lead with “Could your shower curtain be harming your health?”?! Furthermore, lets not understate the case, the last line should read “it is a piece of shower curtain research that any and all scientists that have the right to use the title said just doesn’t hold water.”

Jesus, I need to just watch the news for 30 seconds to remind myself why I don’t make a habit of it. I take solace in hoping that everyone involved in this scare mongering really believes what they said and is trying to figure out how to get by without a plastic shower curtain.

Lets start by opening the project as we left it in Microsoft Speech II: Getting your computer to listen. At this point we should refactor our namespace to be something more useful than “Tutorial1″, so lets change that to “Speech_Project” for now (make sure to update the XAML as well).

Next we need to add some folders for building out our framework. Add two folders named “Core” and “Plugins” to your project:

Now lets move the Say method out of the Window1 class and into its own class called Talker. We can use this later when we add more options to the output (SMS, IM, Email, etc.). So create a Talker class in the Speech_Project.Core namespace and move the say method into it:

using System.Speech.Synthesis;
 
namespace Speech_Project.Core
{
    public class Talker
    {
        public static void Say(string input)
        {
            var synth = new SpeechSynthesizer();
            var sayThis = new Prompt(input);
            synth.Speak(sayThis);
        }
    }
}

Next we need to define the interface to which all of our plugins will conform. Plugins for now will need to tell the engine which grammars they support and then have an event handler for when their grammar is triggered (see Tutorial 2). Finally, they’ll need to tell the engine which grammars are theirs by providing a method for getting the grammar name. Lets create an ISpeechPlugin interface that looks like the following:

using System.Speech.Recognition;
 
namespace Speech_Project.Core
{
    public interface ISpeechPlugin
    {
        Grammar getGrammar();
        void handleSpeechInput(string input);
        string getGrammarName();
    }
}

In addition to the plugin interface we should add a plugin attribute class. This will allow us to get some meta data about the plugins when we load them, look for version collisions, and other useful things down the road. So lets add a SpeechPluginAttribute:

using System;
 
namespace Speech_Project.Core
{
    [AttributeUsage(AttributeTargets.Class)]
    class SpeechPluginAttribute : Attribute
    {
        public SpeechPluginAttribute(string name, string description, string version)
        {
            _description = description;
            _name = name;
            _version = version;
        }
 
        private string _description;
        private string _name;
        private string _version;
 
        public string Description
        {
            get { return _description; }
            set { _description = value; }
        }
 
        public string Name
        {
            get { return _name; }
            set { _name = value; }
        }
 
        public string Version
        {
            get { return _version; }
            set { _version = value; }
        }
    }
}

Now that we have our Interface defined, lets create our first plugin. Start by creating a new Project in your solution for your Personality plugin, of the type class library. Change its Build properties to output to the Plugins directory of the main Project (to get this working in VS debug mode make sure it is /bin/debug/Plugins/). Next make sure you reference the main project and set the Main class to be an instance of the ISpeechPlugin Module (and make sure you add our custom attribute).

Now we can take the code that was in the Window1 class and move it into this Plugin. So lets take the grammar construction code and the event handler and put it in the Personality class like so:

using Speech_Project.Core;
using System.Speech.Recognition;
 
namespace Speech_Project.Plugins
{
    [SpeechPluginAttribute("Personality Plugin", 
        "This plugin handles all of the personality like responses for the plugin", 
        "1.0")]
    public class PersonalityPlugin : ISpeechPlugin
    {
        private string _grammarName = "PersonalityPlugin";
        #region ISpeechPlugin Members
 
        public Grammar getGrammar()
        {
            //Create a set of choices
            Choices thisChoices = new Choices("Computer",
                "What is your name?",
                "What is your favorite blog?"
                );
 
            //Create a grammar based on these choices
            Grammar thisGrammar = new Grammar(thisChoices.ToGrammarBuilder());
            // Set the Grammar name
            thisGrammar.Name = _grammarName;
 
            return thisGrammar;
        }
 
        public void handleSpeechInput(string input)
        {
            switch (input)
            {
                case "Computer":
                    Talker.Say("Yes master?");
                    break;
                case "What is your name?":
                    Talker.Say("My name is Anna");
                    break;
                case "What is your favorite blog?":
                    Talker.Say("Qurbit of course");
                    break;
            }
        }
 
        public string getGrammarName()
        {
            return _grammarName;
        }
 
        #endregion
    }
}

Now that we’ve done a little refactoring (yes you should do more: abstract out the strings, etc.), lets get to the fun part–dynamic binding. Lets start by creating an Engine class that will do the heavy lifting. The engine class will look in the plugins folder for any .dll files and if they implement the ISpeechPlugin will instantiate these into a List via Reflection. After loading all of the plugins, we can get our engine to load all of their grammars. Then when the engine recognizes any of the Choices in the loaded grammars we can trigger the event handler for the correct ISpeechPlugin.

using System.Collections;
using System.IO;
using System;
using System.Collections.Generic;
using System.Reflection;
using System.Speech.Recognition;
 
namespace Speech_Project.Core
{
    class Engine
    {
        public static List<ISpeechPlugin> _plugins;
 
        public static void LoadPlugins()
        {
            //Get all the assemblies in the Plugins folder
            List<Assembly> assemblies = LoadAssemblies();
 
            //Load the plugins 
            _plugins = GetPlugIns(assemblies);
 
        }
 
        private static List<Assembly> LoadAssemblies()
        {
            //Look in the plugin dir for .dlls
            DirectoryInfo dInfo = new DirectoryInfo(Path.Combine(Environment.CurrentDirectory, "Plugins"));
            FileInfo[] files = dInfo.GetFiles("*.dll");
            List<Assembly> plugInAssemblyList = new List<Assembly>();
 
            if (null != files)
            {
                foreach (FileInfo file in files)
                {
                    plugInAssemblyList.Add(Assembly.LoadFile(file.FullName));
                }
            }
 
            return plugInAssemblyList;
        }
 
        private static List<ISpeechPlugin> GetPlugIns(List<Assembly> assemblies)
        {
            List<Type> availableTypes = new List<Type>();
 
            foreach (Assembly currentAssembly in assemblies)
                availableTypes.AddRange(currentAssembly.GetTypes());
 
            // get a list of objects that implement the ICalculator interface AND 
            // have the CalculationPlugInAttribute
            List<Type> pluginList = availableTypes.FindAll(delegate(Type t)
            {
                List<Type> interfaceTypes = new List<Type>(t.GetInterfaces());
                object[] arr = t.GetCustomAttributes(typeof(SpeechPluginAttribute), true);
                return !(arr == null || arr.Length == 0) && interfaceTypes.Contains(typeof(ISpeechPlugin));
            });
 
            // convert the list of Objects to an instantiated list of ICalculators
            return pluginList.ConvertAll<ISpeechPlugin>(delegate(Type t) { return Activator.CreateInstance(t) as ISpeechPlugin; });
        }
    }
}

At this point, we have an infrastructure that allows us to abstract out the functionality in a given plugin, and so long as there is a .dll in the plugins folder that implements the ISpeechModule interface we can get an instance of it. Now all we need to do is get the engine to route the events to the correct plugins when it recognizes input. So lets add a SpeechRecognized Event handler to the engine that will look through the plugins to see if any have a handler for the recognized grammar name and if so then call the handler in that plugin

        public static void SpeechRecognized(object sender, SpeechRecognizedEventArgs e)
        {
            foreach (ISpeechPlugin plugin in _plugins)
            {
                //This is the correct handler
                if (e.Result.Grammar.Name == plugin.getGrammarName())
                {
                    plugin.handleSpeechInput(e.Result.Text);
                }
            }
        }

Finally, lets add an InitializeSRE method to the engine that will create the SpeechRecognitionEngine object and load the grammars from the plugins as well as attach the SpeechRecognized handler:

        public static SpeechRecognitionEngine InitializeSRE()
        {
            //Create the speech recognition engine
            SpeechRecognitionEngine sre = new SpeechRecognitionEngine();
 
            //Set the audio device to the OS default
            sre.SetInputToDefaultAudioDevice();
 
            //Reset the Grammars
            sre.UnloadAllGrammars();
 
            //Load the plugins
            LoadPlugins();
 
            //Load all of the grammars
            foreach (ISpeechPlugin plugin in _plugins)
            {
                sre.LoadGrammar(plugin.getGrammar());
            }
 
            //Set the recognition mode
            sre.RecognizeAsync(RecognizeMode.Multiple);
 
            //Add an event Handler
            sre.SpeechRecognized +=
                new EventHandler<SpeechRecognizedEventArgs>(Engine.SpeechRecognized);
 
            return sre;
        }

Now we can just call Engine.InitializeSRE in our window constructor and it will load all the plugins and handle all of the event routing. Build your plugin and run the WPF application and give it a try. Pretty simple huh? Now we can just keep building plugins to extend the functionality of our Speech Project. Stay tuned for more plugins.

The central tenet of this paper is that rationality in human decision making (Homo Economicus), isn’t a realistic assumption, despite the niceties it provides for model creators. It Notes that historically there have been 3 school of thought on human decision making:

1. Classical thinkers such as Pierre, Laplace or Piaget asserted that humans make decisions based solely on the use of probability ans statistics. In fast “the Enlightenment probabilists derived the laws of probability from what they believed to be the laws of human reasoning”. Despite much of science moving on past this, Economics and Psychology have held on to this view for the most part
2. In the mid-late 20th century, the “heuristics and biases” movement, pioneered by the likes of Tversky and Kahneman concluded that “human inference is systematically biased and error prone.” Despite the seeming repudiation of the 1st school of thought, this school holds on to probability and statistics as the normative way to make decisions, we humans are just in actuality broken or not up to the task.
3. The authors subscribe to Herbert Simon’s notion of bounded rationality, characterized by both cognitive and ecological bounds. The cognitive bound is simply that humans do not have the mental tools to quickly do the complex calculations necessary to perform multiple regression. Secondly, the ecological bound states the the human mind has adapted to live in real world environments.

The authors have taken these Simon’s two assertions and go on to propose a number of satisficing algorithms, that:

“operate with simple psychological principles that satisfy the constraints of limited time, knowledge, and computational might, rather than those of classical rationality”

For the experiment to test their algorithms, they take on a two alternative choice task where the subject is forced to make an inference based on knowledge that the user already has. Forcing them to retrieve this from memory presumable simulates the ecological bound. The theories that they propose build on Gigerenzer’s previous work on Probabilistic Mental Models (PMM). The theory integrates 3 central ideas:

1. “inductive inference needs to be studied with respect to natural environments” (Brunswik & Simon)
2. “inductive inference is carried out by satisficing algorithms” (Simon)
3. “inductive inferences are based on frequencies of events in a reference class” (Reichenbach)

Ultimately, the algorithms that conform to the PMM tenets don’t perform exhaustive memory search nor do they perform complex interactions between data points. Gained by not performing these expensive operations is very fast decision making. The authors assert that this is very valuable when being chased by a tiger, or say deciding whether or not to exit the freeway. Always getting off at the correct exit is less important than always deciding quickly enough that you don’t crash. Not taking into account all information conveniently allows for easily modeling of incomplete information instances, which are presumably the vast majority of real world decision making cases.

In their experiment they assess the capacity of a subject to guess whether city a has a greater population than that of city b. They do this for a set of cities known, this set is the reference class. The subject has certain data about each city, such as whether it has a soccer team or a university. These data points are known as cues. They model limited knowledge (incomplete information) as both an imperfect knowledge of all objects in the reference class and limited knowledge of the cue values (facts about the cities).

The authors finally propose a very simple satisficing algorithm called the “Take the Best Algorithm“. This algorithm assumes a subjective rank order of cues according to their validities. It has 5 steps:

1. Recognition Principle: This is invoked when recognizing an object is a predictor of the target variable. In the case of population, there is an obvious correlation. Thus, if we are predicting highest population size and one of the two cities is know, pick it. If neither are known, randomize. If both are known go to step 2.
2. Search For Cue Values
3. Discrimination Rule: Decide if the cue discriminates.
4. Cue-Substitution Principle: If so use it to pick, otherwise pick another cue (step 2)
5. Maximizing Rule For Choice: Choose the value the cue would predict. If they are the same randomize.

This algorithm searches only a portion of the total knowledge, that is it reads only a subset of the cues for either object. It also does not attempt to integrate information, but rather just substitute in the best cue.

The authors then assert that experimental evidence has shown that the less is more effect can be shown to exist empirically when one uses this algorithm. This may explain why US students make slightly more correct inferences about German cities and vice versa.

Note that one may not know a cue value for all objects in the reference class. Thus, the authors define two properties that they use to evaluate cues. Ecological Validity is the proportion of cases in the reference class in which this cue is an accurate predictor (not a bad prediction). Discrimination rates are the relative frequency with which the cue can be used to discriminate (not the same value).

They then present data comparing the Take the Best (TTB) algorithm wit h the following other algorithms:

1. Tallying
2. WeightedTallying
3. Unit-Weight Linear Model
4. Weighter Linear Model
5. Multiple Regression

Ultimately they conclude what you would predict. The TTB algorithm is the fastest due to the small number of comparisons needed. They also assert that it does just as well if not better than all of the other algorithms. While their data does seem to bear this out with 84 German cities and 6 cues, even with perturbation of the amount of known data (they randomize over which objects are recognized and which cue values are known), their methodology for cue selection isn’t explained in much detail. I will be curious to see if there is follow up work that tries to see if these results were come by due to a fortuitous selection of cues and objects int he reference class.

At the end the suggest two simpler algorithms, the “Take the Last” algorithm and the “Minimalist” algorithm that require even fewer cue comparisons (calculations) and do only marginally worse then the TTB algorithm.

They conclude the paper by addressing “One Reason Decision Making”, that is the decision is made based on one good reason and there is no compensation between cues. This is the case often when using the TTB algorithm. They note that the recognition principle is an example of one reason decision making that exploits lack of knowledge. They suggest as support for why one reason decision making may be plausible and good is that much evidence supports the notion that humans are ill equipped to deal with correlation between cues.

In summary the authors suggest that despite the cleanliness of many of the normative models that are so prevalent in much of academia, cognitive models based on normative assumptions are unrealistic and possibly unnecessary. The brain is constrained in the complexity of calculations it can perform, the speed with which it can perform them, and the data upon which it can draw to make those calculations. Thus, any realistic cognitive algorithm would take that into account. They propose that the TTB algorithm is a reasonable attempt at an algorithm that meets all of those criteria.

For the most part I intend to study the Behavioral Economics literature and keep my notes here. Given that I will also be writing about technology and the person who cares about both of these two topics is rare, I suggest to the would be reader that they subscribe to one of my topic feeds (programming, econ, etc) to help filter the noise.