So, you think you know everything about Buddyscript?

Well, let’s see how much of such a statement is true. Try to answer this short quiz! If you score all of them it doesn’t mean you are an expert, but you could say you are getting there ;-)

1-    Let’s talk about matching. Say we want to match precisely on full URLs (and only full URLs) during a conversation, which one of the following cases would be the way to go:

a)    subpattern AFullUrl /^www`.`S+`.(com|fr|org|net)$/ {minlength=3, example=”www.microsoft.com”}

b)    subpattern AFullUrl + www =Anything (com|fr|org|net) {canskip=”no”, minlength=3}

c)    subpattern AFullUrl www.=Anything.(com|fr|org|net) {canskip=”no” minlength=3}

2-    Nl-bricks? Only one of the following affirmations is true:

a)  “NL- bricks” along with “regular subpatterns” are processed during the “patternization” process, but “NL-bricks” have precedence over “regular subpatterns”.

b)  “NL- bricks” are the minimum expression of a meaning unit, and are “patternized” into “regular subpatterns” at compile time.

c)  “NL-bricks” are “skippable”, as opposed to “regular subpatterns” that need to have the property {canskip=”yes”}

3-    Defining macros in Buddyscript, only one of the following statements is true:

a)    If the code contains any line breaks, it should be placed immediately below the first line of the statement. If you want your macro to contain an empty line, add a line containing only the character “\”. The first empty line defines the end of the macro.

b)    If you want your macro to contain an empty line, add a line containing the character “\” at the end of the line before. I will add an empty line right between this and the following line. The first empty line defines the end of the macro.

c)    Macro definitions can’t contain line breaks. The first empty line defines the end of the macro.

4-    Only one of the following statements is true:

a)    The “always match as” statement allows you to associate a variable name with a particular subpattern throughout a group of patterns only.

b)    The “always match as” statement allows you to associate a variable name with a particular subpattern throughout a group of patterns, a package, or an entire domain.

c)    The “always match as” statement allows you to associate a variable name with a particular subpattern in a routine, or in a multi-line of code.

5-    What do you know about composite subpatterns? Only one of the following is true!

a)    Composite sub-patterns let you assemble multiple static subpatterns together, so that a candidate string is filtered through the first subpattern, then the result of this filtering is passed to the second one, etc.

b)    The minimum number of compositions is one, but there’s not a limit on the maximum number of compositions you can add.

c)    The default values for the properties of a composite subpattern are those of the outer-most subpattern, since it's going to be evaluated last.

6-    Hum, have you ever tried to create datasources? Let’s see if you know which one of the following is true:

a)    The “postprocess-block“ of a datasource is a BuddyScript routine used to set the offset and the total count of rows when the “preprocess-block” didn't return any of those values, or if the value returned is not accurate.

b)    “Expire” is a common property to the different datasource kinds. This information allows the platform to remember (cache) the result for a certain time. By default, expire is set to “now” (information is not stored). If you want the information to be cached, set it to “never”, or when the information won't be valid anymore (in 5 minutes, in 2 days, etc.)

c)    Datasources are BuddyScript tools to access external data. The external data retrieved can be accessed through many methods, including gateway calls, SQL queries, SOAP methods, datatables, etc.

7-    How about rephrasing rules? We want to use the property “MACRO_BEST_ONLY” when:

a)    We need to make sure the rule it’s only applied when it matches best from all of the other rephrasing rules.

b)    We want the rule to be applied only when it matches best, and no other rule can be applied after.

c)    We want to avoid matching on rules scoring lower than 80.

Solutions: 1-a, 2-b, 3-a, 4-b, 5-b, 6-c, 7-a

 If you failed the quiz, and you want to know more about Buddyscript, besides this blog you can browse the Windows Live Agents forum in MSDN, and of course read the documentation shipped with the platform.

Audits are an important piece to the ongoing maintenance of your agent. The purpose of auditing is to take a look at how users are interacting with the agent, and identify ways to improve the experience. Any issues that you uncover in audits should be addressed in a regularly scheduled agent update window.

There are two types of audits—Query and Session.

Session audits answer the question “Was this session satisfactory to the user?“ and are best for evaluating the overall user experience. Did the user find the information he or she was looking for? Was the answer complete? Did the user express any frustration that was not appropriately addressed? As a client, do you feel that the interaction served your purpose in deploying the agent?

Query audits answer the question “Was the query answered by the correct topic?” and are ideal for fine-tuning the Natural Language comprehension and finding commonly unanswered queries.

Once your agent launches, it’s preferred that you audit every 2-3 days for the first two weeks and then once every 2 weeks for the remainder of the agent’s life. Of course, it is up to you to determine how often you audit your agent. If your agent has a ton of traffic, you might want to audit more often.

If you choose not to audit your agent, the agent will never improve and you will supply users with a poor experience. Auditing your agent will not only improve it, but improve your development style and guidelines for future agents.

Auditing Guidelines

Session Audit

Suggestions for how to score sessions in a session audit:

• Misunderstood: If the query is not recognized correctly and the user...
  • eventually gets the correct answer – Yes
  • would have gotten the right answer if he had continued another step (selected menu topic, typed more, etc) – Yes
  • doesn’t get the right answer  – No
• Content: If the query seemed to be understood reasonably well but the response...
  • clearly does not answer the question – Yes, but consider improving content
  • does not answer the question but includes a URL to a page that should logically answer the question – Yes, test the link, consider improving content if it does not
• Menu: If a query is answered with an appropriate menu...
  • but should have been answered directly to a topic on the menu – Yes, notify WLA Team of misunderstood query
  • but is beyond the scope of the agent (too specific, technical)  – Yes
  • but the user does not select any of the options – Yes
  • but the user does not select the correct option – Yes, consider  improving the menu if possible
• Long/multiple: If the query is long or contains multiple questions and...
  • hits “catch” or gives the correct answer  – Yes, bravo
  • user eventually gets the correct answer – Yes
  • user doesn’t eventually get the right answer – No
  • user gets the wrong answer and gives up – N/A
• Dialogs: If a user engages agent in a dialog it cannot handle (responds to an answer with “I already tried that” or similar – Ignore that query
• Ambiguity: If the user hit an ambiguity and...
  • the correct topic is listed, but the query was specific enough that it should have been answered directly – Yes, notify WLA Team
  • neither option is correct – No
  • at least one of the options is incorrect/irrelevant –  Yes, notify WLA Team
• Entry message/empty session: If session only contains entry messages – N/A
• Looping: If topics are looping – No
• Nonsensical: When the user types nonsense, mark the query or session as N/A.             

Query Audit

* All internal query audits should use the option to "Only include queries relevant to natural language."

Suggestions for how to score queries in a query audit:

• Non-NL Queries: Despite selecting the option* above, the auditor may still need to manually discard non-natural-language queries
  • Menu selections: a # (followed by line of NL-comprehension code in parentheses) – N/A
  • Any dialogue queries such as: more, yes, no, show related topics – N/A
  • “?” – N/A
• Nonsensical: When the user types nonsense or question unrelated to products covered – N/A
• Beyond the scope of the agent: When the user asks a very technical or specific question and
  • gets a menu or somewhat related answer – Yes
  • doesn’t get an approximate answer – N/A
• Mismatched: Any reasonable query that
  • does not match an appropriate response in the agent – No
  • is known not to have an appropriate response in the agent and
    • doesn’t match anything at all relevant – No
    • matches something somewhat relevant – still Yes, but consider improving content
  • mismatched because the user tried to engage in a dialogue such as “I already tried that” – N/A
• Menu: If a query matches to an appropriate menu but was specific enough to have matched directly to a topic– No
• Long/multiple: If the query is long or contains multiple questions and...
  • gives the correct answer  – Yes, bravo
  • matches reasonably well – Yes
  • hits “catch” – Yes
  • mismatches – N/A
• Unanswered/Catch section: If the user makes a valid query (not nonsense) that will probably be asked again at some point – No
• Ambiguity: test each query in this section
  • the correct topic is listed, but the query was specific enough that it should have matched directly – No
  • neither option was correct – No
  • at least one of the options is incorrect/irrelevant – No

Ideally you start creating a new Agent using the WLATemplate! Detailed information on how to instantiate a new project [currently available languages are: English, Chinese (simplified), French, German, Italian, Japanese, and Spanish] from the WLATemplate: http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!446.entry

As you can imagine, this is only the beginning... In order to make your Agent a success and stand out in the crowd, you have to customize it to whatever you want your Agent to be, simple things like giving your Agent a personality and making "him" or "her" more natural. Here are a bunch of former blog posts that talk about exactly that:

Creating a Personality Spec - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!317.entry

Customizing Chat - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!230.entry

Welcome Messages and ABGreetingProc - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!408.entry

Changing PSM, Friendly Name, and Icon - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!315.entry

Best Practices for Developing a Windows Live Agent:

Part 1 [Use the WLATemplate Project as a Starting Point] - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!149.entry

Part 2 [Create Good Subpatterns] - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!150.entry

Part 3 [Use Dialogs] - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!151.entry

Part 4 [Use Canonical Questions] - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!152.entry

Part 5 [Remember User Information] - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!156.entry

Part 6 [Create a Clear and Concise Welcome Message] - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!160.entry

Part 7 [Be Careful When Using Public Variables] - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!163.entry

Part 8 [Make Your Project Modular] - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!164.entry

Part 9 [Make Your Agent Chatty] - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!166.entry

Part 10 [Conclusion of Best Practices Series] - http://windowsliveagents.spaces.live.com/blog/cns!5BCD45E519E07634!176.entry

-----------------------------------------

Blog post contributor(s): Mirco