Microsoft Bot Framework with UWP

Microsoft Bot Framework with UWP

botframe1

Short introduction

We are living in revolutionary time and I think that you can agree with me. Self-driving cars, space travels and bots. Yes you see clearly – bots. Microsoft presented this year that we are able to create intelligent bots which can help as with everyday activity. Check flight, translate word or locate shop. All these you can do with Microsoft Bot Framework. In this article I would like to show how you can create your first bot and communicate with it via Universal Windows Platform application.

What do I need to start?

1) Visual Studio 2015 Community (for free) or higher

2) Microsoft account

3) Be registered in Bot Dev Portal

4) Bot Emulator

5) Microsoft Cognitive Services account

 

Let’s start

I will try to show you step by step how to create bot.

1) Download and install bot application project template for Visual Studio:

a. Under this link you can download project template

b. Once you do it, please copy it to Visual Studio 2015 templates directory:

botframe2

c. Once you open Visual Studio, please select “File” -> “New project” and try to find below template:

botframe3

Type the project name – in my case this is: “AssistantBot”.

 

Bot project structure

botframe4

As you can see Bot project structure is similar to Web Api project – and that’s how it works!

If you open “MessagesController” from “Controllers” folder you can see class with whole description.

This controller is responsible for retrieving message from user, analyzing it and returning response.

    //Access to your bot can be secured:
   [BotAuthentication]
   public class MessagesController : ApiController
    {
      /// <summary>
      /// POST: api/Messages
      /// Receive a message from a user and reply to it
      /// </summary>
    public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
    {
      if (activity.Type == ActivityTypes.Message)
      {
        ConnectorClient connector = new ConnectorClient(new Uri(activity.ServiceUrl));
        // calculate something for us to return
        int length = (activity.Text ?? string.Empty).Length;

       // return our reply to the user
       Activity reply = activity.CreateReply($"You sent {activity.Text} which was {length} characters");
       await connector.Conversations.ReplyToActivityAsync(reply);
     }
    else
     {
       HandleSystemMessage(activity);
     }
    var response = Request.CreateResponse(HttpStatusCode.OK);
    return response;
  }

   private Activity HandleSystemMessage(Activity message)
    {
      if (message.Type == ActivityTypes.DeleteUserData)
       {
        // Implement user deletion here
        // If we handle user deletion, return a real message
       }
      else if (message.Type == ActivityTypes.ConversationUpdate)
       {
        // Handle conversation state changes, like members being added and removed
        // Use Activity.MembersAdded and Activity.MembersRemoved and Activity.Action for info
        // Not available in all channels
       }
     else if (message.Type == ActivityTypes.ContactRelationUpdate)
     {
       // Handle add/remove from contact lists
       // Activity.From + Activity.Action represent what happened
     }
    else if (message.Type == ActivityTypes.Typing)
     {
      // Handle knowing tha the user is typing
     }
    else if (message.Type == ActivityTypes.Ping)
     {
     }

    return null;
  }
 }

Test bot locally

You can test your first bot locally. To do it we are going to use Bot Emulator you downloaded earlier.

Launch project in local browser (the way is exactly the same like for Web Api):

botframe5

Once browser is opened you should see notification that you bot is running:

botframe6

Now find installed Bot Emulator in Start menu and launch it:

a. Local port and emulator Uri will be filled automatically

b. Type Bot url – this is the address from started browser (localhost and port). Remember to add “/api/messages” at the end because this is endpoint to communicate with your bot:

botframe7_7

You can talk with your bot – to do it write some sample message in text box:

botframe8

 

Add some more intelligence to your bot

As you can see your bot returns number of characters from the sentence you sent. What about some more advanced conversation? Let’s add more intelligence to our bot.

Let’s create bot which can talk with us and return some more sophisticated answers!

To achieve that we should use “Dialogs”. Dialog is a conversational process between user and bot which can keep state about the conversation. Developers can create their own implementations of Dialogs. Below we will create one.

Each “Dialog” is based on the “IDialog” interface with “StartAsync” method.

1. Add new class to the project called “AssistantBotDialog “:

 

 [Serializable]
 public class AssistantBotDialog : IDialog<object>
 {
    public async Task StartAsync(IDialogContext context)
    {
      context.Wait(MessageReceivedAsync);
    }
    public async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> argument)
    {
     var message = await argument;
     await context.PostAsync(message.From.Name + " you said: " + message.Text);
     context.Wait(MessageReceivedAsync);
    }
 }

This implementation is responsible for handling new messages from users and also for saving state, so our bot knows exactly with which user it is talking to.

Now when you launch bot you can talk with it via emulator – nothing special at this point:

botframe9

Now when our bot knows how to recognize users we can add some functionality connected with spell checking. Yes! To do this we need Microsoft Cognitive Services: Bing Spell Check API.

Create free Cognitive Services account and add Bing Spell Check API

1. Once you register, select “Bing Spell Check Api”:

botframe10

2. Copy your API key – it will be required below

 

Integrate Bot with Cognitive Services

Now we have to add functionality responsible for checking spelling of text that we sent to our bot.

1. Add new folder called “CognitiveServices” and class inside it called “CognitiveClient”. It should look like below:

 [Serializable]
 public class CognitiveClient
  {
    [NonSerialized]
    HttpClient _httpClient;

    public CognitiveClient()
     {
       _httpClient = new HttpClient();
       // Request headers:
       var apiToken = ConfigurationManager.AppSettings["CognitiveServicesApiToken"];
      _httpClient.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", apiToken);
     }
   //Send sentence to Cognitive Services and get response:
   public async Task<CheckSpellingResponse> CheckSpelling(string sentence)
   {
     var queryString = HttpUtility.ParseQueryString(string.Empty);
     queryString["text"] = sentence;
     queryString["mode"] = "Spell";
     var uri = "https://api.cognitive.microsoft.com/bing/v5.0/spellcheck/?" + queryString;
     var response = await _httpClient.GetStringAsync(uri);
     var checkSpellingResponse = JsonConvert.DeserializeObject<CheckSpellingResponse>(response);
     return checkSpellingResponse;
   }
 }

2. In the same folder add “CheckSpellingResponse” class which represents spelling response from Cognitive Services:

 public class CheckSpellingResponse
  {
    public string _type { get; set; }
    public List<FlaggedToken> flaggedTokens { get; set; }
  }

 public class Suggestion
  {
    public string suggestion { get; set; }
    public int score { get; set; }
  }

 public class FlaggedToken
 {
   public int offset { get; set; }
   public string token { get; set; }
   public string type { get; set; }
   public List<Suggestion> suggestions { get; set; }
  }

botframe11

 

3. Remember to add you Api key to “web.config” file:

botframe12

We are using it to connect to the Cognitive Services API.

4. Now we need to integrate Spelling check with our bot. Open “AssistantBotDialog” class. Below you can find code with Cognitive Services integration. Now it should look like this:

 

  [Serializable]
  public class AssistantBotDialog : IDialog<object>
   {
     CognitiveClient _cognitiveClient;
    public AssistantBotDialog()
    {
      _cognitiveClient = new CognitiveClient();
    }
   public async Task StartAsync(IDialogContext context)
   {
     context.Wait(MessageReceivedAsync);
   }
  public async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> argument)
   {
     var message = await argument;
     //Pass sentence to Spelling API and get reponse:
     var checkSpellingResponse = await _cognitiveClient.CheckSpelling(message.Text);
     StringBuilder stringBuilder = new StringBuilder();
     var standardReponse = message.From.Name + ", I checked your sentence, ";
     if (checkSpellingResponse.flaggedTokens.Count != 0)
       standardReponse = standardReponse + "here is correct spelling:" + stringBuilder.ToString();
     else
       standardReponse = standardReponse + "it is correct!";

   await context.PostAsync(standardReponse);
   checkSpellingResponse.flaggedTokens.ForEach(flag => flag.suggestions.ForEach(suggestion => stringBuilder.AppendLine(suggestion.suggestion)));
   await context.PostAsync(message.From.Name + ", I checked your sentence, here is correct spelling: " + stringBuilder.ToString());
   context.Wait(MessageReceivedAsync);
  }
 }

5. Now we can test our bot – launch application and emulator:

botframe13

botframe14

Remember that with free subscription you can only send 7 requests per 1 minute!

 

Publish your bot on Microsoft Azure

Once we tested our bot locally it is time to publish it – to make it available for others. To do it you need to have active Azure subscription. For this article you can use trial version where you have 200$ to use.

1. Sign in to the portal

2. Now in Visual Studio right click on the project and select “Publish”:

botframe15

3. Select “Microsoft Azure App Service”:

If you are not signed in to Azure, you should do it before this step (dialog should appear):

botframe16

4. Click “New” button on the right:

botframe17

5. This is very important step. You should type the url under which your bot will be available for public use:

“Web App Name” should be unique. You should also specify resource group and app service plan.

botframe18

6. Once you fill everything, click “Create” button. If your web app was created, you should see below dialog. Click “Validate connection” to check connection with Azure Web App:

botframe19

7. Now we are ready to publish our bot! Click “Publish” button. After few seconds you should see opened browser with url to your bot on Azure:

botframe20

You should also see your created web app on Azure portal:

botframe21

This is not the end. To make sure that our bot is working properly, we need to register it in Bot Dev Portal.

 

Register bot in Bot Dev Portal

1. Go to the Bot portal and sign in with your Microsoft Account.

2. Select “Register a bot”:

botframe22

3. You should fill the information about your bot. You can add icon. Also fill all required fields below:

Note that “bot handle” is name of your bot from Azure Web App url and “messaging endpoint” is your Azure Web App url + “/api/messages” to communicate with your bot:

botframe23

4. You should also register your bot in Microsoft Apps portal. To do it create “Create Microsoft App ID and password”:

botframe24

This will automatically redirect you to apps portal.

“App id will be generated automatically” Click “Generate app password to continue”:

botframe25

You should see the dialog with password. Copy it somewhere for now because it will not be visible anymore.

Also copy App ID – we will use it further:

botframe26

Now click “Finish and go back to Bot Framework”:

botframe27

Remember also to check “http” to “https” for messaging endpoint:

botframe29

5. Once you get back to bot registration, accept term and click “Register” at the bottom:

botframe28

6. If you configured everything correctly, you should see confirmation that your bot is registered:

botframe30

 

Fill settings in Visual Studio and republish your bot

Once you registered bot, you should update “web.config” file with three things:

a. Bot ID – “bot handle” value from Bot Portal

b. App ID – ID you have copied earlier from Microsoft Apps Portal

c. Password – Password you have copied earlier from Microsoft Apps Portal

botframe31

Now once again click “Publish” on the project and republish your bot.

Now it is available worldwide!

 

Test your bot and talk to it

Now when your bot is available public you can try to talk to it. The easiest way is to add it to Skype. It is so simple!

Click “Add to Skype” in the Bot Portal:

botframe32

If you already have Skype account it will ask you to open app and your bot will be automatically added to contact list:

botframe33

botframe34

If you would like to publish your bot in Bot Directory – you should click “Publish” button add wait for the review and acceptance of your bot:

botframe35

botframe36

Now you can talk with your bot. Great! It is time to connect it from Universal Windows 10 App.

 

Talk with bot with Universal Windows 10 App

To make easier communication with our bot we will use REST. To enable it for communication in the Bot Portal select your bot and at the bottom click “Add” next to “Direct Line”. Direct Line is a simple REST Api to communicate with your bot from web and mobile applications:

botframe37

1. Click “Add new site” and paste address (without https prefix). In my case it was: assistantbot10.azurewebsites.net:

botframe38

botframe40

Copy first secret key – we will use it further.

2. Click bottom button “I’m done with configuring Direct Line”.

3. Now Direct Line should appear on the channels list:

botframe39

Now we are ready to create UWP application and connect to our bot.

 

Create new UWP app and configure it to talk with the bot

1. In Visual Studio add new Universal Windows App project. I called it “AssistantBotClient”:

botframe41

2. Now we need to add NuGet called “Microsoft.Bot.Connector.DirectLine”:

botframe42

IMPORTANT

If you have problems with installation probably you should update “.NETCore.UniversalWindowsPlatform” NuGet.

It is time to create some simple UI to display messages exchanged with our bot:

1. Open “MainPage.xaml” and paste below code. This is simple UI with Textbox to insert your message and list (with item template for message)  where responses from bot are displayed:

<Page
 x:Class="AssistantBotClient.MainPage"
 xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
 xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
 xmlns:local="using:AssistantBotClient"
 xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
 xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
 mc:Ignorable="d">
  <Page.Resources>
   <DataTemplate x:Key="MessagesListDataTemplate">
    <Grid Background="#FF1B9BD6" Height="60">
     <TextBlock Text="{Binding Text}" TextWrapping="Wrap" VerticalAlignment="Center" Margin="10, 0, 10, 0"/>
    </Grid>
   </DataTemplate>
 < /Page.Resources>

 <Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
   <Grid.RowDefinitions>
     <RowDefinition Height="89*"/>
     <RowDefinition Height="380*"/>
     <RowDefinition Height="87*"/>
     <RowDefinition Height="84*"/>
     </Grid.RowDefinitions>
     <TextBlock Text="Assistant Bot" Grid.Row="0" HorizontalAlignment="Center" VerticalAlignment="Center" FontWeight="Bold" Foreground="#FF1D59B0" FontSize="20"/>
     <ListView x:Name="MessagesList" Grid.Row="1" ItemTemplate="{StaticResource MessagesListDataTemplate}"/>
     <TextBox x:Name="NewMessageTextBox" Grid.Row="2" KeyUp="NewMessageTextBox_KeyUp"/>
     <Button x:Name="button" Content="SEND" HorizontalAlignment="Center" VerticalAlignment="Center" Click="button_Click" Grid.Row="3"/>
  </Grid>
</Page>

2. Now open “MainPage.xaml.cs” and paste below code. I also included comments so you can understand what’s happening:

using Microsoft.Bot.Connector.DirectLine;
using Microsoft.Bot.Connector.DirectLine.Models;
using Microsoft.Rest;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.IO;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices.WindowsRuntime;
using System.Threading.Tasks;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Core;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;

// The Blank Page item template is documented at http://go.microsoft.com/fwlink/?LinkId=402352&clcid=0x409

namespace AssistantBotClient
{
  public sealed partial class MainPage : Page
   {
     DirectLineClient _client;
     Conversation _conversation;
     ObservableCollection<Message> _messagesFromBot;
     public MainPage()
      {
        InitializeComponent();
        //Set binding context to update message list items:
        DataContext = this;
      }

     protected async override void OnNavigatedTo(NavigationEventArgs e)
      {
       base.OnNavigatedTo(e);
       //Create collection for bot messages:
       _messagesFromBot = new ObservableCollection<Message>();
      //Initialize conversation with bot:
      await InitializeBotConversation();
      }

  private async void button_Click(object sender, RoutedEventArgs e)
   {
     await sendMessageToBot();
   }

   //Handle button click when user wants to send message to bot:
   async Task sendMessageToBot()
   {
      //Message object with name of the user and text:
      Message userMessage = new Message
      {
       FromProperty = "Daniel",
       Text = NewMessageTextBox.Text
      };
     //Post message to your bot:
     if (_conversation != null)
       await _client.Conversations.PostMessageAsync(_conversation.ConversationId, userMessage);
   }

  async Task InitializeBotConversation()
   {
     //Initialize Direct Client with secret obtained in the Bot Portal:
     _client = new DirectLineClient("<<YOUR BOT SECRET>>");
     //Initialize new converstation:
     _conversation = await _client.Conversations.NewConversationAsync();
     //Wait for the responses from bot:
     await ReadBotMessagesAsync(_client, _conversation.ConversationId);
   }

    //This method is responsible for handling messages from bot:
    async Task ReadBotMessagesAsync(DirectLineClient client, string conversationId)
     {
      //You can optionally set watermark - this is last message id seen by bot
      //It is for paging:
      string watermark = null;

      while (true)
      {
       //Get all messages returned by bot:
       var messages = await client.Conversations.GetMessagesAsync(conversationId, watermark);
       watermark = messages?.Watermark;

      //get messages from your bot - FromProperty should match your Bot Handle:
      var messagesFromBotText = from x in messages.Messages
      where x.FromProperty == "AssistantBot10"
      select x;

    //Iterate through all messages:
    foreach (Message message in messagesFromBotText)
    {
      message.Text = "Daniel" + message.Text;
      await Windows.ApplicationModel.Core.CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
     () => {
            //Add message to the list and update ListView source to display response on the UI:
            if (!_messagesFromBot.Contains(message))
             _messagesFromBot.Add(message);
            MessagesList.ItemsSource = _messagesFromBot;
           });
    }
      await Task.Delay(TimeSpan.FromSeconds(2)).ConfigureAwait(false);
   }
  }
 
  private void NewMessageTextBox_KeyUp(object sender, KeyRoutedEventArgs e)
  {
    if (e.Key == Windows.System.VirtualKey.Enter)
      Windows.ApplicationModel.Core.CoreApplication.GetCurrentView().CoreWindow.IsInputEnabled = false;
  }
 }
}

Now launch the app. Type some sentence or word and click “SEND” button. You should see the effect like below:

botframe44

 

Sum up

Great! Now you know how to create your own Microsoft Bot, how to publish it and how to talk with it via Universal Windows 10 App.

If you would like to read more about bot framework and see more samples, please visit below pages:

Bot Framework GitHub

Bot Framework documentation

Below there are two links to my GitHub where you can find created samples:

Assistant Bot Project

Assistant Bot UWP Client Project

Now go and talk to your bot!

Advertisements