Xamarin Test Cloud – All you have to know: part3

Xamarin Test Cloud – All you have to know: part3


Short introduction

This is the last article about Test Cloud. If you read previous two you are ready to publish your first tests to Xamarin Test Cloud. We will use code from the previous article. It’s time to start and see how you can test your mobile applications on thousand of devices.

What do I need to start?

1) Xamarin Test Cloud account (you can use free tiral version)

2) Xamarin Test Recorder (either for Windows or MAC)


Let’s start

Before we start it is worth to mention that there are two ways to submit your application for tests:

1) Using IDE – you can directly submit your tests via Visual Studio or Xamarin Studio

2) Using command line


What are the requirements?

Xamarin Test Cloud requires IDE Xamarin Studio 5.9 or Xamarin for Visual Studio 3.11.

You need also to add your e-mail to the team and (I will explain further) and have Team Api Key.

Firewall traffic should be possible for below Test Cloud  addresses (with ports either 80 or 443):

Last thing for Android apps: INTERNET permission has to be enabled in the app manifest.


Briefly about Test Cloud structure

In Xamarin Test Cloud users and applications are grouped into Organizations. In one Organization can be many teams:


1) Organization – coordinate teams, subscriptions, users and applications. Each organization can have many users as an administrators who are responsible for creating teams.

2) Team – always team consists of users and at least one application which is used by these users. Each team has always API Key used to assosiate and application and users that are testing it ( with their accounts in Test Cloud).

Three type of users:

1) User – anyone registered with Xamarin Test Cloud. User can be assigned to one or more teams. User can also belong to many organizations.

2) Team Managers – users with “Manage Permissions” – it is possible for them to add or remove users from team. Each team has always at least one manager.

3) Administrator – this is the highest level in the hierarchy. Admins are responsible for organizations – deleting, adding and modifying teams is their role. Organization has always at least one Administrator. Admin has rights to:

– Adding or inviting new users
– Adding more administrators
– Creating more teams


Configure Organization

Each new Xamarin Test Cloud account has automatically one Team with one User created who is both – Team Manager and Administrator.


Explore Xamarin Test Cloud Portal

1) Sign in to Test Cloud

2) Click your account in the right top corner and select “Account settings”


As you can see there is one team created and you are included to it.

3) Please note two options:

– New test run

– Show API Key

These are crucial to enable testing through the Test Cloud.


4) If you click on “member” dropdown you will be able to add users to your team:


5) It is possible to rename team of course:



Submit your tests to Xamarin Test Cloud

Once you have your application ready for tests you can submit it to the Test Cloud via Visual Studio or Xamarin Studio.

IDE rebuilds the application and the tests and upload the binaries, placing them in a queue for execution on Test Cloud.

We will use sample from the previous article – “CreditCardValidator”.


– When submitting an Android project you have to set build configuration to “Release” and project as a startup:


– When submitting an iOS project you have to set build configuration to “Debug” and project as a startup:


1) Right click on the UITests in “Tests” pad and select “Run in Test Cloud”:


2) There are two possible options:

a) First one is connected with choosing app package for tests:

In this case we will use Android project – in the dialog find release .APK file in “Release” folder:


Click “Upload and Run”.

Here you have to know that NUnit NuGet package version 2.6.4 is required – all above are not yet ready.


b) Second one enables you to choose which project you would like to test directly:

Right click on “Test Apps” on the right and select “Add App Project”:


Now with right click on the Tests you can choose app project fo upload:


3) Dialog will appear and you should sign in with your account:



4) Now you can choose on which devices you would like to test your app:



You can also check specification of each model – that is great because you getting familiar with different devices:



5) Once you select devices you have to choose series of the test (in our case it is master) and system language:



6) You can see that the app is being processed and tested:



7) Once tests are comopleted you should be able to see full report:


There is information about number of tests (with result also) and devices on which these tests were launched.

During tests it is possible to take screenshots:


There are also: Device Logs and Test Failures so you can see exactly what happened and why test failed.

In this case I used test cases for iOS so it is obvious that they not passed (there is no UINavigationBar control):



Sum up

This was the last article for the Xamarin Test Cloud series. I hope that I helped you understand possible options for testing for Xamarin Platform. Now you know what are the frameworks, how to use REPL console, how to write tests and deploy them to Test Cloud. Of course I could not include everything (different cases) but I hope that this series will encourage you to broad your knowledge about UI Tests.