Throughout a testers life, we would have asked ourselves the question  ‘To automate or not to automate’.

Automation means taking your manual tests and automating them using a tool and language of your choice.  As a tester, I have also looked into automated testing and to make things clearer for us, I have drawn up a list of points as to why I feel projects will benefit from automated testing:

Save Time – This is one of the biggest benefits for me, especially when it comes to regression testing. As we all know, regression testing is the retesting of the application when new features have been introduced or a change is made to an existing feature that has been previously tested (the change can be as a result of change request, defect fix, refactoring). The aim of regression testing is to ensure the application still works as expected and in order for us to verify this, we need to run all test scripts associated with the change. There is a risk here, that due to time constraints we may not run all tests associated to the change which may result in undiscovered defects.

These issues can be overcome by setting up our automated tests to run over night or after each deployment – this creates time for the tester to perform exploratory testing, concentrate on areas which cannot be automated and concentrate on other tasks.  

Speed –As automated tests are run by tools, these are run much faster than human users which adds to the first benefit of saving time.

Repeatability – the same tests can be re-run in exactly the same manner eliminating the risk of human errors such as testers forgetting their exact actions, intentionally omitting steps from the test scripts, missing out steps from the test script, all of which can result in either defects not being identified or the reporting of invalid bugs (which can again, be time consuming for both developers and testers to reproduce)

Maintenance of the test suite – From my experience I have found myself in situations where the test suite has become out of date – new functionality has been introduced or existing features have been changed in the way they work and the test cases are no longer up to date as the tester(s) has had no time to go back and update the test scripts. When tests are automated and run after each build, those that are out of date will fail hence, forcing the tester to go back and fix the test script – this process ensures the test scripts are kept up to date and quality of the software is maintained

Reusable – the automated tests can be reused on different versions of the software, even if the interface changes.

Increase Coverage – testers can create a test suite with tests to cover every feature within the application.

Cost Reduction – This can be of benefit when the number of resources required for regression testing are reduced.

In a nutshell, automation helps to save time by reducing the time taken to run tests;  Increases the quality of the software and testing process through reliability, repeatability and comprehensiveness of the test suite;  Utilizing manpower more effectively by applying skills and time where they are needed most and increasing test coverage.

Of course, automating tests does not mean we are eliminating the testers role. There will always be a place for a human tester within a project team, as not every test associated with a feature can be automated and not every project is suitable for automated testing. As a tester, automation is about making our lives easier, by using it to overcome problems such as time and testing more efficiently to ensure quality is maintained not just within the application being developed but also within the testing process.

Having a mobile version of an application has now become a must. With mobile devices ranging from different OS platforms and tabletdevices – as testers, we have a lot on our plate!!

However, there are key points that we must consider whilst drafting out test plans and scripts for any mobile device. In this blog I attempt to cover some of the key points along with functional testing we should consider for mobile testing…

User Interface

Unlike our desktops, mobile devices come with various interfaces, from small screens that can be re-orientated when the device is physically rotated, touchscreen only devices to those which allow a combination of touchscreen and hard keypads, soft keypad only and not forgetting the various navigation methods such as hard keys and trackballs.

Being familiar with the device you are/will be testing on is important, if you are not, ensure you have a play around with it before testing. Check for the following:

• Check for overall colour scheme/themes of the device. For example, Windows phone allow you to change the accent colour and background colours.
• Style and colour of icons
• Progress indicators when pages are loaded
• Menu’s – how they are invoked and typical items they contain
• Overall responsiveness of the application on the device

Screen Orientation/resolution: Applications should be tested in portrait and landscape view. Rotating the device fast, and checking to see the application’s response and errors

Touchscreens: Things to consider when testing touch screens are:

• Multi-touch such as : pinch to zoom and single touch.
• Long touch and short touch. For example, on some phones, pressing and holding an item will bring up a context menu, or a secondary function of a button.

Button Size & Position: Buttons and icons should be large enough to be seen clearly and be clickable by a fingertip.

Workflow: Use of radio buttons and checkboxes to minimize the amount of typing required, as this can be time consuming.

External Factors

Factors such as interaction with other devices and interruptions from the devices own functions such as incoming phone calls should be considered.

Factors to consider are:

• Loss of Network Connections
• Memory Card usage
• 3G Network, 4G network, 2G network
• No SIM card in the device
• In airplane mode
• Test intermittent network scenarios that a user might encounter in the real world such as:
  • Walk out of Wi-Fi range so the connection automatically switches to 3G/2G (for example, in a large building like a hospital or airport, or outdoors)
  • Ride in a lift or on a train where the network connection may go up and down
  • No network connection available at all

Device Options such as Screen Timeout/Auto on/off: Is your application subject to screen dimming or automatically turning off even when it is actually busy? For example, you wouldn’t want your screen to dim or turn off while watching a slideshow of your photos.

Screen orientation: You may be able to enable/disable automatic orientation switches when the device is rotated. Does your application apply the setting set on the device?

Font: Does choosing a different font family, size, or style affect the appearance and usability of your application?

Connections: Using one of the connections on a device, such as Bluetooth or Microsoft Direct Push (only on Windows Phone devices), could have adverse effects on your application. How does enabling/disabling Bluetooth or other connection types affect your applications behaviour?

Emulator Use

Can be a great way to achieve testing coverage across multiple devices, but it is not safe to assume that just because your application works on an emulator, it will work on the actual device itself. When using Emulators, it is important to consider the following points:

• Not all activities can be realistically emulated, like switching network connections, or taking a picture or video.
• Some activities don’t work at all on emulators, like streaming video on a Blackberry emulator.
• Due to lower device power and memory, the application could exhibit slower performance overall when run on an actual device (versus in an emulator on your powerful desktop computer).
• If the emulator and actual device have different resolutions, your screens may not display as you expect.

Stress Testing

Mobile device applications have much less memory and power available than PC’s

Security Testing

This applies to applications which require sensitive data storage such as banking applications, and how the application behaves under various device permission schemes.

Mobile devices will no doubt continue to grow, and as testers we will need to keep up to date with the latest gadgets out there to keep on top of the latest changes in OS and functions belonging to a gadget. No doubt the device functions discussed in this blog will continue to grow.