AUTOMATED GUI TESTING OF MOBILE JAVA APPLICATIONS

June 16, 2010 — anuragkhode

Sharing with you very Interesting thesis by Marcin Zduniak from Poznan University of Technology(Poland) on AUTOMATED GUI TESTING OF MOBILE JAVA APPLICATIONS.

This thesis describes certain preliminary results of the attempt to design and implement a framework for capturing and replaying user interaction in applications written for the Java 2 Micro Edition environment.

The thesis :-

  • Provides Theoretical background to software testing in general and to GUI  and mobile application testing in particular.
  • Describes some of existing solutions that attempt to solve the problem of mobile applications testing.
  • Introduces high-level design and architecture of the RobotME framework.
  • Gives  some limited details of implementation, required to understand the proposed solution in the thesis.

The thesis seems interesting.You can find the complete thesis here:-

AUTOMATED GUI TESTING OF MOBILE JAVA APPLICATIONS

Posted in J2ME Application Testing, Mobile Application Testing, Mobile application Testing and Automation tools. Tags: . 1 Comment »

While testing data calls in J2ME mobile application

November 25, 2009 — anuragkhode

When data call request is in active state, the stability of application needs to be tested by removing the battery. This act should not affect the application adversely after restarting the application again.I am just sharing one scenario for a j2me Application which may help you while testing your application.

Scenario: Removed a battery when forground data call was active.

Issue Observed:

  • While testing my J2ME application when I came across this scenario, I found that RMS of the application was totally corrupted when application restarted after reinserting the battery. Interestingly user was unable to go beyond the splash screen showing a complete white screen forever.
  • This happened when I removed the battery when data call progress status was 50% only and the issue was reproducible 100% of the times.
  • Interestingly application was working fine for all other states of data call.

On discussion with the development team, corruption of RMS is natural after playing with the application like that.

Tester’s Perspective:

However a Testing Engineer should make sure that application should gracefully handle this situation by giving some user friendly message, something like, “Unable to launch the application as the database is corrupted. Please delete the application from your phone and reinstall” or so on.(Infact the issue should not come)

Using this experience, I applied same calculations for another application and was successful to replicate the same bug. (This is error guessing which gradually comes and enhances with the experience.)

What is RMS ?

  • It is a Record Management Syatem which provides a mechanism through which MIDlets can persistently store data and retrieve it later.

Read More

 

Posted in J2ME Application Testing, Mobile Application Testing. Leave a Comment »

Testing Checklist for Mobile Applications

October 23, 2009 — anuragkhode

By-Anurag Khode,Copyright 2009-10

No. Module Sub-Module Test Case Description Expected Result
1 Installation
Verify that application can be Installed Successfully. Application should be able to install successfully.
2 Uninstallation
Verify that application can be uninstalled successfully. User should be able to uninstall the application successfully.
3 Network Test Cases
Verify the behavior of application when there is Network problem and user is performing operations for data call. User should get proper error message like “Network error. Please try after some time”
4

Verify that user is able to establish data call when Network is back in action. User should be able to establish data call when Network is back in action.
5 Voice Call Handling Call Accept Verify that user can accept Voice call at the time when application is running and can resume back in application from the same point. User should be able to accept Voice call at the time when application is running and can resume back in application from the same point.
6
Call Rejection Verify that user can reject the Voice call at the time when application is running and can resume back in application from the same point. User should be able to reject the Voice call at the time when application is running and can resume back in application from the same point.
7
Call Establish Verify that user can establish a Voice call in case when application data call is running in background. User should be able to establish a Voice call in case when application data call is running in background.
8 SMS Handling
Verify that user can get SMS alert when application is running. User should be able to get SMS alert when application is running.
9

Verify that user can resume back from the same point after reading the SMS. User should be able to resume back from the same point after reading the SMS.
10 Unmapped keys
Verify that unmapped keys are not working on any screen of application. Unmapped keys should not work on any screen of application.
11 Application Logo
Verify that application logo with Application Name is present in application manager and user can select it. Application logo with Application name should be present in application manager and user can select it.
12 Splash
Verify that when user selects application logo in application manager splash is displayed. When user selects application logo in application manager splash should be displayed.
13

Note that Splash do not remain for fore than 3 seconds. Splash should not remain for fore than 3 seconds.
14 Low Memory
Verify that application displays proper error message when device memory is low and exits gracefully from the situation. Application should display proper error message when device memory is low and exits gracefully from the situation.
15 Clear Key
Verify that clear key should navigate the user to previous screen. Clear key should navigate the user to previous screen.
16 End Key
Verify that End Key should navigate the user to native OEM screen. End Key should navigate the user to native OEM screen.
17 Visual Feedback
Verify that there is visual feedback when response to any action takes more than 3 seconds. There should be visual feedback given when response time for any action is more than 3 second.
18 Continual Keypad Entry
Verify that continual key pad entry do not cause any problem. Continual key pad entry should not cause any problem in application.
19 Exit Application
Verify that user is able to exit from application with every form of exit modes like Flap,Slider,End Key or Exit option in application and from any point. User should be able to exit with every form of exit modes like Flap,Slider,End Key or Exit option in application and from any point.
20 Charger Effect
Verify that when application is running then inserting and removing charger do not cause any problem and proper message is displayed when charger is inserted in device. When application is running then inserting and removing charger should not cause any problem and proper message should be displayed when charger is inserted in device.
21 Low Battery
Verify that when application is running and battery is low then proper message is displayed to the user. When application is running and battery is low then proper message is displayed to the user telling user that battery is low.
22 Removal of Battery
Verify that removal of battery at the time of application data call is going on do not cause interruption and data call is completed after battery is inserted back in the device. Removal of battery at the time of application data call is going on should not cause interruption and data call should be completed after battery is inserted back in the device.
23 Battery Consumption
Verify that application does not consume battery excessively. The application should not consume battery excessively.
24 Application Start/ Restart
1. Find the application icon and select it 2. “Press a button” on the device to launch the app. 3.Observe the application launch In the timeline defined Application must not take more than 25s to start.
25 Application Side Effects
Make sure that your application is not causing other applications of device to hamper. Installed application should not cause other applications of device to hamper.
26 External incoming communication – infrared
Application should gracefully handle the condition when incoming communication is made via Infra Red [Send a file using Infrared (if applicable) to the device application presents the user] When the incoming communication enters the device the application must at least respect one of the following: a) Go into pause state, after the user exits the communication, the application presents the user with a continue option or is continued automatically from the point it was suspended at b) Give a visual or audible notification The application must not crash or hung.
Posted in J2ME Application Testing, Mobile Application Testing, Test Cases. Tags: , . 30 Comments »

Test Plan for a mobile applications

October 22, 2009 — anuragkhode

Source-WIKI Forum Nokia

1 Introduction

1.1 Overview

This document explains the testing methodology for a mobile application, and is to be used as a guide for the testing activity.

The intended audience for this document: The Project Managers Product development team members Test engineers

1.2 Scope

The scope of testing as explained in the document is to test the operating characteristics of an application that runs on mobile devices supporting J2ME. The tests are organized by requirement category such as usability, functionality, security, etc. The procedure for carrying out testing in terms of preparation of test cases, test environment setup, defects logging and reporting are explained.

The article does not address the following:

  • Content censorship (i.e. assessment against standards for violence, gambling, political messaging etc.) for the purpose of preventing the deployment or sale of an application. Distribution, DRM etc.
  • Testing requirements specific to a particular manufacturer’s (or network operator’s) device, user interface, and standards (e.g. WAP) implementation.

1.3 References Mention the documents of references

1.4 Acronyms

Acronym Expansion

DRM Digital Rights Management

J2ME™ Java™ 2 Platform Micro Edition
2 Test Plan and Strategy

2.1 Unit Testing

2.1.1 Objective

The objective of Unit testing is to verify that a particular module of source code is working properly. Unit tests are designed to test a single class or component or module in isolation. Developers run unit tests, and only for the components they are working on.

2.1.2 Entry Criteria

  • Test cases are reviewed
  • Build is complete and self test done
  • Unit Test environment is set up

2.1.3 Exit Criteria

  • All planned test cases are executed
  • Units are working as per the expected results
  • Defect are fixed in the code and tracked to closure

2.1.4 Logging Tests and Reporting

The developer will fix the defects that are found in unit testing. Additionally, if defects corresponding to other modules or components are found during unit testing, these will be reported.

2.2 System Testing

In System Testing, separate units (packages / modules / components), or groups of units of the application are united and tested as a completely merged application. The purpose of System Testing is to identify defects that will only surface when a complete system is assembled. Verification of the system at this stage might include: functionality, usability, security, installation etc. It is intended to validate the application as a whole.

The rest of this document mainly explains how System Testing is performed by the testing team.
2.2.1 Testing Procedure

The steps in testing consist of:

  • Creation of all the test scenarios and test cases
  • Preparation of a test case document that has a brief description of the test case , steps to conduct tests and expected result
  • Defect Report generation.

Outlined below are the main test types that will be performed

a. Application Characteristics (AC) – Information about the application is provided to help the testing team in the testing work.

b. Stability (ST) – Focusing on the application being stable on the device.

c. Application Launch (AL) – Once an application is loaded it must start (launch) and stop correctly in relation to the device and other applications on the device.

d. User Interface (UI)
e. Functionality (FN) – Documented features are implemented in the application and work as expected. Sources for the information are user manuals, formatted application specification documents and online documentation.
f. Connectivity (CO) – the application must demonstrate its ability to communicate over a network correctly. It must be capable of dealing with both network problems and server-side problems.

g. Personal Information Management (PI) – The application accessing user information needs to be able to do it in an appropriate manner and not to destroy the information.

h. Security
2.3 Regression Testing

This is an additional step, and is done prior to taking up system testing which is to test new functionality. Regression testing consists of running a set of standard tests to ensure that old functionality has not been broken by new functionality. Regression tests are also run if a new release is made after fixing a number of defects.
2.4 Pass/Fail Conditions

It is expected that an application must pass all the tests in each test category to be successful.
2.5 Test Report

For each report, the following information is provided: • The name of the application

• The version number of the application

• Device used for testing

• Device firmware version
For each error reported, the following information is provided: • Description of the error

• Frequency of occurrence of error: Systematic or Random or Once

• Location of the error in the application

• Steps to reproduce the error

3 Schedules for Testing

This will be decided in consultation with the project manager.
4 Risks and Assumptions

4.1 Risks:

The following may impact the test cycle:

  • Device availability
  • Any new feature addition/modification to the application which is not communicated in advance.
  • Any delay in the software delivery schedule including defect fixes. Any changes in the functional requirements since the requirements were signed-off/formulated

4.2 Assumptions:

  • Every release to QA will accompany a release note specifying details of the features implemented and its impact on the module under test.
  • All “Show-Stopper” bugs receive immediate attention from the development team.
  • All bugs found in a version of the software will be fixed and unit tested by the development team before the next version is released
  • All documentation will be up-to-date and delivered to the system test team.
  • Devices, Emulators and other support tools will be fully functional prior to project commencement.
  • In case of lack of required equipment or changes in the feature requirements, the test schedules may need to be reviewed.

5 Entry and Exit Criteria

5.1 Entry Criteria

  • Development of the application is complete
  • Successful completion of unit testing for the applications
  • Release of software to the test environment
  • Dedicated resources are allocated
  • Approved test bed to carry out system testing.
  • Test environment is up and working

5.2 Exit Criteria

The Following is the criteria when the testing will be stopped for this module:

  • All test cases have been executed and at least 95% have passed successfully. The remaining 5% do not impact critical functionality
  • All test results have been evaluated and accepted.
  • There are no showstoppers or high criticality defects unresolved or outstanding

6 .Test Metrics

Following metrics will be captured and reported as part of the Test

  • Summary Report
  • Test Design effort
  • Test execution effort
  • Number of Test Cases executed
  • Number of Defects and their classification
  • Test Coverage (Number of test cases executed/Number planned)

7 Logging Tests and Reporting

Some third party applications will be used for reporting bugs found during test execution. The QA team will log defects in the tool as testing progresses.

8 Roles and Responsibilities
The roles and responsibilities of the testing team for the project are as follows:

8.1 Project Manager / Test Manager Responsibilities:

  • Overall responsibility for managing the testing process
  • Approval of test documents
  • Approval of inspections/reviews done as per the test plan
  • Providing resources for the project

8.2 Test Lead

Responsibilities:

  • Requirement gathering
  • Planning and estimating for testing
  • Tracking and monitoring the testing as per the test plan
  • Reporting the project status

8.3 Test Engineer

Responsibilities:

  • Creating the test cases as per the test plan
  • Executing the test cases
  • Documenting the results and logging errors.

9. Deliverables

  • Test Plan
  • Test Cases Document – Document with a description and expected result for each test case.
  • Test Results – The Pass/Fail status of each test cases and the list of issues.
Posted in J2ME Application Testing, Mobile Application Testing. Tags: , . 12 Comments »

Nokia Test Criteria for J2ME Applications

October 10, 2009 — anuragkhode

Introduction:

This document describes the test cases conducted for Java™ Platform, Micro Edition (Java™ ME) applications in addition to Unified Testing Criteria (UTC). Meeting Unified Testing Criteria is a requirement for the Java Verified™ Program. All applications that will be embedded to Nokia devices as part of a Total Product Offering (TPO) or operator variant project should be tested according to these test cases. Meeting Nokia test criteria is a requirement for third-party applications that are delivered via Nokia sales channels, including:

  • Applications preinstalled to ROM, memory card, or hard-disk drive;
  • CD-ROMs in devices’ sales packages;
  • Downloads, Nokia Web/mobile Web page downloads.

The test cases are applicable for applications targeted at Series 40 and S60 devices. These tests are done simultaneously with Unified Testing Criteria available at www.javaverified.com.
*********************************************************************
2.1 Nokia values for applications

NOK-01: Nokia values
Test description:
Applications must be in line with Nokia values and therefore must not have any references to the issues listed below.


Steps to conduct the test:
While using the application, check that the application does not have any references to:

  • Excessive violence and gratuitous depictionof violence against humans/animals (for example, no suicidal bombers, killing of innocent or outsiders);
  • Depictions of pornography or pedophilic imagery;
  • Drinking of alcohol;
  • Substance abuse including drugs and drug taking;
  • Smoking;
  • Overuse of swearing;
  • Racism;
  • Overtly political messages, such as terrorism, generic battles/fights between the races, slavery/slave trade. Contentious political/global events in historical context, for example Battle Field 1942, are OK;
  • Religious imagery, unless in real-life context;
  • Gambling using real money;
  • Promotions of criminality, criminal actions(depiction of criminal activity where the player is the perpetrator of the crime);
  • Material targeted and marketed solely at young children, for example Teletubbies;
  • Nokia, for example name or logo (except if explicitly agreed upon with Nokia).

Expected test result:
The application does not have any references to the listed items.

**********************************************************************
2.2 Interaction requirements, pausing the application
NOK-02: Interaction requirements, pausing the application
Test description:
Application goes to paused state when the Power key, Menu key, or End key is pressed.
Steps to conduct the test
1. Open the application.
2. Press the Power key, Menu key, and End key
(red key).
3. Check that the application goes to paused
state when one of the keys is pressed.
4. Check that it is possible to continue using
the application after pausing it.
Expected test result:

  • The application goes into the paused state when the Power button is pressed. After that it is possible to continue using the application with the Continue option (and it shows the profiles and other options).This is valid only for applications whose status needs to be paused, that is, in S60 devices.
  • When the Menu key is pressed, the application goes to the background and to the paused state, and the device’s other applications work without problems (for example, taking an image, viewing it, modifying the profiles, starting another MIDlet). After this it is possible to continue the application with the Continue option.The paused state is valid only for applications whose status needs to be paused, that is, in S60 devices.
  • The application goes into the paused state when the red key is pressed, and the telephone view is displayed. After that it is possible to go to the application and continue it with the Continue option. This is valid only for applications whose status needs to be paused, that is, in S60 devices, except for S60 2nd Edition, Feature Pack 3

*********************************************************************
2.3 Specific tests for games

NOK-03: Specific tests for games

Test description

The game indicates the current status clearly and provides feedback. Each rule/function of the game is clear. When the game is over, feedback is provided and there is no paused game session to continue.


Steps to conduct the test:

1. Open the game and start playing.
2. Check that the current status of the game is clear and feedback is provided.
3. Check that the rule/function of each element is clear.
4. Quit the game.
5. Check that some feedback is provided when the game is over.
6. Check that there is no paused game session to continue when the game is over.

Expected test result:

  • The current status of the game is clear and feedback is provided (for example, number of lives available, current level, current score).
  • The rule/function of each element of the game is clear (for example, how the player and the enemies are represented and how they interact).
  • When the game is over, feedback is provided. This includes a warning that the game is over, total score (if any), who won (for example, if the game is played between the device and the player), other game-dependent features (for example, level), and what to do next.
  • When a game is over, there is no paused game session to continue.

**********************************************************************
2.4 Touch UI specific test cases

NOKT-01: UI element size (touch UI only)

Test description

The application UI elements must be large enough for a pleasant user experience.

Steps to conduct the test:

1. Start the application.
2. While using the application, check UI element sizes and visible areas vs. active areas in the following application features:

  • Main menu and submenus;
  • Settings menu;
  • Main functionality of the application.

Expected test result:

  • The target minimum size of a UI element is 7 x 7 mm. In the Nokia 5800 XpressMusic device, the 7 x 7 mm is equivalent to 70 x 70 pixels.
  • The active area of the component must not be smaller than the visible area of the component.

Notes:
For more UI and user experience guidelines and a
usability checklist, visit
www.forum.nokia.com/usability.

The test house will randomly choose one item per feature for measuring. The developer is expected to make sure that all items are in the correct size range.

Exceptions:

When components are located near the edge of the display, the touchable area can be fully extended to the edge of the display (that is, beyond the component’s visible graphics).For example in games where the game board is
essential to fit the screen (for example, Sudoku) or in applications that are meant to be used with Stylus, the size of the UI element can be smaller
than 7 x 7 mm (equivalent to 70 x 70 pixels).

NOKT-02 Touch interaction: Select and activate (touch UI only)


Test description

Touch interaction must follow the S60 UI Style Guide.


Steps to conduct the test:

1. Start the application.
2. While using the application, check the
interaction styles in the following areas of
the application:

• Main menu and submenus;
• Settings menu;
• Main functionality of the application.


Expected test result:

1. The following will operate with one tap:
• Buttons
• Icons
• Items within Options menus

2. The first tap selects the item and the
second tap opens the item unless
another item is selected.

Notes:

The test house will randomly choose one item per
feature for measuring. The developer is expected to
make sure that all items behave correctly.

Exceptions:
It is possible to open or activate an item from a
focusable view directly with a single tap if the
focus is already on the given item.

*******************************************************************

Posted in J2ME Application Testing. Tags: . Leave a Comment »

How to run J2ME applications on devices

October 9, 2009 — anuragkhode

There are three steps to install J2ME applications in device.

  • via USB Cable
  • via OTA
  • via Bluetooth

Steps to Install the application via USB Cable are :

  1. Connect Device with PC, with the help of Cable Cord .
  2. Select  Device Memory or Memory Card folder of Device.
  3. Select and Copy Jad,Jar files of Application from PC.
  4. Paste them in Device Memory or Memory card folder.
  5. Disconnect the connected device and PC connection.
  6. Open Device Memory or Memory card folder in device.
  7. Click on the Jar file of the Application for install the application in device.
  8. Allow all the permission which device ask at the time of installing app.
  9. Set the Device permission for application from Application Settings , Permission like :Network Connection , User’s Read Write data , Ask for Permissions.
  10. Open Application Manager Screen of Device .
  11. Click on Application Logo
  12. Application gets open , Run the application.

Steps to Install the Application through Bluetooth are :

  1. Switch on  Bluetooth in device (Settings->Connectivity->Bluetooth)
  2. Switch on laptop bluetooth.
  3. Select the Jad,Jar files of Application.
  4. Search the Device on which you want to send Application Files.
  5. Select the Device and send Application Jad,Jar file from PC with the help of Bluetooth.
  6. Application’s Files are received through message.
  7. Click on Jar file Message in Inbox of your device
  8. Application Installation process  gets started.
  9. Allow all the permission which device ask at the time of installing app.
  10. Set the Device permission for application from Application Settings , Permission like :Network Connection , User’s Read Write data , Ask for Permissions.
  11. Open Application Manager Screen of Device .
  12. Click on Application Logo.
  13. Application gets open , Run the application.

Steps to Install the Application through OTA (Over The Air) are :

  1. Open Device Web Browser(WAP).
  2. Enter the link for Application site for Download OR Enter the link of Getjar Site.
  3. After opning the site , Select Device from Device List.(In Most Cases there is no need to select device there is only need of GetJar site)
  4. After Selecting the Device, Click on Download Link
  5. Application Jar file gets downloaded on device  and automatically start the installation process.
Posted in J2ME Application Testing. Tags: . 1 Comment »

Introduction to J2ME Emulators

September 29, 2009 — anuragkhode

Aim: The aim of this article is to provide instructions on how to run application/games on the emulator, this is so that the application/games can be tested, the document does not intend to fulfill any other purpose other than instructing how to run application only to test the application/games.

What is an emulator?

An emulator, in the most general sense, duplicates (provides an emulation of) the functions of one system with a different system, so that the second system appears to behave like the first system. Unlike a simulation, it does not attempt to precisely model the state of the device being emulated; it only attempts to reproduce its behavior. A simulator is a specially accurate emulator.

J2ME emulators:

The basic J2ME emulator to be used for this exercise is The Sun Microsystems J2ME™ Wireless Toolkit 1.0.4_01. The emulator is installed after Java 2 SDK Std Edn. V 1.3.

The steps to run applications are as follows:

  1. After installation, the emulator can be easily accessed from the start menu. Before running the application, select the “default device selection” and select a device from a drop down box.

NOTE: Devices are present in the WTK (Wireless ToolKit) folder. This folder is usually named as WTK104 unless changed in the path that it was installed to. The devices are present in the folder called “devices” inside the folder “wtklib”. For example: C:\WTK104\wtklib\devices.

  1. The next step is to provide a path to the application. Select “Run MIDP application” from the start menu under the name that the WTK was installed to. The path to the application .JAD (Java Application Descriptor) is provided.

NOTE: Take care that the path is not too long; the emulator does not work with long

Paths

Example: The emulator is installed with the name” J2ME Wireless ToolKit 1.0.4_01”. This is the method

followed:

Click on start Select” J2ME Wireless ToolKit 1.0.4_01” Select “default device selection” -> Select a device from the drop down box and click “ok” Select start Select” J2ME Wireless ToolKit 1.0.4_01” Select “Run MIDP application” Provide the path to the .JAD file and click on start.

Posted in J2ME Application Testing. 3 Comments »