Introduction

The IGB project integrates testing and development, and everyone plays a role. This approach is a type of "agile testing".

Notice that the JIRA kanban board includes testing lanes. When developers modify the code to implement stories described in a JIRA issue, the team performs testing and code review as the issue moves to completion. This helps reduce the testing burden at the end of a development cycle when we release a new version to users. Also, it ensures we maintain high quality of intermediate development versions. This enables interested users to download and run pre-release versions with confidence.  

When we are ready to release a new version to users, we also perform general smoke-testing of a release candidate as described in the pages below.

To manage the testing process, we create a shared document (usually a Google doc) that testers edit together. We use this shared document to describe problems with screen images and text. We also use the document to track progress through the testing protocols. 

Overview - agile testing 

When developers finish a first draft of a new feature or bug fix, they move the corresponding JIRA issue into the "Needs 1st Level Review" column of the Kanban board. At that point, someone else on the team assigns the issue to themselves and starts the testing process. Before moving the story to the next lane, we perform a code review. (This is described elsewhere.) In addition, we perform functional testing of the new code, using the developer's branch-specific installer. If any bugs or errors affecting function are noted, the tester returns the issue to the To-Do column and re-assigns it to the original developer. Once the issue passes both the code review and functionality review, we move the issue the next lane and un-assign it.

Later, when changes have been merged into the master branch (the main line of development), we double-check that the issue can build under Bitbucket pipelines. If it passes this hurdle, we next move the issue to the Needs Testing lane of the kanban board. We "un-assign" the issue, and it stays in that lane until someone has time to perform functional review with the master branch installer. It's important to perform additional testing at this stage to ensure that the developer's changes interact well with other changes in the code base. At this point, the master branch installer is a pre-release version of IGB. 

Overview - release testing 

Before making an official release of IGB, we test release candidate installers on the three major platforms - Mac, Windows, and Linux. This is important because the platforms are very different, and they handle security differently. Also, their security models change over time. For example, sometimes an older version of the IGB installer that worked great on older versions of the Apple OS suddenly becomes in-installable on a newer version. 

For release testing, we typically set up a bank of computers side-by-side and go through each smoke testing and systems testing protocol together, testing each protocol on the three platforms simultaneously. This saves a lot of time because it allows comparing behavior across operating systems and uncovers OS-specific problems. 

General testing procedures

To check IGB functionality on a system, you should test in two ways:

• Test on a system that mimics a completely naive users who has never installed IGB on their computer before - new IGB users
• Test on a system that is running a functional copy of the current IGB release - existing IGB users

Note that you will need to understand how the IGB installation process works on the system you're testing. We use the Install4J tool to install IGB and a Java virtual machine onto the user's computer. Investigate and understand where the installer is copying the IGB code (jar files) and the Java virtual machine. In addition, be aware that the Install4j software runs every time IGB launches. The installer sends a request to the IGB server and requests a file called updates.xml. If a new version of the software is available, the installer helps the user download and install it. Note that IGB also saves information about user preferences to the local system. Therefore to mimic a completely naive system, you must clear the user preferences. 

Naive system quick start. 

To remove all traces of IGB and mimic a naive system:

• If IGB is already installed, launch it and clear the IGB preferences. (See User's Guide for details.)
• Open the installation directory and run the uninstall script. 
• Delete the .igb ("dot-IGB") directory in the ~/home (Mac and Linux) directory or its Windows equipment on your computer. 

Installing the test target

The test target is the version of IGB that you are planning to test. This is either a branch-specific installer (in the case of agile testing) or a release candidate (for release testing.)

Download the test target installer and run it. Instead of installing it in the default location, install the test target version (branch build or release candidate) on your computer in a directory named for the release. In addition, answer "no" when asked if you want to create a shortcut.  Following this convention will ensure fewer conflicts. 

Once you have set up your system, check