Software Testing Tools Blog – Testertools

Search Engine Land have featured a great article on A/B Testing, Prices and Processes.  Titled   When A/B Testing Prices, Proceed With Caution and written by Scott Brinker the article provides a great example of A/B Testing in practice.

I had an interesting A/B testing experience over the holidays. This time, it wasn’t an A/B test that I was running, but rather an A/B test in which I was an (initially) unsuspecting participant.

It reminded me of the negative side effects that certain kinds of tests can have on customers — sometimes your best customers — and the steps that marketers should consider to mitigate those risks.

This is a cautionary tale, but I don’t want to lambast the the company running the test. They ultimately handled the situation fairly. So I will refer to them anonymously as a business software provider named Acme.

For more read When A/B Testing Prices, Proceed With Caution.

Parasoft Corporation, industry leader in automated defect prevention and detection, today introduced a new release of Parasoft Jtest: the industry’s most comprehensive Java testing solution. Parasoft Jtest automates static analysis, unit testing, coverage analysis, runtime error detection, and code review.

Parasoft is recognized by software development professionals as the leader in software development lifecycle automation. By integrating software development management, quality lifecycle management, and dev/test environment management, we help teams deliver better software faster. Since 1987, over 10,000 companies — including 58% of the Fortune 500 — have turned to Parasoft to ensure that quality software can be produced consistently and efficiently. For more information, visit the Parasoft web site and ALM Best Practices blog.

The latest release significantly extends support for a variety of secure application development guidelines and compliance reporting (e.g., for CWE-SANS, OWASP, and NIST SAMATE). The extended coding guidelines are available for desktop, pattern-based, and data flow-based execution. The extended static analysis for security analysis works hand-in-hand with Jtest’s runtime error detection, which identifies security vulnerabilities that occur as the application is exercised during any automated or manual tests.

Moreover, Jtest’s desktop analysis optimizations slashes the time and effort required to find and fix critical defects — as well as confirms their successful removal. This helps teams ensure that the risks exposed by static analysis, unit testing, and runtime error detection are remediated rapidly and efficiently. Specific capabilities introduced include continuous static analysis, one-click retesting, quick mode, debug mode, and more.

Additional highlights of Parasoft Jtest 9.2 release include:

• Enhanced Spring support with static analysis and unit testing
• Enhanced Ant/Maven plugin facilitates continuous integration and testing
• Support for Eclipse JUnit launch configurations
• Redesigned metrics reporting

Keynote Systems (NASDAQ: KEYN), the global leader in Internet and mobile cloud testing and monitoring, today announced that Google has selected Keynote Mobile Internet Testing Environment (MITE) as the heart of its new free GoMoMeter tool, helping businesses evaluate the mobile-readiness of their websites. Keynote MITE enables the GoMoMeter to show businesses how their sites look on their customers’ mobile devices. MITE also helps the tool assign a mobile-readiness score for each site, delivering results to users with consistent, near real time performance.

“We selected Keynote,” said Ryan Harms, senior producer at Mullen Advertising, “because of the support of the Keynote team and their core software’s unique ability to address our business case in a cost-effective manner. Keynote’s mobile device emulation capabilities make the service highly scalable. We’re expecting traffic in the thousands of users every day. Because a solution relying on real mobile phones would have required us to physically add more phones to grow, we didn’t think that a hardware-based solution could scale quickly and easily enough to meet our needs. Keynote’s MITE software emulation approach is infinitely scalable; it’s very easy to spin up more software profiles. Finally, Keynote has been tested at an enterprise level and meets our quality standards.”

The GoMoMeter tool is designed to allow a business person to evaluate the mobile readiness of their website. The user simply types in the site’s URL to see what their site actually looks like on a customer’s mobile device. The tool also combines information from the mobile device emulation (e.g., whether the text and images are visible without pinching or zooming), the file loading time statistics, and the answers to key questions (e.g., can buttons and links be used with a thumb?) to create a mobile website usability score. This tool is the centerpiece of Google’s new http://www.howtogomo.com/en/b/w website and educational campaign, which was developed by Mullen Advertising to introduce businesses to the benefits of mobility and help companies evaluate their site’s mobile readiness.

“As mobile device usage increases, mobile optimized websites are incredibly important to the growth initiatives of many businesses. These sites must provide an excellent experience to visitors using one of many thousands of mobile devices. Keynote MITE improves the mobile Web user experience by automating the testing of mobile websites and optimizing mobile Web content based on device-specific design and performance best practices across thousands of emulated mobile devices. By integrating MITE’s unique device emulation and rendering capabilities, via an API, into the GoMo site, Google is making it that much easier for enterprises to quickly see and experience their website as a mobile user would,” said Vik Chaudhary, vice president of product management and corporate development at Keynote. “Keynote and Google share a commitment to improving the mobile user experience, and educational tools such as the GoMoMeter are a much needed step forward in advancing awareness for the mobile challenges online enterprises face.”

FaSur Technologies has officially released the automated software testing tool, Lorica Version 3.0.

The first release already has been used with clients in higher education, health care, social media and insurance.

Lorica ATS can test on any web application and is compatible on all major web browsers (like Chrome, Firefox, Internet Explorer and Safari). The Lorica testing tool is available in two pricing models: Licensing and TaaS (Testing-as-a-Service).

Micro Focus is focusing on performance testing of mobile web applications through SilkPerformer, a software application performance testing solution. <

”Micro Focus SilkPerformer offers simulation capabilities for a variety of mobile phones, such as Android, iOS and Blackberry devices. It also simulates the bandwidth limitations of mobile network connections and supports all existing and upcoming mobile phone standards like GPRS, EDGE, UMTS, HSDPA, HSPA+ and LTE, while its mobile browser simulation emulates mobile device traffic from different parts of the world,” said Archie Roboostoff, Borland solutions portfolio director, Micro Focus.

iPass, a leading provider of enterprise mobility services, has deployed SilkPerformer and SilkTest, to facilitate access from any device on any network while giving the enterprise essential visibility and control.

”The exemplary use of Silk by iPass is just one example of how we are helping enterprises outpace the building momentum of today’s mobility trend. In fact, the business advantages that Silk provides are felt right across iPass operations,” said Nitin Dang, country GM, Micro Focus India.

TestPlant, the UK-headquartered international software company, has launched the eggPlant Mobile Testing Solution ( http://www.testplant.com/mobile/mwc). eggPlant is the world’s leading robotic test tool for GUI testing used by many leading software companies, by significant defence, security and aerospace organisations and by global media broadcasters and on-line entertainment businesses. The mobile testing solution extends eggPlant’s reach into software running on smartphones and tablets across the major mobile operating systems.

The launch is in partnership with RealVNC, the original developer and leading provider of VNC(R) technology used by millions of computer users around the world. Testing software or apps on mobile devices is a major pain point for app developers, for device manufacturers and for network operators. There has not been an effective in house automated solution to testing software on the full range of devices. Existing techniques are either manual, which is extremely tedious and error-prone, or electro-mechanical, which is limited, does not reproduce a real user experience test and can place IP outside the enterprise environment.

eggPlant uses RealVNC’s VNC Mobile Solution to connect directly to a device. A true user experience test is created as eggPlant is capable of recognising icons, colours, text and fields on any display. A test script is automatically generated as eggPlant is trained to move around an app using its image store to spot and compare display prompts. Multiple tests become effortless and eggPlant reports test failures with helpful screen images of the exact point of failure. An automated test script can be created on one device (or emulator) to be repeated many times on other devices.

Typemock, the leading provider and pioneer of easy unit testing solutions, announced today their prediction that a rise in unit testing will occur in 2012 as software development continues to climb significantly. Eli Lopian, founder and CEO of Typemock , speaking at a developer seminar said, “In 2012, we will see a major shift in the efforts of software developers to focus on better code. Unit testing is significantly more important today as we see an eruption in software development in the coming years.” In addition, Lopian added “Unit testing is a key aspect of all Agile methodologies, and, in order to remain agile, unit testing must be properly executed.”

In an article on the Testing Tools Landscape, a leading analyst firm noted, “Now it is time for quality management and testing to respond to this faster-moving environment. Functional testing tools are not enough. Quality must move beyond the purview of just the testing organization and must become an integrated part of the entire software development life cycle (SDLC) to reduce schedule-killing rework, improve user satisfaction, and reduce the risks of untested non-functional requirements such as security and performance.” In addition, a recent Forrester report recommended that developers engage in automated testing, including unit testing, in order to build higher quality software.

According to one notable developer, there will also be more consideration for unit testing in software development. Kevlin Henney, co-author of several software architecture and programming practice books and former columnist for Better Software Magazine, was interviewed by Typemock on his predictions for the upcoming year, commenting that there will be “increased acceptance of unit testing and techniques surrounding it, particularly in domains where it has been considered off the menu.”

Corey Haines, self-proclaimed Software Journeyman and co-founder of Code Retreat had this to say: “Honestly, I see more of the same. Lots of people exposed to it for the first time. Lots of people trying it, finding it hard, then saying it doesn’t work. Lots of people trying it, finding it hard, keeping with it, and seeing benefits over the years.”

“Unit testing is like staying healthy,” said Lopian. “Staying healthy requires best practices such as eating right and working out. Similarly, development teams need the right practices in order to innovate faster. Just as it’s hard to start working out, many find it’s hard to unit test and thus stop – despite its well-known benefits. This is why Typemock’s vision has always been providing easy unit testing tools for agile software developers to release better software, even with legacy code.”

JCGs (Java Code Geeks) have released an article on common mistakes of Software Testing.

Seven mistakes of software testing lists a number of common mistakes that are made in each of the test phases.

Though most developers know the importance of testing, it seems that a lot of them still aren’t testing enough. And if they write tests, they test just test wrong.

UNIT TESTS

If some tests are written in applications, most of them will be unit tests. It’s easy to test a utility class by just calling all utility methods, passing some values and checking if the expected result is returned.

A first mistake arises here. Most people don’t think out of the box, or not enough. You can test that 1 + 1 =2 , that 2 + 1 = 3 and that 3 + 1 = 4. But what’s the benefit of doing almost the same test 3 times? It’s better to test the boundary cases. Are the arguments of the sum( ) method primitive types or Objects? If they are Objects, what happens if you pass null values? If an exception is thrown, is that the expected one? Does it clearly tell what the problem is?

JCG is an independent online community focused on creating the ultimate Java-to-Java developers resource center; targeted at the technical architect, technical team lead (senior developer), project manager and junior developers alike. JCGs serve the Java, Scala, Android, SOA, Agile and Telecom communities with daily news written by domain experts, articles, tutorials, reviews, announcements, code snippets and open source projects.

To read the full article visit Seven mistakes of software testing

Byteman is a tool which simplifies tracing and testing of Java programs. Byteman allows you to insert extra Java code into your application, either as it is loaded during JVM startup or even after it has already started running. The injected code is allowed to access any of your data and call any application methods, including where they are private. You can inject code almost anywhere you want and there is no need to prepare the original source code in advance nor do you have to recompile, repackage or redeploy your application. In fact you can remove injected code and reinstall different code while the application continues to execute.

The simplest use of Byteman is to install code which traces what your application is doing. This can be used for monitoring or debugging live deployments as well as for instrumenting code under test so that you can be sure it has operated correctly. By injecting code at very specific locations you can avoid the overheads which often arise when you switch on debug or product trace. Also, you decide what to trace when you run your application rather than when you write it so you don’t need 100% hindsight to be able to obtain the information you need.

When testing your application you can use Byteman to inject faults or synchronization code, causing your application to perform unusual or unexpected operations required to exercise a test scenario. Byteman provides a library of built-in functions which allow you to do anything from generating simple error conditions to propagating complex error flows which require coordinated actions in different parts of your application. However, you are not limited to these operations. You can inject almost any Java code into your application so long as the classes you refer to are in scope at the injection point. You can also replace or extend the available built-in functions by supplying a POJO (plain old java object) as a plugin. So, Byteman makes it easy for you to program even the most complex test scenarios.

Byteman works by modifying the bytecode of your application classes at runtime. Since it only needs access to bytecode this means it can modify library code whose source is either unavailable or unable to be recompiled. This even includes the Java code which forms part of the Java virtual machine, classes such as String, Thread etc. So, with Byteman you can trace what the JVM is doing on behalf of your application code or cause JVM classes like FileInputStream or Map to throw exceptions when your application calls them.

Byteman uses a clear, simple scripting language, based on a formalism called Event Condition Action (ECA) rules to specify where, when and how the original Java code should be transformed. An event specifies a trigger point, a location where you want code to be injected. When execution reaches the trigger point the rule’s condition, a Java boolean expression, is evaluated. The Java expression (or sequence of expressions) in the rule action is executed only when the condition evaluates to true. Normally execution continues from the trigger point once the inejcted code has been executed. However, rule actions may also throw an exception or force an early return from the triggering method.

Byteman 2.0.0 is now available for download under the GNU LGPL. It requires a JDK 6 or higher JVM. The release includes a user guide which documents the scripting language and explains how to use byteman to inject faults or tracing and monitoring code. There is also a tutorial showing you how to get started with Byteman and a follow-up tutorial explaining how to use Byteman to perform fault injection testing. See the documentation page for more information.

ARCAD Software is now shipping version 8.12 of its application lifecycle management (ALM) suite for the IBM i OS. Among the enhancements is the capability of its regression testing product, called Verifier, to test multiple types of user interfaces simultaneously, thereby making it easier for IBM i shops to test Windows, Web, and mobile interfaces with the same effort used to test interactive 5250 screens or batch jobs.

The ARCAD ALM suite is a collection of tools used to aid developers in the development, testing, and roll-out of new and enhanced applications. The software is primarily used on the IBM i server, although some components support other platforms.

The main enhancement delivered by ARCAD with version 8.12 of the suite is a new data-driven capability in its regression testing engine. The primary impact of this database enhancement is that tests are no longer dependent on the user interface that is used. ARCAD customers can now test Windows-based interfaces to IBM i applications, right along side the equivalent Web and 5250 interfaces to the same IBM i application.

The technology used to create the interface–whether it was .NET, Java, Delphi, C++, or LANSA for a thick-client Windows interface, or PHP, AJAX, Flash, Java Server Pages (JSP), or plain HTML for a Web interface–no longer dictates the developer to run multiple regression tests to ensure proper functioning. This also applies to programs without user interfaces, such as IBM i batch jobs and integration routines that utilize Web services technologies or MQ Series.

The vendor says the “openness” of the underlying data capture mechanism has been enhanced, “so that ARCAD-Verifier can easily ‘connect’ to your processes, targeting capture and replay operations.” The results of the regression test are then compared to data created, modified, or deleted in the IBM i database.

As ARCAD CEO Philippe Magne explains, many test automation projects fail because the tool being used is limited to a comparison of user interfaces. “By focusing on the comparison of data stored in the database, ARCAD-Verifier gives greater flexibility of use and a wider scope, all with the same level of security,” he says in a press release.

Magne says the enhancement to Verifier makes it easier to use than other testing tools. “This ‘data driven’ testing tool becomes very easy to use, and can be placed under the sole responsibility of end-users, with no need for the development team to be involved.”

This feature had been requested by many of ARCAD’s customers, Magne says in a November newsletter. “Many of you had asked for this capability and we struggled for two solid years to find the stable, durable solution we were looking for. Our search is now over and our mission is accomplished.”

ARCAD is based in France, and has its U.S. headquarters in the former SoftLanding Systems stomping grounds of Peterborough, New Hampshire. For more information, see the company’s website at www.arcadsoftware.com.