Software Testing Tools Blog – Testertools

An ongoing problem with specialized platforms is the lack of support for unit testing. Developers are forced to compromise the quality of their tests or their build process in order to get anything working. Recently MonoTouch has made progress in this area, but Windows Phone and Mono for Android still lag behind.

Windows Phone 7 / Silverlight

For Windows Phone 7 there are two testing routes available to developers. The first is to hack the Silverlight unit testing framework so that it will run on a Windows Phone device. Many Silverlight developers are already familiar with this, as the unit testing framework itself is advertised as “experimental” and modifications are needed to support a non-trivial number of tests. Jeff Wilcox provided one such modification for unit testing Silverlight 3 and Windows Phone 7.0.

Pro: This offers more accurate tests than the other options.
Con: Cannot be included in the build process. No IDE support.

Christofer Löf went took a completely different route. Rather than trying to push a decent testing framework into Windows Phone, he pulled the code he wanted to test out of it. By using the Portable Library Tools to implement his models and view-models he was able to run the same code under a standard unit testing framework as well as the device. Unfortunately the Portable Library doesn’t support observable collections, necessitating the use of a “ListFactory” to provide the correct type.

Pro: Can be included as part of a build cycle. May be any standard unit test framework. IDE Integration.
Con: Limited API available through Portable Library Tools

A third method is to use separate projects that share the same source code. Ideally the code will be identical, but preprocessor directives can be used to smooth over most differences. This approach makes the most sense when you will are already planning on sharing the same code with MonoTouch or Mono for Android as they would need separate builds.

Pro: Can be included as part of a build cycle. May be any standard unit test framework. IDE Integration.
Con: Longer build times. Differences between the platforms may be problematic.

iOS / MonoTouch

MonoTouch recently gained an on-device unit testing framework. And unlike the Windows Phone version, this one is officially supported. The Touch.Unit framework is essentially just a test runner with some basic IDE support in MonoDevelop. At its core is NUnitLite, a subset of the respected NUnit framework tailored to fit on limited resource devices.

Pro: This offers more accurate tests than the other options.
Con: Cannot be included in the build process. No IDE support.

Prior to the release of Touch.Unit, developers would use separate projects that share the same source code much as we see for Windows Phone. But of course in this case they would target the full Mono runtime and NUnit instead of the Microsoft alternatives.

Pro: Can be included as part of a build cycle. IDE Integration.
Con: Longer build times. Differences between the platforms may be problematic.

Android / Mono for Android

Currently the only option for unit testing Mono for Android projects is the multiple-project option. Mono for Android is a less mature product than MonoTouch, but eventually we should see it getting a comparable testing framework

LDRA, the leader in standards compliance, automated software verification, source code analysis and test tools, has created LDRA Certification Services (LCS), a division devoted to helping customers develop certification-ready products. The LCS team includes Federal Aviation Administration (FAA) Designated Engineering Representatives (DER) and safety engineering experts, who offer product certification solutions at a fixed price. They are able to offer this client-centered, risk-free approach, based on their vast experience.

The LCS team is also aligned with world-class product development and verification specialists, who use the LDRA tool suite and other tools to address critical project requirements at the highest design assurance levels. By guiding certification applicants through the compliance process, the LCS comprehensive solution provides company management confidence that their certification efforts are accurate, complete, and fully compliant.

Leading the LCS team is Todd R. White, a systems and equipment FAA DER with Level A authority on all aircraft systems and equipment for both software and airborne electronic hardware. Todd’s expertise is flanked by Marty Gasiorowski, a systems, safety, software and complex electronic hardware FAA DER, and Dr. Holly Hildreth, a safety engineer with expertise in a wide range of US and international standards, to provide certification expertise in DO-178/DO-278, DO-254, DO-297, IEC 61508, IEC 62304, ISO 26262, IEEE 1012:2004 and CENELEC EN 50128. The team is well known and highly regarded in the certification community, playing an instrumental role in crafting the RTCA DO-178C standard.

“Todd and his team have a stellar reputation in the certification community, earning recognition from certification authorities, government agencies and industry leaders alike,” confirmed Ian Hennell, LDRA Operations Director. “Their collective expertise and policy-making experience in all phases of software, hardware, system development, verification, configuration management and quality engineering assures certification applicants that industry standards will be interpreted and applied correctly.”

LDRA’s leading certification technology, developed over nearly 40 years of helping clients achieve certification readiness, provides a suite of tools fine-tuned to the needs of safety-critical markets. From requirements traceability to analysis, unit testing and validation, the LDRA tool suite supports certification objectives at all levels of design assurance. Widely-used certification templates and checklists establish standard-specific processes that allow applicants to meet management, development and verification objectives at a fixed price.

“LDRA has earned a strong reputation in the certification community by developing a suite of world-class certification tools that not only help certification applicants produce fully compliant products, but help them produce safe products, too. That is the goal. They have earned this reputation over many years of helping companies show compliance,” commented Todd R. White, FAA DER and LCS Team Lead. “It’s an honor to work in close affiliation with LDRA in this effort to empower companies with the expertise and resources needed to outline a correct plan of action that can then be carried out with confidence. Such an approach offers the most effective way to minimize certification risk and its associated cost.”

Hennell confirmed, “We look forward watching the LCS team, combined with the field-proven LDRA tool suite, aid certification applicants across industries as diverse as avionics, automotive, industrial and medical to adopt superior development processes and programming standards. With best-of-breed expertise and certification tools, companies will be brought to certification readiness using methodology that is guaranteed to reduce risk, time and development costs.”

The comprehensive LCS solution is complemented by a team of safety-critical software and electronic hardware development and verification specialists. This renowned team has over 10 years’ experience designing, building and/or testing products that meet the rigorous DO-254 criteria. Using the LDRA tool suite, this team has expanded its capabilities by bringing in top industry practitioners to develop and verify software in compliance with DO-178B/C and IEC 62304. All work is performed at a fixed price.

LCS verification and certification services can be applied across industry standards in various vertical markets. Certification services include: avionics (DO-178B/C), industrial safety (IEC 61508), automotive (ISO 26262), medical (IEC 62304), nuclear power (ieee 1012:2004), and transportation (CENELEC EN 50128) systems.

Todd R. White and other DO-178C Committee Leaders will be offering an LDRA-sponsored, 3-day training event on DO-178C entitled “Learn DO-178C from the People Who Wrote It!” on March 13–15 in Ontario, California. The training will focus on the recently published DO-178C, DO-278A, DO-248C, and Technology Supplements. Those interested in attending please visit: LearnDO-178C.com.

The full capabilities of the LDRA certifiable support packages will be demonstrated at Embedded World in Nurnberg, Germany from February 28 to March 1 in Hall 4-410. Come to the stand to see how a software project can achieve certifiable readiness through requirements traceability from requirements creation through development and validation for the certification standard you need.

QASymphony, a SaaS provider focusing on bringing intelligent software testing tools to market, announces its launch today. “The Cloud-Powered Software Development LifeCycle is beginning to be fully realized,” said Vu Lam, CEO of QASymphony. “Today developers and testers require testing tools that provide more than basic record-keeping and collaboration. They require tools that support cloud-based software development and testing processes along with agility, scalability and, most importantly, intelligence into how the software is being designed, deployed and used. Our mission is to provide these tools from our cloud-based platform.”

As a first step, QASymphony is releasing its first product, qTrace, an easy-to-use, defect capture and reporting tool that provides insight into how the software was being used. qTrace captures all steps, screens, and system information associated with a defect along with out-of-the-box integration enabling the user to submit the defect trace to any bug tracking system of choice.

qTrace provides testers an easier and faster way to report defects. In the past, software testers worked through an application to find bugs and the drudgery starts; retracing steps, doing screen captures, making notes, compiling a report, opening a defect tracking tool, and then finally submitting a ticket. Not fun, not productive, and often time resulting in defect reports that were missing key pieces of information.

With qTrace, testers simply turn on the tool as they start their testing session. qTrace runs in the background and captures every screen and every step performed during testing. qTrace then automatically generates a detailed defect report from the captured information that includes how the application was being used along with associated environmental information. While in stealth mode the company had qTrace in beta testing, and customers found that qTrace decreases defect capture time by 30-50% and decreases fix time by 10-30%.

“qTrace’s seamless integration with our bug tracker is a huge plus; combined with the ease of use, qTrace definitely helps us reduce testing cycles,” said Lakshmi Janardhan, QA manager for Compuware Corporation.

“We are really excited about getting qTrace in use across our entire testing organization,” said Gwendolyn Sanders, senior QA analyst at Silverpop. “The beta showed us that we could cut in half the time it takes to capture screens and record defects.”

“qTrace is so easy to use. It took me a few minutes to download and install, then I was able to start capturing and submitting defects right away without having to look at a user manual,” said Kevin Konishi, development program manager, Next Step Learning.

qTrace Benefits Include:

• Ease of use. With qTrace, testers use simple point-and click commands to start recordings, modify records, and publish reports.
• Automation. qTrace automates the capture of complete bug reports—including the steps that precipitated the defect’s occurrence. Testers can easily add details or modify the record, or just move forward with submitting a ticket.
• Seamless Integration. Through its integration with leading bug-tracking tools, qTrace enables testers to create new defect records and submit complete details into a bug-tracking tool—with a single mouse click. qTrace also provides an API that allows custom integration with any defect tracking tool.

•Defect reporting quality improvement. By eliminating a lot of manual, error-prone efforts, qTrace helps improve the accuracy and specificity of defect descriptions.

• Productivity boost. With qTrace, teams can spend less time on documentation, and more time on testing. Consequently, testers get better at what’s most important: validating functionality. qTrace helps turn even novice testers into proficient testing experts.
• Product development acceleration. With qTrace, application development teams can speed QA cycles, enhance collaboration and provide better information to development, so the organization can more quickly get quality products to market.

“All of us at QASymphony are passionate about software quality. We are launching the company on Valentine’s Day because we love software developers and testers, and we are confident that in time software developers and testers will come to love us as well,” said Vu Lam, QASymphony CEO. “ Our roots are in software development; we are passionate developers and testers ourselves. Having provided development and testing services to over 75 companies in the past, we understand their needs. The products we will introduce to the market will help testers and developers release high quality products faster. We believe our first product, qTrace, will become an indispensable application for them.”

HP has launched a new LoadRunner-in-the-Cloud offering that allows its partners to provide a branded offering for application performance testing in the cloud, which would help their customers deliver high-quality applications.

Qualified partners can use the on-demand service model of the cloud to deliver consumption-based performance testing of Web 2.0, AJAX and mobile and other applications to their customers using the HP LoadRunner-in-the-Cloud offering.

In addition, LoadRunner-in-the-Cloud offering complements the HP performance testing services portfolio.

The new HP offering is coupled with a partner’s cloud infrastructure to enable customers to accelerate time-to-market with access to the hardware and software resources required to conduct application performance testing anytime, anywhere and also to deliver high-quality applications with a comprehensive view of system performance that identifies and eliminates potential bottlenecks before the application is deployed in production.

The LoadRunner-in-the-Cloud offering simplifies performance testing of complex applications with HP LoadRunner’s TruClient, an offering that reduces testing cycles by cutting script time and correlation for Web 2.0 applications by as much as 50%.

Further, it also ensures cost-efficiency through subscriptions, with which customers can consume and pay for the application performance testing resources they need on a daily basis and enable HP partners to deliver high-quality applications faster.

HP Software Product and Solution Marketing senior director Matthew Morgan said organisations that do not have robust performance testing may face application delivery delays or risk performance bottlenecks if they are unable to get timely access to testing resources.

“The HP LoadRunner-in-the-Cloud offering allows partners to deliver cost-effective, on-demand access to leading performance testing in the cloud,” said Morgan.

Development and testing teams in large, data-driven organizations are constantly faced with numerous challenges when it comes to implementing changes to SOA (Service-Orientated Architecture) frameworks. Having little control over systems and databases is a way of life for most testing teams, and the reasons why is simple to explain but until now has been difficult to address.

“Developers and testers work within the constraints of large, complex infrastructures, containing numerous cross-system dependencies,” explains Huw Price, Managing Director of Grid-Tools.

“A lack of access to systems, inconsistent responses based on the same input and little control over up- and cross-stream data are all factors that impact on the quality of testing and the speed at which developers can work.”

There are tools in the market that allow users to record and playback message streams and whilst they can help to create more stable environments, they are not able to build complex data, based on rules and templates, needed to improve the quality of test cycles.

In addition, some problems cannot be solved by simply playing back data from past traffic, for example, an upstream or cross-system WSDL that has not been written yet or a response that should return a new ID each time it is called, not a recorded value.

“It became clear to us that virtualization is the route that will deliver the optimal solution,” said Huw Price, “however, although existing virtualization products offer a degree of control, they still require developers to create static data in spread sheets, or maintain complex scripts or write code.”

These are problems that can now be solved with the introduction of Intelligent Virtual Services™ (IVS,) the new SOA virtualization solution from Grid-Tools.

IVS uses a virtual service layer to replicate the behaviour of HTTP, MQ and JMS message system. This virtual layer eliminates the constraints of cross-system dependencies in traditional SOA testing. By creating a virtual service layer, IVS enables testing teams to work in a stable, isolated environment, minimising disruption and delays in waiting for data to flow downstream. The result is high quality, efficient SOA message testing and development.

IVS also allows users to create stubbed responses (also called virtual endpoints), either through echo or record and playback responses. However, where IVS differentiates itself from its competitors is by utilising the power of Grid-Tools’ Enterprise Data Masking™ and Datamaker™ solutions in combination with the powerful Portus SOA gateway from Ostia to provide fully compliant, intelligent, fit for purpose virtual responses for SOA testing and development.

“By using an IVS virtual layer, the quality, efficiency and cost-effectiveness of SOA testing will be largely improved,” says Huw Price, “We are confident that Intelligent Virtual Services™ will transform the way SOA testing is being delivered and organizations will see a significant improvement to their bottom line.”

Spirent Communications, a leader in testing wireless networks, services and devices, today announced that PCTEST Engineering Laboratory, Inc. has expanded its automated mobile device testing capabilities with the addition of several Spirent hardware and software solutions. With these new capabilities, PCTEST can now perform an even wider range of CDMA and LTE mobile device testing including multimode testing. These new competences include data throughput, data retry, Simultaneous Voice and LTE (SVLTE), InterRAT (iRAT) testing, enhanced Location Based Service (LBS) testing, Simultaneous Voice and Data (SVD) and more.

“Spirent systems have been a big part of our growth and leadership in providing device testing services over the years,” said PCTEST president Randy Ortanez. “These new expanded LTE testing capabilities offer our customers even more complete testing, all under one roof.”

“We’re proud to know that PCTEST continues to seek out and deploy Spirent solutions for their most critical testing applications,” said Spirent vice-president Hesham ElHamahmy. “The collaboration between our companies over the years has been a big part of our mutual success.”

For more information on testing LTE devices, visit http://www.spirent.com/Devices-and-Equipment/LTE_devices .

Search Engine Land have featured a great article on A/B Testing, Prices and Processes.  Titled   When A/B Testing Prices, Proceed With Caution and written by Scott Brinker the article provides a great example of A/B Testing in practice.

I had an interesting A/B testing experience over the holidays. This time, it wasn’t an A/B test that I was running, but rather an A/B test in which I was an (initially) unsuspecting participant.

It reminded me of the negative side effects that certain kinds of tests can have on customers — sometimes your best customers — and the steps that marketers should consider to mitigate those risks.

This is a cautionary tale, but I don’t want to lambast the the company running the test. They ultimately handled the situation fairly. So I will refer to them anonymously as a business software provider named Acme.

For more read When A/B Testing Prices, Proceed With Caution.

Parasoft Corporation, industry leader in automated defect prevention and detection, today introduced a new release of Parasoft Jtest: the industry’s most comprehensive Java testing solution. Parasoft Jtest automates static analysis, unit testing, coverage analysis, runtime error detection, and code review.

Parasoft is recognized by software development professionals as the leader in software development lifecycle automation. By integrating software development management, quality lifecycle management, and dev/test environment management, we help teams deliver better software faster. Since 1987, over 10,000 companies — including 58% of the Fortune 500 — have turned to Parasoft to ensure that quality software can be produced consistently and efficiently. For more information, visit the Parasoft web site and ALM Best Practices blog.

The latest release significantly extends support for a variety of secure application development guidelines and compliance reporting (e.g., for CWE-SANS, OWASP, and NIST SAMATE). The extended coding guidelines are available for desktop, pattern-based, and data flow-based execution. The extended static analysis for security analysis works hand-in-hand with Jtest’s runtime error detection, which identifies security vulnerabilities that occur as the application is exercised during any automated or manual tests.

Moreover, Jtest’s desktop analysis optimizations slashes the time and effort required to find and fix critical defects — as well as confirms their successful removal. This helps teams ensure that the risks exposed by static analysis, unit testing, and runtime error detection are remediated rapidly and efficiently. Specific capabilities introduced include continuous static analysis, one-click retesting, quick mode, debug mode, and more.

Additional highlights of Parasoft Jtest 9.2 release include:

• Enhanced Spring support with static analysis and unit testing
• Enhanced Ant/Maven plugin facilitates continuous integration and testing
• Support for Eclipse JUnit launch configurations
• Redesigned metrics reporting

Keynote Systems (NASDAQ: KEYN), the global leader in Internet and mobile cloud testing and monitoring, today announced that Google has selected Keynote Mobile Internet Testing Environment (MITE) as the heart of its new free GoMoMeter tool, helping businesses evaluate the mobile-readiness of their websites. Keynote MITE enables the GoMoMeter to show businesses how their sites look on their customers’ mobile devices. MITE also helps the tool assign a mobile-readiness score for each site, delivering results to users with consistent, near real time performance.

“We selected Keynote,” said Ryan Harms, senior producer at Mullen Advertising, “because of the support of the Keynote team and their core software’s unique ability to address our business case in a cost-effective manner. Keynote’s mobile device emulation capabilities make the service highly scalable. We’re expecting traffic in the thousands of users every day. Because a solution relying on real mobile phones would have required us to physically add more phones to grow, we didn’t think that a hardware-based solution could scale quickly and easily enough to meet our needs. Keynote’s MITE software emulation approach is infinitely scalable; it’s very easy to spin up more software profiles. Finally, Keynote has been tested at an enterprise level and meets our quality standards.”

The GoMoMeter tool is designed to allow a business person to evaluate the mobile readiness of their website. The user simply types in the site’s URL to see what their site actually looks like on a customer’s mobile device. The tool also combines information from the mobile device emulation (e.g., whether the text and images are visible without pinching or zooming), the file loading time statistics, and the answers to key questions (e.g., can buttons and links be used with a thumb?) to create a mobile website usability score. This tool is the centerpiece of Google’s new http://www.howtogomo.com/en/b/w website and educational campaign, which was developed by Mullen Advertising to introduce businesses to the benefits of mobility and help companies evaluate their site’s mobile readiness.

“As mobile device usage increases, mobile optimized websites are incredibly important to the growth initiatives of many businesses. These sites must provide an excellent experience to visitors using one of many thousands of mobile devices. Keynote MITE improves the mobile Web user experience by automating the testing of mobile websites and optimizing mobile Web content based on device-specific design and performance best practices across thousands of emulated mobile devices. By integrating MITE’s unique device emulation and rendering capabilities, via an API, into the GoMo site, Google is making it that much easier for enterprises to quickly see and experience their website as a mobile user would,” said Vik Chaudhary, vice president of product management and corporate development at Keynote. “Keynote and Google share a commitment to improving the mobile user experience, and educational tools such as the GoMoMeter are a much needed step forward in advancing awareness for the mobile challenges online enterprises face.”