Testing the Internet of Things presents unique challenges. For
starters, the IoT itself is a vast system of networked devices, sensors
and computing infrastructure, with potentially tens of billions of
moving parts within it by the end of this decade. Cisco has estimated
that 50 billion “things” could be connected to the IoT by 2020 – enough
for each person on the planet to have six of them to his or her name.
In addition to its unprecedented scale, the IoT is also a complex
realm of crisscrossing wireless connections and long device idle times.
Everything from environmental sensors and motion detectors to networked
refrigerators and thermostats will be counting upon available
connectivity via Wi-Fi, Bluetooth, ZigBee, 4G LTE and other
technologies. At the same time, these nodes within the IoT will behave
differently than many traditional devices: They may consume data mostly
through “bursty” transmissions rather than in a steady stream.
Software quality assurance members tasked
with performing software testing of applications and services for the
IoT must be prepared for a new frontier. They must be ready to take on
fresh challenges with data security, connectivity, maintenance and
embedded systems that they have may not encountered much in previous
projects. Let’s look at five key skills that IoT testers will need for
success:
1) Comfort with conducting gray-box testing
Testing something for the IoT will often put teams in a gray-box
situation, meaning that they will need to not only verify the
functionality of the application (as in white-box testing) but also dive
into its internal structures (as in black-box testing). This is because
developers may have limited control over how their creations will
ultimately be used – a similar scenario to what already happens with
services such as Netflix, which can be accessed from a multitude of
interfaces – and as such won’t get the transparency of white-box
testing.
Success with gray-box testing depends on gathering as much
information as possible about relevant APIs, third-party services,
connectivity protocols, etc. This data can then be used to craft
effective test cases that can cover a wide range of events and
circumstances.
2) Knowledge of embedded systems
As mobile applications entered the mainstream over the past decade,
testers had to become accustomed to issues that had rarely, if ever,
been relevant with desktop programs. For example, they had to address
touch-based interfaces, mobile connectivity fluctuations and limited
battery life. The IoT will force a similar shift, in part because of its
many embedded systems.
Unlike platforms like Apple iOS or Google Android, these systems
aren’t known for their well-maintained operating systems and support
tools. More often, they are essentially kludged together. Testers may
need to work more directly with end-user interfaces and tools such as
code tracers to ensure proper coverage in their tests.
3) Ability to perform effective simulation
The scale and complexity of the IoT means that many of its
environments cannot be recreated in full for testing purposes. Testers
have to rely instead on simulation, often with many virtual networks
complementing a single physical one. This is the most efficient way to
recreate the feel of hundreds or even thousands of nodes operating in
tandem.
Simulators provide advantages on top of their cost-effectiveness and
scalability. For example, they can accelerate the testing process for
systems with long idle times by only processing interesting events,
rather than the long intervals between them. Idle execution is sped up,
saving time and money for testers.
4) Dealing with data spikes and bursty usage patterns
In the virtual desktop infrastructure world, there’s a phenomenon
called the “boot storm,” when everyone tries to sign into their VDI
solutions at the same time (usually in the morning). The result is lag
and an overall poor user experience. What does this have to do with the
IoT, though?
IoT application and service testers may have to take a similar
situation into account when working with any software that governs areas
such as utilities or transportation. What could rush hour traffic due
to highway systems? Can the network handle the data spike from all the
lights coming on at once in a building after an outage? These scenarios
are new territory for most QA teams.
5) Data security wherewithal
The IoT has been a fixture of cyber security conversations for years
now. The potential issues are well-known. As Nicholas Evans noted for
Computerworld, security is sometimes an afterthought for IoT vendors.
Moreover, the sheer diversity of devices, applications and services in
play makes it hard to implement an effective catch-all defense strategy.
IoT testing teams must be attentive to everything from transit encryption to API integrity and use of
test management add-ons for Atlassian and
utilities as needed. The stakes for protecting connected cars, homes
and businesses from intrusion are higher than ever with the IoT.
Thanks K Shilpa Sutar
