Testing the Internet of Things presents unique challenges. For
starters, the IoT itself is a vast system of networked devices, sensors
and computing infrastructure, with potentially tens of billions of
moving parts within it by the end of this decade. Cisco has estimated
that 50 billion “things” could be connected to the IoT by 2020 – enough
for each person on the planet to have six of them to his or her name.
In addition to its unprecedented scale, the IoT is also a complex realm of crisscrossing wireless connections and long device idle times. Everything from environmental sensors and motion detectors to networked refrigerators and thermostats will be counting upon available connectivity via Wi-Fi, Bluetooth, ZigBee, 4G LTE and other technologies. At the same time, these nodes within the IoT will behave differently than many traditional devices: They may consume data mostly through “bursty” transmissions rather than in a steady stream.
Software quality assurance members tasked with performing software testing of applications and services for the IoT must be prepared for a new frontier. They must be ready to take on fresh challenges with data security, connectivity, maintenance and embedded systems that they have may not encountered much in previous projects. Let’s look at five key skills that IoT testers will need for success:
1) Comfort with conducting gray-box testing
Testing something for the IoT will often put teams in a gray-box situation, meaning that they will need to not only verify the functionality of the application (as in white-box testing) but also dive into its internal structures (as in black-box testing). This is because developers may have limited control over how their creations will ultimately be used – a similar scenario to what already happens with services such as Netflix, which can be accessed from a multitude of interfaces – and as such won’t get the transparency of white-box testing.
Success with gray-box testing depends on gathering as much information as possible about relevant APIs, third-party services, connectivity protocols, etc. This data can then be used to craft effective test cases that can cover a wide range of events and circumstances.
2) Knowledge of embedded systems
As mobile applications entered the mainstream over the past decade, testers had to become accustomed to issues that had rarely, if ever, been relevant with desktop programs. For example, they had to address touch-based interfaces, mobile connectivity fluctuations and limited battery life. The IoT will force a similar shift, in part because of its many embedded systems.
Unlike platforms like Apple iOS or Google Android, these systems aren’t known for their well-maintained operating systems and support tools. More often, they are essentially kludged together. Testers may need to work more directly with end-user interfaces and tools such as code tracers to ensure proper coverage in their tests.
3) Ability to perform effective simulation
The scale and complexity of the IoT means that many of its environments cannot be recreated in full for testing purposes. Testers have to rely instead on simulation, often with many virtual networks complementing a single physical one. This is the most efficient way to recreate the feel of hundreds or even thousands of nodes operating in tandem.
Simulators provide advantages on top of their cost-effectiveness and scalability. For example, they can accelerate the testing process for systems with long idle times by only processing interesting events, rather than the long intervals between them. Idle execution is sped up, saving time and money for testers.
4) Dealing with data spikes and bursty usage patterns
In the virtual desktop infrastructure world, there’s a phenomenon called the “boot storm,” when everyone tries to sign into their VDI solutions at the same time (usually in the morning). The result is lag and an overall poor user experience. What does this have to do with the IoT, though?
IoT application and service testers may have to take a similar situation into account when working with any software that governs areas such as utilities or transportation. What could rush hour traffic due to highway systems? Can the network handle the data spike from all the lights coming on at once in a building after an outage? These scenarios are new territory for most QA teams.
5) Data security wherewithal
The IoT has been a fixture of cyber security conversations for years now. The potential issues are well-known. As Nicholas Evans noted for Computerworld, security is sometimes an afterthought for IoT vendors. Moreover, the sheer diversity of devices, applications and services in play makes it hard to implement an effective catch-all defense strategy.
IoT testing teams must be attentive to everything from transit encryption to API integrity and use of test management add-ons for Atlassian and utilities as needed. The stakes for protecting connected cars, homes and businesses from intrusion are higher than ever with the IoT.
Thanks K Shilpa Sutar
In addition to its unprecedented scale, the IoT is also a complex realm of crisscrossing wireless connections and long device idle times. Everything from environmental sensors and motion detectors to networked refrigerators and thermostats will be counting upon available connectivity via Wi-Fi, Bluetooth, ZigBee, 4G LTE and other technologies. At the same time, these nodes within the IoT will behave differently than many traditional devices: They may consume data mostly through “bursty” transmissions rather than in a steady stream.
Software quality assurance members tasked with performing software testing of applications and services for the IoT must be prepared for a new frontier. They must be ready to take on fresh challenges with data security, connectivity, maintenance and embedded systems that they have may not encountered much in previous projects. Let’s look at five key skills that IoT testers will need for success:
1) Comfort with conducting gray-box testing
Testing something for the IoT will often put teams in a gray-box situation, meaning that they will need to not only verify the functionality of the application (as in white-box testing) but also dive into its internal structures (as in black-box testing). This is because developers may have limited control over how their creations will ultimately be used – a similar scenario to what already happens with services such as Netflix, which can be accessed from a multitude of interfaces – and as such won’t get the transparency of white-box testing.
Success with gray-box testing depends on gathering as much information as possible about relevant APIs, third-party services, connectivity protocols, etc. This data can then be used to craft effective test cases that can cover a wide range of events and circumstances.
2) Knowledge of embedded systems
As mobile applications entered the mainstream over the past decade, testers had to become accustomed to issues that had rarely, if ever, been relevant with desktop programs. For example, they had to address touch-based interfaces, mobile connectivity fluctuations and limited battery life. The IoT will force a similar shift, in part because of its many embedded systems.
Unlike platforms like Apple iOS or Google Android, these systems aren’t known for their well-maintained operating systems and support tools. More often, they are essentially kludged together. Testers may need to work more directly with end-user interfaces and tools such as code tracers to ensure proper coverage in their tests.
3) Ability to perform effective simulation
The scale and complexity of the IoT means that many of its environments cannot be recreated in full for testing purposes. Testers have to rely instead on simulation, often with many virtual networks complementing a single physical one. This is the most efficient way to recreate the feel of hundreds or even thousands of nodes operating in tandem.
Simulators provide advantages on top of their cost-effectiveness and scalability. For example, they can accelerate the testing process for systems with long idle times by only processing interesting events, rather than the long intervals between them. Idle execution is sped up, saving time and money for testers.
4) Dealing with data spikes and bursty usage patterns
In the virtual desktop infrastructure world, there’s a phenomenon called the “boot storm,” when everyone tries to sign into their VDI solutions at the same time (usually in the morning). The result is lag and an overall poor user experience. What does this have to do with the IoT, though?
IoT application and service testers may have to take a similar situation into account when working with any software that governs areas such as utilities or transportation. What could rush hour traffic due to highway systems? Can the network handle the data spike from all the lights coming on at once in a building after an outage? These scenarios are new territory for most QA teams.
5) Data security wherewithal
The IoT has been a fixture of cyber security conversations for years now. The potential issues are well-known. As Nicholas Evans noted for Computerworld, security is sometimes an afterthought for IoT vendors. Moreover, the sheer diversity of devices, applications and services in play makes it hard to implement an effective catch-all defense strategy.
IoT testing teams must be attentive to everything from transit encryption to API integrity and use of test management add-ons for Atlassian and utilities as needed. The stakes for protecting connected cars, homes and businesses from intrusion are higher than ever with the IoT.
Thanks K Shilpa Sutar
No comments:
Post a Comment