I was pleased to attend ISTA conference 2017 in Sofia, the capital of Bulgaria. The conference goal is to present innovations in software technologies and automation. In this article I will write my personal opinion about the lectures.

There were three tracks with lectures running simultaneously, so I chose the ones that were most interesting from my point of view. The three halls were named confusingly: Alpha, Beta, and Panorama. Beta was the largest, Alpha the middle-sized, and Panorama the smallest. The event was very well organized. I hoped the videos and slides would be uploaded soon so I could catch up with lectures I found interesting but could not attend because I was in another track.

Day One

Innovate, automate, accelerate

Birger Thorburn told us about his recent project. He moved from a project with a five-year release cycle to one that had to deliver a new product after 12 months. On top of that, the clients were among the 25 largest banks, and the team was located on three continents.

The team managed the tight deadlines through automation. They set up continuous deployment and used hundreds of virtual machines every night to run tests. The VMs started in a clean state before the tests and shut down after execution.

It is very important to track progress so you know when you have to panic. The first release was only the beginning; the team’s goal was to release every month. It does not matter which Agile methodology you use—use the one that suits you best. His preference was Kanban. You should automate everything.

One of his main points was that smart people will transform the future through technology. Working with software should not make us forget that there is a physical world that needs our attention too.

Unfortunately, he did not give many details about the project or the tools used to achieve automation. He mentioned Kubernetes, Grafana/Splunk, Terraform, and Kafka on one slide. The speaker moved quickly to conclude that there were no questions. I personally heard the disappointment of the person next to me.

Security, Big Data and Other Challenges to the IoT

Martin Harizanov stated that the Internet of Things (IoT) has been around for decades, although it was previously known as “connected computers.” IoT is not limited to physical objects connected to the Internet.

Security

One security challenge is ensuring that only authorized devices connect to your cloud service. He mentioned HMAC verification. Users should see only their own devices. Security patches should be installed properly, and devices should be updated regularly. Unfortunately, not all devices can be updated at the same time. Martin mentioned FOTA. Hackers tried to find weaknesses in each of his projects within two weeks of launch. Their goals were to turn devices into zombies, gain unauthorized access to personal information, or cause denial of service.

Martin gave an example of an API weakness in a high-class Chinese camera. A hacker could easily collect data about 300 cameras because user IDs were sequential; it was easy to get one ID and increment it. An unencrypted HTTP communication channel exposed users’ credentials in plain text.

The effects of security breaches in IoT are similar to other IT sectors:

• Loss of confidence in the company
• Financial losses
• Loss of personal data, which is not harmless
• Legal implications

The speaker attributed security issues mainly to tight time frames and poor architecture design. There should have been protection to prevent data from 300 cameras being obtained from a single IP address. Scan logs from time to time to find attack attempts.

Even banks get hacked, but you should reduce and mitigate the risk.

Big Data

Some big-data challenges are constantly increasing data volume, data quality, data diversity, data relationships, and security.

You should put effort into data encryption, data anonymization, and rolling-code signatures. Overall, it was a nice lecture, and the speaker was enthusiastic and knowledgeable.

Overcoming the Diversity of Smart Devices

Alexander Kostadinov and Dimitar Ivanov gave us insight into the developer’s perspective on IoT diversity.

2017 was focused on monetization of IoT systems. Many small providers disappeared. The speakers gave examples of IoT devices such as smart kitchens, smart door and bike locks, irrigation controllers, and home energy monitors.

The most widely used device-communication methods are REST, CGI, and BLE.

Alexander and Dimitar mentioned problems that were not caught in simulators. It is extremely important to have the actual device well in advance. Some issues they faced included:

• Broken devices that could not be reset because a key was lost during an update
• An overloaded network
• Interference when too many devices communicate on one frequency protocol
• Bugs in device firmware that could be fixed slowly
• Too many protocols and devices
• Every vendor having its own API
• Different APIs even for devices from the same vendor
• Difficult automation

Their proposed solutions were:

• Use a higher abstraction level; open-source solutions are available.
• Use distributed teams.
• Make integration of new devices easy.
• Contribute to standardization discussion groups.

The Future of Computing

Laurent Bugnion gave an interesting presentation. Innovation is in cloud computing. Smart devices are becoming more affordable. People will be judged not by what they know, but by how quickly they can find an answer.

Blockchain is not about cryptocurrencies. It is about storing information in a decentralized way. The data cannot be changed, and everyone has the same copy.

The speaker presented several Cortana demos. He set a reminder by voice to call his father when he returned home. This was interesting because Cortana had to use his geolocation to decide when to display the reminder.

Laurent showed several demos using Azure services. “Serverless” means you do not have to worry about the server, not that there is no server. Someone else takes care of the servers, and you use the services.

One demo generated thumbnails from pictures. Emotion API takes a picture as input and returns the probability that it shows a particular emotion, such as happiness. Another demo added a floating astronaut above the audience.

Laurent showed a holographic building, extracted some holographic pipes and their schematics, and demonstrated a holographic chat with HoloBeam. From my point of view, this could be a great help for future engineers. It was a very interesting presentation.

Testing without Borders

Tania Vladimirova spoke about testing without borders: a free, proven, open-source, easy, simple, portable, and scalable solution. She mentioned JFrog Artifactory. I found only a paid version with a free trial. Her team uses JFrog to package the test environment (Ruby gems and ready-to-use Docker images) and executes functional tests in parallel.

You should start automation from the SCM (source control management) system.

They use Cucumber with Ruby and Docker. Jenkins is used for continuous delivery. Tania mentioned OpenShift, Kubernetes, Ansible, Chef, Selenium, and SoapUI. Zabbix is an open-source monitoring tool.

I was disappointed by this lecture. I expected a demo, but there were only slides.

Make It Visible

Nikolay Stanoev shared his experience from several years of visual testing. The team had around 80% functional-test coverage but still experienced production visual issues, such as broken layouts caused by content changes and regression bugs.

Selenium + ImageMagick

The team wanted cross-browser visual tests that were easy to write and maintain and could integrate with its existing framework. It started with Selenium and ImageMagick. The free-tool solution was an epic fail because there were too many false positives, caused by changing content, dynamic text and data, shiny animated elements, GIFs, and image carousels—content the team did not control.

The second approach marked problem areas and excluded them from comparison. Its main issues were higher maintenance cost and more complex code (too many if statements).

Applitools

The team switched to the paid Applitools solution. For almost a year, they sent false positives to Applitools, and the company provided fixes. Nikolay liked these features:

• A choice of many programming languages
• Different match options; for dynamic data, they use layout comparison
• Easily ignored regions
• Comparison of a single page element
• Full-page comparison, not only the visible viewport
• Comparison of floating elements whose exact position is unknown

Nikolay showed a short demo. The team now spends 0–5 minutes per day maintaining visual tests and reduced testing time by 30%. It uses Applitools for bug-fixing tests and keeps a baseline (golden) image for each browser. Nikolay found that the tool is not useful during redesigns; for large redesign projects, he recommended not running visual tests at all.

I recommend seeing this presentation if you need to execute visual tests against a web application.

Automating Web Security Testing

Yavor Papazov gave an excellent presentation on web security testing. Rapid development often affects operational stability in terms of security. Security mistakes generally do not receive fast feedback: a developer can introduce a security bug that goes unnoticed for years. There are two ways to manage security:

1. Resilience instead of security. Give up security at release and work to improve it afterward. Chaos Monkey was mentioned as a tool for testing resilience.
2. Embed security early in the software production pipeline. This leads to automated security testing.

Security test cases have negative requirements, which are harder to test than positive requirements. For example, an evil hacker should not be able to log in.

“Make it secure” is not well defined. Several projects try to define common security weaknesses:

• OWASP
• Common Weakness Enumeration (CWE)

Mentioned tools included:

• Zed Attack Proxy (OWASP ZAP)
• BDD Security (Continuum Security)
• Mittn (F-Secure)
• Gauntlt

There was a demo of the Strict Transport Security header. It tells the browser to connect only using HTTPS, which is better than a server redirect. Mozilla SSLyze was mentioned as a TLS-testing tool.

XSS vulnerabilities can be automated, but not completely, because the number of possible test cases would be huge.

Unfortunately, there are no ready-to-use recipes that work for everyone. Yavor’s advice is to start small, as Agile methodology advocates. Some security vulnerabilities can be translated into functional tests. Have metrics in place as a starting point. Automated security testing will be standard in the future.

My next article is about the second conference day of ISTA 2017. Don’t miss it if you found the first-day article useful.