ISTA 2017 Highlights Day One

I was pleased to attend ISTA conference 2017 in Sofia, the capital of Bulgaria. The conference goal is to present innovations in software technologies and automation. In this article I will write my personal opinion about the lectures.

There have been 3 tracks with lectures that run simultaneously, so I choose the most interesting from my point of view. The 3 halls are named confusingly. Their names are Alpha, Beta and Panorama. Beta is the largest, Alpha the middle, and Panorama is the smallest. The organization of the event was excellent. I hope the videos and slides will be uploaded soon, so I could catch up with the lectures that  I found interesting, but I was not able to attend, due to my attendance in another track.

Day One

Innovate, automate, accelerate

Birger Thorburn told us about his recent project. He has moved from a project with release cycle of 5 years to a project that should deliver new product after 12 months. On top of that the clients were top 25 largest banks. The team was located on 3 continents.

The team managed with the tight deadlines through the means of automation. They setup continuous deployment. They used every night hundreds of virtual machines (VMs) to run the tests. The VMs were with clean state. They were up before the start of the tests, and shut down after the test execution.

It is very important to track the progress in such progress, so that you will know when you have to panic. The release 1 of the product is just the beginning, his team goal is to have a release every month. It is not important which Agile methodology you will use. Use the one that suits you best. His preferences were towards Kanban. You should automate everything.

One of his main points was that smart people will transform the future through technologies. Working with software should not make us forget that there is a physical world that needs our attention too.

Unfortunately he did not give us many details about the project and the tools that they have used to achieve the automation. He mentioned Kubernetes, Graphana/ Splunk, TerraForm, Kafka in one of his slides. The speaker was a little bit quick when deciding that there were no questions. I personally heard the disappointment of the guy next to me.

Security, Big Data and other challenges to the IoT

Martin Harizanov stated that IoT (internet of things) has been around for decades, however it was known before as “connected computers”. IoT are not only physical objects, connected to the Internet.

Security

One of the security challenges is to ensure that only authorized devices are connected to your cloud service. He mentioned HMAC verification. The users should see only their devices. Security patches should be properly installed. Devices should be updated regularly. Unfortunately not all devices could be updated at the same time. Martin mentioned FOTA. Hackers tried to find weaknesses in each of his projects, within 2 weeks of its start. Hackers goals is to turn devices into zombies, to get unauthorized access to personal info and/or denial of service.

Martin gave us an example for API weakness in high-class Chinese camera. A hacker was able to easily collect data about 300 of these cameras. User IDs were sequential, so it was easy to get one, and then to just increase the value by one. Unencrypted communication channel (HTTP) exposed in plain text the credentials of the users.

Effects of security breaches in the IoT world are very similar to the other IT sectors:

• Loss of confidence in the company;
• Financial losses;
• Personal data loss, that is not harmless;
• Legal implications.

The speaker attributes the causes of the security issues mainly to tight time-frames and poor architecture design.

There should have been protection in place so that it is not possible to get data from 300 cameras from a single IP address. You should scan the logs from time to time in order to find attack attempts.

Even banks get hacked, but you should reduce and mitigate the risk.

Big Data

Some of the big data challenges are the constantly increasing volume of data, the data quality, the data diversity, data relations, security.

You should put effort in data encryption, data anonymization, encryption, rolling code signature.

Overall nice lecture, the speaker was enthusiastic and knowledgeable.

Overcoming the diversity of smart devices

Alexander Kostadinov and Dimitar Ivanov gave us an insight of the dev perspective of IoT diversity.

2017 is with focus on monetization of IoT systems. Many small providers disappeared. Speakers gave us some nice example for IoT devices like smart kitchen, smart door and bike locks, irrigation controllers, home energy monitors, etc.

Most widely spread ways of device communication are REST, CGI and BLE.

Alexander and Dimitar mentioned problems that were not catched on simulators. It is is extremely important to have the actual device well in advance.

Some of the issues that they faced were:

• Broken devices, that cannot be reset due to lost key during update;
• Overloaded network;
• Interference when too many devices are communicating on one frequency protocol;
• Bugs in device firmware, that could be fixed slowly;
• Protocols and the devices are too much;
• Every vendor has own API;
• Even if the vendor is the same, there could be different APIs for the different devices;
• The automation is hard.

The “solution” is:

• Higher abstraction level, there are open source solutions;
• Distributed teams;
• Make the integration of new devices easy;
• Contribute to standardization discussion work-groups.

The future of computing

Laurent Bugnion made an interesting presentation. Innovation is in cloud computing. The smart devices are more and more affordable. Man will be judged by not what he knows, but by how fast he could find the answer.

Blockchain is not about crypto currencies. It is about storing info in decentralized fashion. The data cannot be changed and everybody has the same copy.

The speaker presented us several Cortana demos. He set a reminder with his voice and Cortana’s help to remember to call his father when he returns home. Which is interesting because Cortana should use his geolocation to determine when to display the reminder.

Laurent showed us several demos that used Azure services. Serverless means that you don’t have to worry about the server, not that there is no server. Somebody else takes care of the servers, you just use the services.

There was demo with generating thumbnails from pictures. Emotion API uses picture as an input and returns the probability that this picture shows certain emotion, like happiness for example.

There was demo with added floating astronaut above the audience.

Laurent showed us holographic building, he extracted some holo-pipes and the schemes for them. From my point of view, that could be of great help for future engineers.

He showed us holographic chat with HoloBeam.

It was really interesting presentation.

Testing without borders

Tania Vladimirova spoke about testing without borders. This is testing with free, proven, open source, easy, simple, portable and scalable  solution. She mentioned JFrog artifactory. I found only paid version with free trial for it. Her team use JFrog for packing the test environment (ruby gems, ready to use Docker images). They execute functional tests in parallel.

You should start the automation from SCM (source control management) system.

They use Cucumber on Ruby and Docker. Jenkins is used for the continuous delivery. Tania mentioned OpenShift, Kubernetes, Ansible and Chef, Selenium and SoapUI. You could use Zabbix for monitoring, it is an open source tool.

I was disappointed by this lecture. I was expecting a demo, but there have been only slides.

Make it visible

Nikolay Stanoev shared his experience from several years of visual testing.

They had functional tests coverage of around 80 %, but also visual issues in production like broken layout, due to content changes and regression bugs.

Selenium + Image Magick

His team wanted to have cross browser visual tests that are easy to write and maintain. They wanted to integrate it with theit existing framework. They started with Selenium and Image Magick. Their solution used free tools but was epic fail. They had too many false positives. This was a result of content changes. They have too many dynamic data (text and data changes), shiny animated elements, GIFs  and image carousels. This content was not under their control.

Their second approach was to mark the problem areas and to exclude them from the comparison. The main issue with their second attempt was the increased maintenance cost and the increased code complexity (too many ifs).

Applitools

They switched to a paid solution by Applitools.

For almost a year they sent false positives to Applitools and they provided fixes. Nikolay likes the following features:

• Many programming languages you can choose from;
• Different match options, for dynamic data they use layout comparison;
• Easily ignored regions;
• You can compare single element on the page;
• The comparison is full page, not only the visible viewport;
• You can compare floating elements, that you know are on your page, but you don’t know exactly where.

Nikolay showed us a short demo. Currently his team spend 0-5 minutes per day for maintenance of their visual tests. They have 30% reduced testing time. The team uses Applitools for bug fixing testing. They have baseline (golden) image for each browser. Nikolay found that the tool is not useful during redesigns. In such big projects he recommended to not run visual tests at all.

I recommend you seeing this presentation if you need to execute visual tests against web application.

Automating Web Security Testing

Yavor Papazov gave us an excellent  presentation on web security testing. The rapid development is often affecting the operational stability in terms of security. As general rule security mistakes do not have fast feedback. Developer can introduce security bug, that can go unnoticed for years. So there are two options to manage security:

1. Resilience instead of security. Give up security at release and work to improve it afterwards. Chaos Monkey was mentioned as a tool to test resilience.
2. Ensure security is embedded early along the software production pipeline. That leads to automating the security testing.

Security test case have negative requirements, that are harder to test that the positive. For example, evil hacker should not be able to login.

“Make it secure” is not well defined. There are projects that try to define common security weaknesses.

• OWASP
• CWEs

Mentioned tools:

• Zed Attack Proxy (OWASP ZAP)
• BDD Security (Continuum Security)
• Mittn (F-Secure)
• Gauntlt

There was a demo with Strict Transport Security header. It tells the browser to connect only under HTTPS. This is better than server redirect.

Mozilla SSLyze was mentioned as a tool for TLS testing.

XSS vulnerabilities can be automated, but not fully, as the possible test cases would be a huge number.

Unfortunately there are no ready-to-use recipes that could work for all. Yavor’s advice is to start small, as advocated by the Agile methodology. We can translate some security vulnerabilities into functional tests. Have metrics in place as a starting point. The automated security testing will be a standard in the future.

My next article is for the second conference day of ISTA 2017. Don’t miss if you found the first-day article useful.