Skip to content

My story with t-way testing

Last updated on February 11, 2019

When I started my Ph.D. research around 2009, my supervisor suggested working on t-way testing. The whole software testing field was new to me, although I came with a good background in benchmarking of distributed systems. But now I can say it was not really software testing rather it was benchmarking. I started by reading several books about the software engineering and then other books about software testing. Later, I understand the whole process, and I was able to implement my first draft of code to generate t-way test suites. I met a friend of mine who worked on the use of search algorithms in control engineering. I realized that could be useful in my work and after several weeks of discussion I started using particle swarm optimization (PSO) for t-way testing. I was the first who implement the PSO in t-way. However, I was not the first who use search algorithms. One other researcher used Simulated Annealing (SA). After graduation and getting my degree, I realized that, instead of t-way tasing, I also have something to say about these search algorithms. I have used the t-way testing as a case study then and I with my supervisor implemented several improvements of the search algorithms and applied those improvements into t-way testing.

We enjoyed doing this job actually, and we implemented many new variants of those algorithms by making some small changes and contributions to those search algorithms. We published several papers in that direction. In fact, in software testing in general and t-way specifically, few famous names think that they are guard and god of this field. That is not wrong actually. However, they started criticizing us, indirectly by giving tough comments on our papers and also rejecting many of our papers. In fact, there was one reason behind this behavior. They thought that we are repeating ourselves and even they were asking for new bounds in t-way testing. In fact, those reviewers missing our aim in the papers although we tried to elaborate that in the text.

We aimed to say something in the search algorithm part not the t-way testing part. The t-way testing here was just a case study. Now you may ask why we did not use the standard mathematical benchmarks in optimization methods? Well, we thought to do that also, but as far as we have a real problem why we use some theoretical problem? Using this approach, already many people cited our papers by using our improvements on the search algorithms.

In fact, these days we stopped writing those papers although we still have many things to say. But, when the editors see the t-way testing phrase, they have few choices for reviewers, and when they get our paper, they just reject it without reading.

I think the generation research of t-way testing is getting into saturation nowadays. It is very hard to publish a paper in that direction especially when the paper goes to those well-known names in this field. However, there are many things that we can say in the application of t-way testing. T-way testing is not applied very well to the real-life problems and also in the industry. So, the contribution now is in the application, not the generation.


Published inUncategorized