Skip to content

My Reflection on Search-based Software Engineering

Last updated on April 26, 2022

Note: This blog post represents my opinion and cannot be considered a scientific document. However, it can be considered a discussion subject. Since it is my opinion and it has a high potential not to be true. If you disagree after reading it, just forget about it. I used this blog to write my thoughts about some topics. This is my current opinion; I may change it over time if I learn new things. But feel free to contact me if you want to discuss it. 

It is well-known among other computer science communities that the software engineering community is outstanding in creating new names, terms, and acronyms to be used in publication and presentation. This is seen as a “Reinventing the wheel” concept. I think that is true in many communities, but I also agree that the software engineering community is overdoing it. Search-based Software Engineering (SBSE) is an example of this case. When I started my Ph.D. study, I searched for a new catchy and sexy topic. I was accepted as part of the software engineering program. So, I knew my field, but not my topic. During my starting research, I felt that many ideas are just rephrasing old ideas but with new catchy names. One of the newly published papers during that time was on Search-based Software Engineering (SBSE). The name had all the characteristics of “Reinventing the wheel.” Simply, it is on using optimization methods in the software engineering fields. But from my search in the software engineering field, I immediately knew that this is possibly another new catchy name that will stand for a while, and perhaps it will be good to finish my Ph.D. on this topic. I picked some new optimization methods and used them in software testing, part of the software engineering field. I optimized the test cases by searching for effective test cases using some optimization methods. I got new and nice results and published results in high-quality journals and conferences, and Walla .. I got my Ph.D. 

During that time, I studied in-depth optimization algorithms in terms of type, parameters, algorithms, and other aspects. Since the time of my Ph.D. was limited and the focus was not that broad, I kept this for a while until I finished my study. When I finished my Ph.D., I started exploring optimization methods. But I also kept in mind this SBSE. The community had grown, and many people had followed the trend by that time. But, I always asked myself, the optimization and search algorithm were there, and software engineering was already there, so why did we create this term? The optimization and search algorithms were used a long time ago to solve software engineering problems. So, what was new?  

What made the SBSE more trendy was the “Reinventing the wheel” habit that happened to the optimization community. During that time, several well-established and mature optimization algorithms in the research community have been used for a while, like Genetic algorithm, Simulated Annealing, Particle Swarm, Tabu Search, and several other algorithms. But these were not enough for that community to publish new papers. The trend of creating new nature-inspired optimization algorithms started. So, every day, there was a newly published optimization algorithm in a journal. At the time, it was very catchy. You can imagine that now, we have an optimization algorithm for each animal name. This trend also made the SBSE even more viral. So, I started using some of these newly invented optimization algorithms in my work. I got some new results. However, I found that they are not always good. That is due to the randomness of these algorithms. Then I used another algorithm, and I got new results. Then I used two more and got new results. But, I did not find an algorithm that gives good results all the time. I also did not find any algorithm that could be the winner for all cases—Mmmm.. something strange here. 

So, the trend in the SBSE was following the trend of the optimization community. For example, new algorithm X is published in the optimization community, then immediately after that, a paper appeared in the SBSE community, using X in solving Y problem. Y here is a software engineering problem. And, of course, you should add some SBSE explanation in the paper. And, here it is, you have a good paper to be published. But then, this became a recipe for many papers. You just need to change X or Y in that sentence, and it becomes a new paper. I even followed this trend like any other community member. However, I went back to the original idea and found that by a good parameter tuning of the conventional and old optimization algorithms, you can even get the same results that you get with the new optimization algorithms. I came back to the optimization algorithms and found that the algorithms look different. Still, in fact, they are the same and even the changes presented are not leading to any new conceptual change in the strategy in general. Of course, what I’m saying is just my observation, but there are also some scientific studies and results showing that recently. Going back to those newly developed optimization papers, it looks like they used the same experimental setting and the same implementation, and most of them were published by the same set of authors. So, to be realistic, if you have implemented one of them in a code and used the benchmarks, you may be able to develop another new algorithm in one day or even less than one day. What you need is just a good imagination to pick an animal and be able to express the behavior of that animal in mathematical expression, for example, when the animal is searching for food. Then, use those optimization expressions in the old code and write a new paper. The authors of these papers got thousands of citations because of this habit and the SBSE community. I guess nowadays, that community got to my point that this is not useful and in its maturity level. We don’t need any new optimization algorithms. Still, we need to improve the classical algorithms to be adaptive to the new kind of applications, like making them self-adaptive or distributed. 

Coming back to the Search-based Software Engineering, there are some other issues apart from the naming. One of the problems I found in the papers published within the SBSE area is that most of the authors of these papers have a very shallow understanding of the optimization field. I have read many papers in very good journals but with many conceptual mistakes in the optimization algorithms. Part of this problem is also because the reviewers, who are also from the SBSE community, are more interested in the software engineering problem than the implementation of the optimization. A few years ago, I visited some well-known conferences with the SBSE theme, and I found that many of the attendants and presenters don’t have that good knowledge of optimization, including some well-cited people in the field. But, for the SBSE community, it is the same problem as the optimization community but with a different style. When the optimization reaches some saturation state, the SBSE community invested in finding new problems to be solved by the optimization algorithms. In my opinion, this is better than what happened in the optimization community. 

The people in the SBSE community did not stop at this point, and it seems they liked that trend creation. They invented a lot of new sub-names, like search-based software testing, Search-based energy consumption, learning-based software engineering, and search-based xxxxx. However, not everybody can make a trend, just like social media trends. It depends on the author’s name, how many followers he/she has, how many conferences he visits per year, and how he/she does the marketing. 

In my humble opinion, there are some problems behind this trend creation and “Reinventing the wheel” habit. First, there is a high demand and greediness for publication as a metric of job evaluation in academia. Writing a paper with these trends and showing that you are following a new topic has a higher chance of being accepted, especially when you cite some of those newly published papers to show that you are up to date and following the trend. Second, lack of topics, in which it is easier to follow the trend rather than having some new ideas. Third, the power and influence of some members of those communities. They are hired by good universities that can pay to visit many conferences and then use those conferences to make connections with other people for marketing. 

Bestoun S. Ahmed


Published inUncategorized

Comments are closed.