Alessio Guglielmi's Research and Teaching / Politics
There are some possibilities of abuse in anonymous refereeing, because of the large irresponsibility of the referee towards the author. I received several times careless and, in a few cases, dishonest reviews. I know it happens to others very frequently, so I wonder whether there is anything we can do to limit this phenomenon.
I believe that referees should accept that their reviews are published on the web site of the author, and possibly answered in public (like book critics do for books, say). Alternatively, or in addition, referees could sign their reviews.
Publishing reviews serves two purposes: 1) Careful, positive reviews can persuade others into reading the paper; this way the work of the referee has more value. 2) Questionably negative reviews can be answered and commented in public, and this, of course, places some responsibility on the referee.
Circumstances where this is undesirable might occur, and anonymity should always be perfectly acceptable. Instead, refusing review publication on the web should be more the exception rather than the rule, but again this right should be granted.
There is a danger, in principle: referees might be induced into being too merciful, and this is not good, we do need high standards. It is a matter of finding the right balance, and I think the web provides for a way of making the refereeing system better. If more and more people adhere to some principles of `responsible refereeing´, I believe we will all benefit from a healthier reviewing.
I decided, then, to sign my reviews, apart from exceptional circumstances, and to give explicit permission to publish and comment them.
This whole thing of formally refereeing papers seems to me a thing of the past, when access to printed paper was coveted and limited; the web changes the picture. This is the way it will work when the inevitable resistance is overcome. One puts a paper on the web, another reads it and finds it good and correct, and writes a short comment, or suggests changes, etc. Others find the paper useful and correct, and will cite it and comment it, and so on. The bad stuff will automatically be filtered out and forgotten, without any need of testing the self-control of anybody.
Bertrand Meyer and Doron Zeilberger say it better.
I believe that there is a problem with the conference system in computer science, at least for the part of it that concerns theoretical computer science.
Theoretical computer science is a branch of mathematics that happens to be practised, mostly, in computer science institutions. Contrary to the tradition in mathematics, most theoretical computer scientists publish their work in conference proceedings, and only to a much lesser extent in journals. This practice is strongly encouraged by the fact that conference publications count towards promotion and tenure.
There are many good aspects to conferences, and especially, of course, their role in facilitating communication and circulation of ideas. However, the following negative aspects, in my opinion, largely outweigh the positive ones:
These problems would not be serious if conferences were only a means of exchanging novel and perhaps still immature ideas. However, hiring and career advancement depend on conferences, and this confers them disproportionate impact. We are building a system without solid foundations for the production and vetting of scientific results. Of course, good institutions and good scientists still produce good science, and good conferences have good papers, but what about the average quality? In my opinion, it is pretty low in computer science, and the conference system has a big role in that.
I believe that there also is a subtler and worse problem associated with the peculiar conference system of theoretical computer science. There are many conferences, and we have developed a very competitive culture, to the point that you cannot be a top gun if you don't have papers at each and every top conference. So, we have some of the most brilliant minds constantly working at small problems, those that fit the conference format and that can be ready on time for the next competition. The same do the less brilliant, or those that work in unfashionable areas, and that compete for second-tier conferences. All these people would probably do better and be more creative if they felt free to work on less incremental research.
It is very sad that the first thing we have to tell our students is that academic life is a rat race, and that either they publish or they perish.
So, what can we do? The origin of the problem is not in conferences, of course, but in the fact that the associated publications count for career. Many agree with that, but then many also say that the problem is in the system as a whole, or in the funding mechanisms, or in the culture, and so on. Many also say that everything is going to change soon anyway, because the Internet changes the economics of publication, and so the traditional ways are all doomed. Whatever. The point is that, at least for now and for some of us, refusing conferences can be not only socially responsible and an exemplary behaviour, but even advantageous to our research. It's a pity that some of the very best among us don't even notice this, because they are, of course, very successful, and so gratified, with the present system.
Since 1995, I refuse to be involved with conferences, but with the exceptions that I note below. So, in principle, I don't send papers to conferences and I refuse invitations to speak at them and to referee for them. This applies only, of course, to conferences with published proceedings, which are the conferences that play a role in career advancement. The conferences where people go only to meet colleagues and exchange ideas are, of course, very welcome for me. This policy has probably slowed down my career, but it hasn't killed me, and it has served me well, because I could concentrate on tough problems and then develop an ambitious and broad research program.
Of course, as everybody, I have responsibilities towards my employers and colleagues, and so I have to correct possible excesses. I do this as follows.
Communication of scientific results (so that they have an `impact´, as the managers say) is an important part of research. My employer, and eventually the tax payer, have the right to see their money get into more communication than just journal papers. To compensate for my lack of participation in conferences, I organise several meetings in my field, like workshops and summer schools, and I take great care in communicating my findings as effectively as I can.
I also have to be careful not to stifle the career of my junior colleagues, especially PhD students. For them, I make an exception to my rule of not sending papers to conferences: I do occasionally send conferences papers that are coauthored by colleagues who need a permanent job, provided that they present the paper. Of course, this should not be a free ride, because the conference system costs in terms of refereeing. So, the deal is that they give back the conferences the same amount of refereeing that they receive. This seems to me a fair compromise to everybody.
I am interested in hearing comments (both positive and negative) about my position on the matter: just drop me an email.
These are some pieces of the debate that is going on in theoretical computer science about the conference issue mentioned above: