Do men and women differ in the way they produce scientific output? If so, does this influence the way we judge their research performance? I, with colleagues, raised these two questions in a recent study that followed the track records of a cohort of British and Australian biologists over 15 years.

The questions are politically loaded. In 2005, a storm of outrage broke over a speech given by then-Harvard President Lawrence Summers to the National Bureau of Economic Research in Massachusetts. He argued that the paucity of women at higher levels in scientific academia reflected both less commitment to, and ability for, science as a career. Needless to say, many critics pointed out that women scientists face, if not outright discrimination, then at least many potential obstacles in their careers that men do not.

Alarmingly, numerous studies have shown that, in terms of productivity (number of scientific papers produced), men far outstrip women. In our study men were on average 40% more productive. More disturbingly, we found that this discrepancy in productivity appears surprisingly early in their careers: in the second year after their first publication.

These patterns might appear to lend support to Lawrence Summers’ arguments. Crucially, though, we found that in terms of publication rate per year, women do achieve the same productivity as men (which argues against the Summers hypothesis), except that they trail about 2 or 3 years behind. The picture is therefore one of women being slow starters who subsequently have to ‘play catch-up’. The real problem is that, at any one moment in time, female scientists will appear to be less productive than men of apparently equal experience. Consequently, they struggle in comparison with men to get job appointments, promotion and funding. For example, the percentage success rate of Australian Research Council grants applications over the last six years significantly favours men (28% of applications successful, versus 23% for women).

In this light, the introduction of the Research Quality Framework in Australia poses a real danger that the task of improving the representation of women in senior academic posts will be made more difficult if departments and faculties are keeping a wary eye on the consequences for their RQF assessments.

Of course, quantity of research is only one measure of research performance. Quality of research, as the name of the RQF implies, is surely as important. We wondered if women scientists were concentrating on quality of their science instead of quantity. What we found was that quantity of research is a very pervasive measure, one that is difficult to disentangle from quality.

Take an obvious measure of quality of research: the average number of citations per paper. Surprisingly, this measure is strongly correlated with quantity of output. Scientists who have a greater number of papers get their work cited disproportionately more often, perhaps because their names become more familiar to the scientific community and hence they are more likely to be cited. Tellingly, when we compared men and women with equal numbers of publications, the women’s papers were cited, on average, 20% more often. However, the overall higher productivity of men disguises this difference in quality.

Likewise, the currently popular h-index, which essentially counts the number of high-impact papers that a person has produced, is another measure designed to reflect quality that in fact mirrors quantity. In that case, the more papers you have then clearly the more likely you are to have a number of high-quality papers. For this reason a suggested plan of the RQF to identify the four highest quality publications of individual researchers will also favour scientists with more papers.

What should we be doing to ameliorate this situation?

As a long-term aim, we should address the early lag in publication rates shown by women. Mentoring schemes for women, often offered at later stages in a career (postdoctoral, or junior lecturer), should be in place much earlier as part of their PhD training.

In the short-term, however, we need to ensure that any measures of quality that we apply to assess research performance must reflect just that, and not in fact quantity by proxy (especially if the latter is already being explicitly measured as part of the assessment). When measures of quality covertly reflect quantity, women scientists will continue to be compared unfavourably to their male counterparts.

Matthew Symonds is a post-doctoral research fellow in the Department of Zoology at the University of Melbourne. The research described here was published in the online journal PLoS ONE