Environmental decisions to approve a development, protect a threatened species or reduce contamination in a stream almost always involve expert opinions. We need them because the time, resources and data to assess the risks are usually unavailable or the circumstances are unique. Decision-makers accept expert opinions because they believe experts have valuable, specialised knowledge.

One of the problems with this system is that when experts assume a position of authority, reinforced by professional membership, qualifications and status, it can intimidate people who want to examine expert judgments critically. In court proceedings, experts are tested by the opinions of other experts, by hypothetical questions or by proof that an expert expressed a different opinion on a former occasion. But experts are not questioned in most management situations, leading to a culture of "technical control".

Many people, including some risk analysts and experts, hold a flawed view that knowledge held by suitably qualified experts is a clear, objectively defined "truth", while public understanding is fuzzy, over-simplified or corrupt. Differences between lay and expert knowledge depend on the type of problem, the person applying that knowledge and the context in which that knowledge is applied.

It is not possible to delineate sharply between expert and lay knowledge. Decisions about who is an expert and what constitutes expertise should not be decided by arbitrary rules or qualifications, but by procedures that assess a claim of expertise.

We've known for several decades, through the work of social scientists and psychologists, that expert judgments are often unreliable and biased. Judgments about "facts" are coloured by conflicts of interest. Experts are routinely overconfident in their own ability to estimate facts and are sensitive to a host of psychological idiosyncrasies, including framing, anchoring, social context, novelty, visibility and the potential for "outrage".

Given the necessity of expert opinions, the frailties of expert judgments and the potential for misleadingly optimistic estimates, much can be done to improve the situation. Essentially there are three options - to use tests to distinguish better between experts and non-experts, to train experts, and to use procedures that anticipate and deal with biases.

The prospect of testing the reliability of experts raises challenging questions. Who sets and administers the tests? Which elements of expertise should the tests examine? How does one overcome the fact that experts are likely to be reluctant to be tested? Many of these problems have been solved in a few disciplines, notably, engineering risk assessment in Europe. However, these solutions are almost never employed in environmental decision-making.

Training and testing experts generally improve their performance. Both depend on case studies, experiments, hypothetical scenarios and simulations that provide "facts" that the performance of experts can be calibrated against. Like testing, training of environmental experts is almost never employed in practice.

Structured procedures to elicit judgments can mitigate some of the most important and pervasive psychological biases. Such approaches can ameliorate dominant individuals in group deliberations, and minimise the effects of anchoring, hindsight bias and over-confidence. Yet, like training and testing, they are rarely deployed in real situations.

What counts as expertise depends on context. If experts are tested, then expertise from all domains (including information from people without formal qualifications) may be included. If we provide feedback systems so that experts can learn from their mistakes, and if we employ methods that avoid the worst psychological pitfalls, the quality of the information we glean from experts will improve substantially. It seems professionally negligent not to do so.

Prof Mark Burgman is Director of the Australian Centre of Excellence for Risk Analysis in the University of Melbourne's School of Botany.