Should we trust what scientists say about food?

The event was more than an exercise in theoretical thinking.  The FSA is launching a new General Advisory Committee on Science to offer independent challenge and advice on how the FSA collects and uses scientific evidence.  The subject of the debate reflected the sort of issues that might affect GACS' advice to the agency. 

Professor Kay-Tee Khaw began the debate by exploring why it's so difficult to get accurate information about the effects of diet.  For practical and ethical reasons it is nearly impossible to carry out controlled experiments on what people eat.  Hence scientists have to reply mainly on non-experimental strategies, often dependent on observation.  She pointed out that although there's a strong positive correlation between meat consumption and colon cancer in many countries, there's a similar positive correlation between pairs of nesting storks and numbers of births in Sweden (stork pairs and human births both appear to be falling alarmingly).  Therefore correlation doesn't equal causation and can only be the first step in advancing a theory.  Some associations made by scientists in the past now seem unlikely: caffeine is no longer seen as a risk factor in pregnancy, and antioxidant vitamin supplements don't appear to reduce mortality. 

Of course some observational studies have produced strong results.  There's overwhelming evidence of a link between smoking and lung cancer (with 85% of lung cancers being caused by smoking).  Professor Khaw believes that the defining features in the studies that get it right include very large effects, rare or unusual outcomes, attention to alternative explanations, multiple research designs and confirmation from animal and human experiments.  Small experiments and the involvement of commercial interests are also likely to lead to false conclusions.  It usually takes many overlapping studies of different kinds to home in on the truth.

Professor Erik Millstone has long been interested in the effect of food processing on health: his book Food Additives from the mid 80s explored the safety of what is added in a processed diet.  He argued that 'science is not monolithic and scientists do not speak with one voice'.  Which scientists should we trust?  Millstone maintains we should look at both technical competence and alignment of social interests.  Experts are rarely neutral.  He argued that policy makers themselves may skew science by the questions they choose to ask.  'Policy judgments are concerned with the acceptability of possible risks in exchange for anticipated benefits, and those are socially variable value judgments - they are policy matters not scientific interests'.  The idea of a government led by 'pure' scientific advice is therefore a naive one, even assuming that science always produced unequivocal results.  

The way studies are framed may also be a policy decision, not a scientific one.  Some foodstuffs are considered safe after an assessment of the short-term, direct effect, not the long-term indirect effects. 

He argued that while manufacturers are allowed to suppress the results of studies that work against their interests, it is difficult for food agencies to make reliable decisions.  He called for all studies to be announced at the beginning of the research, with all being published regardless of the result.

Ben Goldacre is a medical doctor and writer of the Bad Science blog.  He said that food is a major obsession and that weak evidence is turned into certainty by the demands of the media and the public.  He noted that the public misunderstanding of science is fed by the newspapers' need for sensational copy - citing the Daily Mail as an example of a paper that wished to divide everything in the world into substances that cause cancer or cure it.  

Goldacre cited various instances where the media had referred to invented data, extrapolated inappropriately or treated lab findings as absolutely conclusive.  Goldacre now treats all media reports on health with great cynicism and said it was only by tracking back from these reports to the original data was it possible to find out which studies hold water.  He argued that there are many publics, not one, and that some can cope with a more sophisticated analysis of the facts, with the shades of meaning and qualification.  Not all though, would have access to the original journals where scientific studies were published, or would have the scientific training to understand them.  Therefore some groups of people are excluded from gaining greater information even if they want it.

Where then, do the views of all these speakers leave the Food Standards Agency?  In an ideal world, high-minded scientists, governments and businesses would work together to uncover the truth about foods, no matter what the commercial and social implications.  Level headed newspaper editors would produce headlines saying 'Very small study on six rats indicates that tomatoes might be bad for you, but don't panic or stop eating them: it may not mean anything.'  The public, mindful of the complexities of research, wouldn't ricochet from fad diet to fad diet.

It seems unlikely that human nature will change soon.  But a challenge from GACS to how the Food Standards Agency looks at science - and so frames policy - is a welcome one.  Professor Colin Blakemore will head the new committee.  He commented that since the committee was sitting inside the FSA and include many people from its current committees, some would question its independence.  But he argued that GACS would nevertheless offer a real challenge to current practice.

Perhaps the message for the consumer is that although such agencies do useful work, we cannot resign responsibility for ourselves, or assume that the best guess based on today's evidence will still look right a year or a decade from now.  While newspapers quibble over individual ingredients, we have to frame a whole, varied diet for ourselves that works over time.