Sunday, June 12, 2011

Genetic testing and potential harm: DTC or trust me I’m a doctor? Recently at a couple of conferences (European Human Genetics conference and Consumer Genetics Conf) there have been various speakers questioning DTC genetics and calling for all health related personal genetics to be delivered through medical practitioners. I argued in the past that unregulated tests delivered through practitioners actually have the potential for more harm, not less. By coincidence last week some discrepancies in a DTC and a via MD test were pointed out to me – and they seem topical.
Breast feeding has many benefits one of which appears to be increased IQ scores – however not all studies agree, some indicating that results may be confounded by maternal intelligence (see Wikipedia). Sometimes inconsistencies in associating an action with an outcome can be resolved by looking at genetic variation (which tends to increase the error bars when not accounted for).
So in 2007 some headlines were made when a study was published by Caspi’s group (PNAS, open access) reporting that there is an increase in IQ but only in infants who were not GG for the FADS2 rs174575 SNP. It seemed like a good study, 2 cohorts whose location could not have been farther apart and nearly 3,000 children tested. The numbers begin to get much smaller though when they are dissected as only approx 8% were homozygous GG:

Interesting results – but just one study and there were some doubts expressed at the time (e.g. “they have a measure of maternal IQ but don't directly include it in the published multiple regression suggests that they tried it, but didn't like the results”)
These are preliminary results but a genetic test is was being offered to medical practitioners to enable “informed decisions” to be made (they don’t cite the study but it must be Caspi). They say:
Genetic testing performed by Existence Genetics enables parents to have access to this extremely useful information about their newborn. Although it may be ideal for most mothers to breastfeed, many are in situations that make the decision not so simple. For example, some women take medications that prevent them from breastfeeding, and others experience pain, poor milk production, and other difficulties.
If the baby doesn’t have this gene then breastfeeding appears to have no effect upon IQ.
Empowered by this groundbreaking research, Existence Genetics now provides women, pediatricians, wet nurses, and lactation consultants with exclusive genetic testing services that will allow them to make a more informed decision about breastfeeding.
Infants who have a specific version of the FADS2 gene will have an increase of approximately 7 IQ points, whereas infants without the variant won’t experience any IQ boost from breastfeeding.
Clearly the only influence that this result could have would be to make it easier for mothers not to breastfeed if there is no IQ benefit. Mothers of FADS2 GG babies will be told that breastfeeding will not increase IQ, there is no difference.
Rule No.1: do not base a genetic test on one study, even if it does look compelling. Break this rule at your peril.
Heres why:
Steer CD, Davey Smith G, Emmett PM, Hibbeln JR, Golding J (2010) FADS2 Polymorphisms Modify the Effect of Breastfeeding on Child IQ. PLoS ONE 5(7): e11570.
In a larger study Steer et al attempted replication in 5,394 children. They were not able to reproduce the results, in fact they reported that breastfed GG children performed better by 5.8 IQ points. GG children actually showed the greatest difference.
And another study:

Martin NW, Benyamin B, Hansell NK, Montgomery GW, Martin NG, Wright MJ, Bates TC. Cognitive function in adolescence: testing for interactions between breast-feeding and FADS2 polymorphisms. J Am Acad Child Adolesc Psychiatry. 2011 Jan;50(1):55-62.e4. Epub 2010 Dec 3.
They also were unable to repeat the Caspi study – unadjusted data showed a benefit if breastfeeding across all genotypes although they also say that when adjusted for sex, SES, paternal and maternal education, and birth weight the significance disappeared – they support the hypothesis that the breastfeeding/IQ association “reflects variation in parental cognitive ability” – in any case there was no FADS2 GG effect.
Based on the follow-up studies there is no support for any personalised advice based on the FADS2 gene – it cannot be used as a reason not to breastfeed and the largest study actually showed increased benefits for IQ in the group that would be advised that not breastfeeding makes no difference
This same doubtful information is also delivered by 23andme: just one study and no mention of the two larger studies. It is though announced as a “preliminary research report”.
I am disappointed that 23andme have not caught the other 2 papers – do they not have an army of researchers keeping up to date? It’s an enormously emotive subject and thus very important, it suggests that they need to improve their literature update methods and I hope they will – try using the “alert me when this article is cited” feature common to almost all journals.
This is does not particularly look good for either of the companies, but there is a big difference – one is delivered by the healthcare practitioner as “useful information” to help you decide to not breastfeed, the other is presented as results of a study. Neither are justified but I would say that the potential for a harmful impact is increased in one of the situations rather than the other. Which has the potential for more harm? Even if you are of the opinion that both are equally harmful, it certainly does not support the claims that because DTC could be harmful it should be banned and all delivered through medical practitioners because that would protect the patient. My view is that DTC or MD is not the problem – transparency is.
I suppose FADS2 is an appropriate name at least
Caspi A, Williams B, Kim-Cohen J, Craig IW, Milne BJ, Poulton R, Schalkwyk LC, Taylor A, Werts H, & Moffitt TE (2007). Moderation of breastfeeding effects on the IQ by genetic variation in fatty acid metabolism. Proceedings of the National Academy of Sciences of the United States of America, 104 (47), 18860-5 PMID: 17984066

Monday, April 25, 2011

If You Want to Practice Evidence Based Medicine then… Use the Evidence

Travelling on a train to Scotland, all for the first time:

Clinical researcher says:  "Look the cows in Scotland are black!".
Physicist replies: "No. Some of the cows in Scotland are black"
Mathematician says:  "No, actually in Scotland there is at least one cow black on one side"
I’ve written about folic acid, homocysteine and various clinical trials of nutrients in the past. I just want to revisit it briefly because I think there are some seriously wrong messages emerging from the conclusions. For an excellent comment on evidence based recommendations see Colby Vorland.

Homocysteine (Hcy) is associated with CVD, may or may not be causal but the evidence is good (Wald et al, BMJ, 2002). Folic acid (Vitamin B9) in reasonable amounts will lower homocysteine  but clinical trials have been carried out, mostly showing not much benefit as far a CVD is concerned. A recent meta-analysis of 8 trials (total 37,485 individuals) seems to confirm this: “Dietary supplementation with folic acid to lower homocysteine levels had no significant effects within 5 years on cardiovascular events”.

BUT… all the studies were in ill people (mean age 65yrs), 7/8 with heart disease or stroke and 1 study on patients with kidney disease:

So a fair conclusion is that people who have already had a heart attack, stroke or some other sort of vascular disease may not benefit from homocysteine lowering – e.g. there was no reduction in second heart attacks, at least over the short term (mean 5 yrs).
Pity, but not so unexpected, after all vitamins are not medicines to be used when one becomes ill at 65 years old – vitamins are for life.
There are also several possibilities why Hcy lowering had apparently no effect on the patients, and also Wald et al point out that aspirin use may mask the effect of homocysteine lowering – it was interesting to see that they felt the need to underline what should be obvious:
“The fact that most of the trials included patients with pre-existing cardiovascular disease leaves open the possibility that folic acid has a useful role in the primary prevention of ischaemic heart disease, where aspirin is generally not used, but not in secondary prevention, where it is routine.”

So what do Clarke et al conclude? It’s starts off OK:
“The present meta-analysis has demonstrated that lowering homocysteine levels by an average of 25% (about 3 µmol/L) for an average of 5 years has no significant effect on the incidence of major vascular events during the scheduled treatment period.”
(They just missed out the aged ill patients bit)

They go on to discuss how observational and epidemiological studies associate homocysteine with cardiovascular disease and stroke (but do not point out that these studies were in the general population, not a diseased subset) and finally:
“One-third of adults in the United States and one-quarter of those in the United Kingdom report taking daily multivitamin supplements containing folic acid. The present meta-analysis, however, found no evidence that routine use of folic acid for 5 years has any material effect on cardiovascular or non cardiovascular events in the North American and European populations studied.”

These are two sentences from different contexts entirely – the results of the meta-analysis apply only to the study group and have NO bearing on the folic acid and homocysteine as far as primary prevention is concerned. The reviewers really should not have let that pass.

There is also a commentary on this article in the same issue which goes even further in the wrong direction. It begins discussing the role of vitamins in health and prevention then discusses the impact of the meta-analysis:
  • Clarke and colleagues definitively answers most of the questions about the efficacy of B-vitamin therapy.
  • There were no trends toward benefit  in any of the primary outcomes overall or in the prespecified subgroups
  • This is a cautionary tale for all who are searching for therapies to prevent disease in the general population.
But as the commentary says there is a “strong scientific rationale” for Hcy lowering and “consistent observational data from high quality cohort studies with more than 10 years of prospective follow-up confirmed the outcomes suggested by hypotheses arising from laboratory and animal data.”
This all remains true and has not been touched by any of the clinical trials but the author goes on:
“However, randomized trials of vitamin therapy with folate, vitamin B6, vitamin B12…demonstrated that none of these therapies are effective for preventing cardiovascular disease…in the general population.”
This is simply not true, the trials did not study the general population and have NO relevance at all for primary prevention. There’s more:
“As trusted counselors with an ethical mandate to first do no harm, we physicians should keep this history in mind as we advise our patients about the risks and benefits of dietary supplements… Vitamins taken in excess of the dose required to prevent deficiency states have not improved  not improved our patients’ health and may harm them. We should recommend therapies to prevent disease in healthy patients only when randomized trials unequivocally demonstrate that net benefits outweigh net harms, and we should continue to emphasize the importance of a nutritious diet, regular physical activity, and no smoking as the best ways to optimize health.”

Couple of things: a) to not recommend is also a recommendation and b) how does the author know about the benefits of physical activity & nutritious diet and possible harms of smoking? All the evidence is from epidemiology and observation – they have not been confirmed by randomized trials!
Maybe the cows in Scotland are black.

Clarke R, Halsey J, Lewington S, Lonn E, Armitage J, Manson JE, Bønaa KH, Spence JD, Nygård O, Jamison R, Gaziano JM, Guarino P, Bennett D, Mir F, Peto R, Collins R, & B-Vitamin Treatment Trialists' Collaboration (2010). Effects of lowering homocysteine levels with B vitamins on cardiovascular disease, cancer, and cause-specific mortality: Meta-analysis of 8 randomized trials involving 37 485 individuals. Archives of internal medicine, 170 (18), 1622-31 PMID: 20937919

Monday, January 17, 2011

Breast feeding: Personal genetics is in it’s infancy…

There is a lot in the new today about breast feeding – should it be exclusive for at least 6 months (WHO) or should it include some earlier introduction of solid foods (BMJ article)? A lot of the reporting is confusing the issue, claiming or implying that the BMJ authors are saying the 6 months is too long (and pointing out that they have been consultants to the baby food industry). The BMJ authors are not saying that, they simply say that it might be prudent to introduce gradually some solid foods, while still breast feeding before 6 months.

There is a useful account at the Marion Nestle blog and I left a comment there which I have expanded upon here. This could be an area where genetic testing can help – blanket advice is never the best for all but confusion (as we have now) is even worse. I blogged about celiac disease and genetic testing a while ago and part of the post addressed exactly the issue of breastfeeding and weaning.

...babies differ! Probably, very likely, one type of advice is not applicable to every situation, but we don't yet have much idea of how to select.

The choice here is not breastfeeding vs. infant formula, it should be made very clear that the evidence suggesting adding solid foods before 6 months does NOT also suggest stopping breastfeeding. Precisely the opposite. The ESPGHAN (European Society for Pediatric Gastroenterology, Hepatology, and Nutrition) issued some precise advice targeted especially at infants with possible celiac disease risk (i.e. a genetic predisposition): For further details see

It is prudent to avoid both early (<4 months) and late (>or=7 months) introduction of gluten, and to introduce gluten gradually while the infant is still breast-fed, inasmuch as this may reduce the risk of celiac disease, type 1 diabetes mellitus, and wheat allergy.

Using genetics to give targeted advice is in it's infancy (excuse me!) - but could be useful in this case. If the universal advice were to suggest introducing solid foods between 4-6 months, there is a danger that it could lead to more frequent introduction actually before 4 months, which according to the evidence is even more risky for allergies and autoimmune diseases. So in this perspective the WHO advice could be considered to be wise, but there are exceptions.

I’m not so sure I understand the SACN logic though:

Currently available evidence on the timing of introduction of gluten into the infant diet and subsequent risk of coeliac disease and T1DM is insufficient to support recommendations about the appropriate timing of introduction of gluten into the infant diet for either the general population or high-risk sub-populations. However, there is evidence suggesting that not being breastfed at the time when gluten is introduced into the diet is associated with an increased risk of subsequently developing coeliac disease.

Source: Draft SACN/COT Statement on the Timing of Introduction of Gluten into the Infant’s Diet

I understand why earlier introduction of gluten may not be the best advice for the general population, but for high-risk sub-populations, I’m not so sure. The evidence is not water tight but that which does exist has been reproducible and indicates that 4-6 month introduction of gluten (while still breastfeeding) is the lower risk option. There are no studies that contradict this and none that suggest that there are more benefits than risks by waiting until after 6 months in high risk subpopulations. There are some clinical trials underway (e.g. PreventCD) which may help, but until then, the benefits and risks calculation suggests following the ESPGHAN advice, see above.

Thursday, December 16, 2010

The Great Health Data Deficit: Are Environmental causes for Disease a Mirage?

Ever since I was a child the message has been eat your greens (we got caned at primary school if we didn’t), you need to exercise, carrots will make you see in the dark, an apple a day keeps the doctor away. It continues today with the “5 a day”, all the healthy food pyramids, anti-junk foods, and so on. Yes the environment can be dangerous, it can kill us.

There are some indisputable examples, a bullet in the head for example, or getting hit by a car is an environmental impact that almost always results in injury and death. There are others on which we more or less agree, like smoking is harmful (evidence is very strong, although it has never actually been proven, who knows, it could be the regular movement of hand to mouth that causes all the damage, we need to do the proper controlled trial…)

For the rest we have always sort of somehow known that eating badly can cause disease – it seemed so obvious, and all the little research studies just confirmed what we knew and lead to the rise of the “big” health and big-organic industries.

But a strange thing has been happening, now that the results of the very large scale experiments are arriving, we are seeing something unexpected. Just a few examples:

What’s going on? – the scientists are now starting to argue with each other about where this “missing causality” must be.

[insert here a few hundred lines of sciency reasonable arguments. Make some valid points about the extremist denialists who say things like junk food is nutritionally useless – show the nutritional value of the Big Mac: protein and important essential fats, lycopene (in the ketchup), fibre (in the carton). Point out that despite all the doom and gloom life expectancy in all the “Western diet” countries is actually the highest it has ever been… and so on]

Then come to the clincher: Just recently the “health bandwagon” scientists have finally admitted that the last 50 years of their research has been useless and new research models are required if we are ever going to find the “missing causality” and the “missing prevention”.

But I say it is time to stop this waste of resources, their call is just an excuse to keep that bandwagon rollin’ along – the conclusion is obvious to anyone who does not have a vested interest:

It all means that genes must be the entire cause of ill health, i.e. junk food, pollution, lack of exercise, etc. do not have any impact whatsoever. We believe that if people live right, agricultur­e and therefore the planet will be more or less irrelevant and our genes will get us in the end, whatever we do

Some may disagree with this, in fact they might find it to be a rather silly conclusion

I do

PS I anyone is wondering what I am going on about please see AND

Sunday, November 28, 2010

Personal Genetics & Utility: Round 2 – Mind the EGAPP
Yesterday I wrote about the false Family History vs. Personal Genetics battle, today I look at the old chestnut of traditional risk factors. There seems to be a lot of fear among some professions that personal genetics is attempting to take over their jobs – it’s been like this from the beginning mainly due to misunderstanding (wilful or otherwise) exactly what personal genetics is and what it’s role in healthcare can be.

The latest salvo is from the EGAPP Working group who published their assessment of genetics vs. traditional risk factors (TRF) in cardiovascular disease risk. They looked at the 9p21 variant as well as 57 other variants in 28 genes associated with CVD and they sought to document

“the extent to which genomic profiling alters CVD risk estimation, alone and in combination with traditional risk factors, and the extent to which risk reclassification improves health outcomes”.
Some conclusions from EGAPP:
  • The magnitude of net health benefit from use of any of these tests alone or in combination is negligible.
  • The EWG discourages clinical use unless further evidence supports improved clinical outcomes.
  • the overall certainty of net health benefit is deemed “Low.”
  • the estimated additional benefit from adding genomic markers to traditional risk factors was found to be negligible.
  • Traditional risk factors such as those used in the Framingham Risk Scores have an advantage in clinical screening and risk assessment strategies because they measure the actual targets for therapy
  • To add value, genomic testing should lead to better outcomes than those achievable by assessment and treatment of traditional risk factors alone.
  • To be useful, genomic testing should provide demonstrable improvement on the predictive value of TRFs.

Fine, fine, fine, all correct and proven, but all missing the point completely. It does not matter that the genetics did not add anything, even with the legendary 9p21 variant. Why should personal genetics be thought of as a replacement for traditional risk factors? EGAPP in it’s narrow scope is correct, but as usual the “negligible benefits” etc. will be, actually are being, quoted widely to trivialise personal genetics, just as the family history study was used to consign genetics to irrelevance.

The world moves on and nothing changes. In the early days, almost 10 years ago, it was the same, the genetic risk had to be “over and above” traditional risk factors. But why? What is expected of genes, are they supposed to possess some transcendent quality so that some sort of independent risk factor emerges from a genetic profile? Or is it that genes code for proteins that function in the various pathways, the perturbation of which can lead to metabolic problems (the traditional risk factors) and eventually disease?

I get told off for car metaphors but here goes. Driving along in the rain, hit the brakes, skid, crash. Skidding is a risk factor for crashing, I can try to reverse the skid, it might work, but I would rather avoid it in the first place by driving better in the rain (at least until pharma comes up with the anti-lock brakes pill).

The aim of personal genetics is to prolong health. High blood pressure, low bone mineral density, arterial plaques, etc., are not present in healthy people. They might be useful indicators in predicting disease, they might be useful values to put into the Framingham calculator, but they are best avoided in the first place.

All this is obvious – so why is it that genetics is compared so frequently to classical risk factors? It’s not a surprise that they don’t contribute more, why should they? Genetic variation does not have this magic “over and above” quality. But it is there from birth, it is there even in healthy people. This was mentioned in an earlier influential paper comparing SNPs & risk factors in type 2 diabetes, the authors claimed that the genetics added little to nothing but did add as a by the way:

“Although genetic information appeared to be useful when only factors known in youth were considered, genetic information in the context of risk factors measured in adulthood did not help to refine the prediction of diabetes risk” (Meigs et al)


So it was only useful in youth, in healthy people. Dammit. When people are actually ill the traditional risk factors win hands down. Of course. So the contribution of genetics is “negligible”. No use for genetics in healthcare yet.
But what about the poor healthy people who want to stay that way? No family history for anything I particular. Normal BMI, fat mass, lipids, blood pressure, glucose, insulin, HbA1c, and so on… Will regular medicine and TRF testing still win? No, it can’t. I’m not saying that genetics will definitely win, but it’s certainly favourite, at least it has the possibility of scoring where regular medicine does not.

EGAPP is probably correct, not ready for use in the clinic – at least not in the clinics that most of us are familiar with. But this is not the same as not ready for use. Most doctors I have spoken to (many) want genetics to be a simple test that classifies risk; high, medium or low, and tells them what to do. This is a reasonable desire and fits in with the way most of them work – a few minutes per patient, clear decision making advice required. They have no time for a long interpretation and explanation of small risk changes, up or down, how to ameliorate raised risks in the long term through diet & lifestyle, etc. It’s not their job, mostly. But if “CVD is a public health care concern” (EGAPP) it needs a public health care approach and if genetics is going to be involved it will not be as a replacement for conventional risk factors but will be incorporated into healthcare long before conventional risk factors even begin to raise their ugly heads.

This is where the research should be going: proper assessments of personal genetics vs. standard health information with healthy people. Is genetics better than classical risk factors in healthy people? (of course this is just the same as asking “is genetics better than NOTHING” – which is exactly the right question). I’m expecting that 23andme will be exploring just this – they have the money, the skills, the database and the experience, and of course some business interest (as everybody does, including MDs and clinical geneticists). But I hope EGAPP will do it too.

It’s not genetics vs. regular healthcare, it’s when, where and how to use genetics in healthcare.

PS – The PHG foundation has a nice report on the Genetics and Public Policy Centre survey.

“A random sample of 1,048 US customers of the three major companies offering personal genomics DTC (23andMe, deCODEme and Navigenics) were surveyed online between June 2009 and March 2010…58% said they learned information that would help improve their health, and as a result of testing, 34% said they were being more careful about their diet and 14% were exercising more…This study provides long-overdue evidence that consumers are satisfied with DTC genetic testing services, and are generally able to interpret their results. It also indicates that there may be direct health benefits resulting from the tests in terms of behaviour modification…this survey indicates not only the absence of harm caused by DTC genetic testing services, but also the possibility of benefits.”


Early days, these were early adopters driven by some reason to take the tests, but a promising start.

Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Working Group (2010). Recommendations from the EGAPP Working Group: Genomic profiling to assess cardiovascular risk to improve cardiovascular health. Genetics in medicine : official journal of the American College of Medical Genetics PMID: 21042222

Thursday, November 11, 2010

Personal Genetics has a Family History of getting beaten up

Over the last few days personal genetics has come in for a bit of a bashing, first it was knocked out by family history then it was clearly nailed into the coffin by traditional risk factors.

Also have a look at Genesherpa’s blog for some more putting the boot in here and here.

Update: Nov 12th, here is the 23andMe blog on the subject

But what is behind the hype and the headlines? In this post I will look at family history and deal with EGAPP in the next.

The latest attack was sparked by a press release of some work presented at the ASHG. We don’t know too much because all we have is an abstract and a video but we have enough. FH was compared to the Navigenics genetic profile (PGS). From the abstract

“None of the 3 hereditary prostate cancer subjects were assessed as high risk on PGS. Based on FHRA, 10 subjects had hereditary breast cancer risk and PGS only identified 1 as high risk (K=0.12). None of the 9 hereditary colon cancer subjects were high risk on PGS.”

Sounds awful and this was the general message in the press, but it was not really so bad, not bad at all in fact. Why? Beacause Navigenics do not include the rare highly penetrant mutations that cause hereditary cancer, it concentrates on the common versions (no BRCA and no MMR genes), e.g. hereditary no polyposis colorectal cancer (HNPCC) makes up only about 3-5% of colorectal cancers. These are the cancers that FH picked up but which Navigenics missed, maybe because it was not looking for them?? (Maybe Navi should include them, maybe they don’t for regulatory/FDA reasons, but that’s a different issue).

What would really be much more informative would be to compare FH and genetics for the remaining 95-97% of cancers.

To be fair to Dr Eng, although I believe the study is flawed, she did say thatused in concert with family health history assessments, those tests could become more effective and accurate

We can only really conclude that FH was better than Navigenics at finding something that FH was looking for but Navi was not. If Navi included the BRCA and MMR genes then maybe they would have at least been on the same playing field and playing the same game. (This BTW is not a correct conclusion from ABC News: Family History Better at Predicting Disease Risk Than Screening)

In any case it’s a false battle, why choose one over the other? Although proper FH is not so straightforward, it requires a lot of effort on the part of the patient, maybe weeks to contact and collect info from all the relatives – it’s not a 20 min questionnaire, there is no doubt that it is useful when available.

Family history measures genetics to a certain extent and it measures environment as well, but it’s not a full genetic analysis and of course will not detect all genetic risks, it cannot. Genetic profiles say nothing about environment. So what does this tell us? Don’t go looking for 100% concordance because you won’t find it, but while there is overlap there are also independent contributions to be made by both – they are complementary…

The Eng study was not helpful at all in comparing FH and genetics for the vast majority of the common complex diseases, it could not be with only 44 subjects. It would be interesting and useful to do a similar but much wider study on CVD or diabetes, not to see which is best but to show how and where they are complementary and how to get the best out of both.

Thursday, November 4, 2010

Nutrigenetics–a little bit of history, but no miracles

Reading The $1,000 Genome by Kevin Davies, as expected it’s a fascinating story and right at the beginning in Chapter 1 there was something that I liked. The first personal genome to be sequenced and interpreted was that of Jim Watson (Craig Venter was first but no interpretation). Davies describes the presentation of Watson’s genome to the man himself and reports that the sequencing was performed by 454 and the interpretation was handled by the team directed by Richard Gibbs of the Baylor Genome Center.

Watson’s genome inventory, for example, revealed 310 genes with likely mutations and 23 with known disease causing mutations, increasing his risk for cancer and heart disease. The Baylor team recommended that he should take folic acid and other vitamins and minimize his exposure to sunlight, particularly during his daily tennis matches. p19

So there you have it, the first advice based on the first interpretation of a human genome sequence was nutrigenetic!

But then I read later in the book about Davies’ experiences with Sciona (actually I read this first, I started reading from the index expecting to see Sciona there, remembering that several years previously the author had contacted me about trying out our test - his review is here and in the book (p. 141):

The report soberly recommended that I should cut back on alcohol and caffeine, eat more cruciferous vegetables, and exercise more. "Brilliant, I thought, I've known that for years!"


Still the ensuing dietary recommendations – increase my intake of folic acid and omega-3 fatty acids – would be standard medical advice from any family physician. In a few cases, specific gene variants prompted more personalized dietary advice in the form of recommended vitamin and antioxidant supplements. This couldn’t hurt, but would they actually do a body good? A Newsweek cover story on the nutrigenomics fad said it best: “Some people will be advised to eat broccoli, while others will be told to eat…even more broccoli”.

Later in the book on page 204 Davies quotes John Sulston

British HGP leader and Nobel prize winner Sir John Sulston said “Nutrigenomics is a very easy scam. Not only is the advice useless…worse, some companies are associated with the companies that will sell you the dietary supplements”. Sulston’s advice was simply to grow your own vegetables.

So Baylor what did you tell Jim???

Well it’s easy to understand the contrasting reactions, from Baylor actually giving Jim Watson nutrigenetic advice, through Kevin Davies’ reasonable but slightly sceptical review to the outright condemnation of Sulston. In my opinion Sulston is right, wrong and impractical all in one go. It is unfortunately an easy scam and there have been many scammers, many still exist, many more are on the way. I would estimate that about 90% of the offerings then and now are rubbish, either scams or just through ignorance. Genetic based nutritional advice is not necessarily useless though, not all of it, the Baylor group presumably would agree with that. Grow your own vegetables is good advice but maybe not so practical for many…

Nutrition has a big problem – it’s the home of many scams, false promises, snake oil, disreputable companies, poisonous ingredients, no standardisation, poor research, overblown claims, and on and on and on. It’s a problem for us all though because inside all the mess of exploitation there is the serious side, the only key we really have for preventative healthcare. There is constant talk about how we need to change the current situation from curing disease to prevention otherwise society will collapse under the burden of obesity and diabetes, heart disease, cancer, etc. But what is preventative healthcare? Is it prescribing statins, aspirin, metformin? No, of course not, it’s boring nutrition and lifestyle where personal genetics really do have a role to play

The Kevin Davies reaction to nutrigenetics is quite common, the advice seems the same as normal healthy eating advice, and it’s broccoli or more broccoli, etc, etc, etc,. But what do you expect? Really, what do you expect? Of course it’s going to be similar to the usual advice – it’s not going to tell you that you should live on a diet of beetroot juice and walnuts while another person needs to eat rabbit and pears. It’s going to say a bit more of this and a bit less of that – there is no magic. It’s looks very similar to what your doctor would say. Yes it does, similar, not the same. Little differences make big differences in the long term. A size 8.5 shoe looks very similar to a size 9 and may feel OK in the morning, but will be hurting by the afternoon. A few calories too many per day will not make a difference except that 30 years later 14 kg have appeared.

That’s nutrition for you and you need to accept it. It’s boring but it’s the best tool we have for preventative healthcare. If you expect more from nutrigenetics, if you expect a load of zing in the advice unfortunately there are many companies out there who will satisfy your need. Why are there so many scammers? Because there are so many miracle addicts desperate for a fix.

Unfortunately the hype around genetics, the great promises made in 2000 around the human genome (although I must say that these were mainly from the politicians) lead to great expectations, so some unexciting nutritional advice is seen as worthless and a waste of money. Ironically this is the sort of expectation that drives the sales of the genetic tests which offer nutritional miracles in a bag of pills and herbs – it’s not a surprise that the company which was labelled fraudulent by the GAO earlier this year, the one selling the bag of herbal panacea, is the company that is making the most money. Boring nutritional advice is not enough is it, surely there must be more than that… yes there is, but don’t rely on it to really work…

Maybe one day there will be harmless pills for preventing all known diseases whatever we do to ourselves, for now though all that nutrigenetics can offer you is a slightly better fitting shoe – anyone offering more is not to be trusted.

Monday, September 6, 2010

The murky side of physician prescribed LDTs

Apparently the LDT community are not too happy with DTC genomics for having stirred up the regulatory hornets nest, well maybe that’s not such a bad thing. I have argued previously that physician prescribed tests can be more dangerous and more necessary of regulation than DTC and below is a good case for the argument.

First a huge huge thanks to @laikas for a) her excellent beautifully detailed and analytical posts on the CFS virus story (XMRV & MRV: here, here and here) and b) for specifically asking me and others to comment. I had been aware of the controversy but no more than that so Laika’s request made me read more – my first inclination was that it reminded me of the helicobacter story, scepticism followed by acceptance, cures and nobel prizes, but reading further it unfortunately looks more like the MMR / Autism debacle which was Laika’s comparison.

I’m not going to discuss the various findings here, no point as Laika has done that far better – what I do want to talk about is the commercial test that was put on the market as soon as the paper was published – it’s not a pretty story.

1. Lombardi et al of the Whittemore Peterson Institute (WPI) published in Science. They report the detection of a retrovirus, XMRV, in 68 of 101 CFS patients (67%) and 8 of 218 (3.7%) healthy controls -

2. In the Science paper there is a “Note added in proof”: V.C.L. [Lombardi] is operations manager of Viral Immune Pathologies Laboratory, which is in negotiations with the Whittemore Peterson Institute to offer a dianostic test for XMRV.

3. In the same month began offering a $650 test for the XMRV – “essentially 100% accurate…”

4. Four independent follow-up studies failed to find the virus and a 5th found a virus, not the XMRV but related MLV sequences were detected (all detailed here by Laika)

5. WPI group now say that they also find similar variants

6. Vipdx update their test so it now detects “other human MLV-related viruses” (so now essentially even more 100% accurate…)

7. Some things were missed out on the Science “note added in proof”, but they are in Laika’s blog posts, such as:

“Furthermore there is an intimate link between WPI and VIP Dx, both housed in Reno. Vip DX is licensed by WPI to provide the XMRV-test. links to the same site as, for Vip Dx is the new name of the former RedLabs.

Interestingly Lombardi (the first author of the paper) co-founded Redlabs USA Inc. and served as the Director of Operations at Redlabs, Harvey Whittemore owns 100% of VIP Dx, and was the company President until this year and Mikovits is the Vice President of VIP Dx. (ME-forum). They didn’t disclose this in the Science paper.”

So that’s it – basically an unregulated clinical test is being offered to a very vulnerable and exploitable group based on 1 paper on a small isolated sample, with no independent confirmation…on the contrary…and some unfortunate conflicts of interest that were not reported when and where they should have been. They say all profits go back into research but that is irrelevant and meaningless (what are profits? What is left after paying everyone? Also the sales will increase the value of the company they own). But even without the COI – what would happen if 23andMe et al offered a “100% accurate” genetic test for CFS risk (or autism, yes, WPI are getting XMRV into that as well…ouch, just got hit by another bandwagon).

What exactly is wrong:

  • No independent confirmation
  • No demonstration of human-human infection
  • It’s a hypothesis that the viral presence is causal (it could be there, if it is, due to reduced immunity in CFS)
  • Test positive what do you do? Answer: of course some doctors are prescribing anti-retrovirals already
  • Some patients will naturally feel better even if by placebo, leading to testimonials of success
  • There will be alarms about infecting family members to drive sales (and fear). In fact this already goes on, by the senior author (Mikovitz) who says in an email to an individual: “To be clear..I do think even if you tested negative now that you are likely still infected with XMRV or its closest cousin..”

In retrospect this last sentence in the Science paper has an interesting tone:

“Finally, it is worth noting that 3.7% of the healthy donors in our study tested positive for XMRV sequences. This suggests that several million Americans may be infected with a retrovirus of as yet unknown pathogenic potential”.


  • Via Twitter: @Vansteenwinckel: yet another commercial test for #xmrv ? pfff #mecfs
  • Via Carlitos, who comments below: 1st International Workshop on XMRV. Tuesday and Wednesday, September 7 & 8, 2010, Bethesda, Washington, DC – please go to Carlitos’ blog for more details, it will also be partially videocast

Monday, August 9, 2010

FDA – Personal Genetics: Is it safe? It’s a marathon, man…

It’s nearly 10 years now and still there is no clarity about the position of personal genetics in the regulatory framework. Maybe that’s going to change soon with the FDA activity and the recently published HGC Principles. It would be good to get it settled one way or another, the uncertainly doesn’t help anyone except those who exploit it to exploit the gullible.

Some elements:

  • DTC vs. DTMD (via physician) – I will argue that DTMD is actually higher risk and needs closer scrutiny
  • Is it medicine? I think this question is a waste of time, it will not be resolved, the definition is too broad, medicine is practised everywhere by everyone - if I take my son’s temperature, put a plaster on a cut or administer medication I am practising medicine.
  • What is the FDA duty bound to do and what will they decide?
    • No regulation – more or less the current situation
    • Tight regulation – medium/high risk requiring pre market approval (PMA)
    • Somewhere in between – the “light touch”

The questions are what regulation, oversight and quality controls are necessary (and possible) to get the maximum benefit, for all

What is a medical device?

SEC. 201. [21 U.S.C. 321]
(h) The term "device" (except when used in paragraph (n) of this section and in sections 301(i), 403(f), 502(c), and 602(c)) means an instrument, apparatus, implement, machine, contrivance, implant, in vitro reagent, or other similar or related article, including any component, part, or accessory, which is--

(1) recognized in the official National Formulary, or the United States Pharmacopeia, or any supplement to them,

(2) intended for use in the diagnosis of disease or other conditions, or in the cure, mitigation, treatment, or prevention of disease, in man or other animals, or

(3) intended to affect the structure or any function of the body of man or other animals, and which does not achieve its primary intended purposes through chemical action within or on the body of man or other animals and which is not dependent upon being metabolized for the achievement of its primary intended purposes.

This unfortunately is very broad, “prevention of disease”. That can apply to soap and a toothbrush and certainly applies to any publication that contains any form of health advice (if you stay healthy it is because you are preventing disease, right?). This is so broad that what it really boils down to is what the FDA decide. In 2007 they decided it was not, now they say it is. According to the definition it always has been – so this is very much a decision of the FDA, it is something they could, or could have avoided.

What are the FDA saying?

In their letters to 23andMe and others they said:

The 23andMe Personal Genome Service™ is a device under section 201(h) of the Federal Food, Drug, and Cosmetic Act (the Act), 21 U.S.C. 321(h) because it is intended for use in the diagnosis of disease or other conditions or in the cure, mitigation, treatment, or prevention of disease, or is intended to affect the structure or function of the body.

A few days ago Alberto Gutierrez and Elizabeth Mansfield of the FDA gave a revealing (taped!) interview to Mary Carmichael as part of her intriguing DNA Dilemma series in Newsweek here are some relevant snippets (whole interviewinterview plus comments, see also Daniel MacArthur’s post on the subject):

AG (to Mary Carmichael):  The law requires us to clear devices or approve devices before they go into the marketplace when they make medical claims…Now clearly the claims they are making are medical claims [re 23andMe]

AG: what is the risk of an undetected false result, what would be the medical action taken, and whether there’s high risk there or not… there might be some claims there that we may consider to be PMA-like [pre market approval]

EM: they have a variety of claims that span our normal risk classification. On a claim-by-claim basis, there are some that might be low, there are some that might be moderate, there are some that might be high

MC: So, would a company need to be approved just to provide a raw SNP list to people?
EM: They would if they made medical claims about that data. If they don’t make any medical claims about that data, then they’re free to provide information as far as we’re concerned.


1. The “law requires”, they have no choice if a device makes medical claims.
2. FDA have decided that they are making medical claims and are to be regulated as devices
3. The type of regulation will depend on risk (class I, II or III)
4. Just providing genetic results is not a medical device

While many, including me, would prefer to have the FDA step back and allow a system of self-regulation, market forces and transparency (with mandatory genetic testing registry) to control the field it looks like the FDA have crossed the Rubicon with (1) and (2). This makes risk classification very important as it determines the regulation (see here) and whether expensive and time consuming pre-market approvals are required. Note that there is no distinction by the FDA on DTC vs. DTMD, a device is a device, but it would affect the risk.

The green light to the genotyping data is helpful because it means we can focus just on the actual interpretation and for this I am going to look at parts of three different services:

1. WebMD a medical website written for an audience that includes general public. It includes gene variant related information which would be relevant and personal to an individual who has already been genotyped.

2. 23andMe – DTC personal genetics service from genotyping to interpretation

3. Existence Genetics – DTMD personal genetics.

Note that like 23andMe “WebMD does not provide medical advice, diagnosis or treatment.” But they do say things like “Your 23andMe results can help you make more informed decisions about your health” or “Better information. Better Health”. So they do (both) expect you to make decisions based on the information presented on the website – is it safe?

Common diseases where current genetic knowledge is low impact, e.g. Heart Attack:

1. WebMD has a large section on Heart Disease including “Tips for Reducing Heart Disease Risk”. There is also an article on the SNP analysed by 23andMe - 9p21

2. 23andMe: “20.9 out of 100 men of European ethnicity who share Greg Mendel's genotype will get Heart Attack between the ages of 40 and 79 (average is 21.2 out of 100)”  in the disease risk overview it says “your risk is 20.9%” and describes it as a typical risk (and is .99x lower risk). The advice is general, very much like webMD – plenty of background about the role of genetics and environment with advice on prevention and keeping healthy

3. Existence Genetics. Much less info available but some sample pages for heart attack among others:

a. Your lifetime risk = 61%,
b. General lifetime risk = 32%
c. Equal to 90% increased risk
d. It says that this disease is very important to my health and wellness and that preventative measures can be taken and lists some specific interventions, what to monitor and also how my genetics may affect medication.

All are services intended for the “prevention of disease”. In my opinion all are low risk (I would actually say no risk given current knowledge of gene impact) and would not merit any more than class I classification. Risk is classified, if I understand what AG was saying (not 100% here!) on

a) What action a person might take
b) What would be the consequence of a false positive result
c) What would be the consequence of a false negative result

Common disease - Potential Risks:

For WebMD false positive or false negative would be applied to the correctness of the advice. For the genetic companies there are additional possibilities of error: the genotyping, the software delivering the interpretation based on the genotype, the interpretation itself. It’s hard to see how an individual can come to any harm by following the advice on either of the consumer services and for EG the actions will be mediated via a medical professional (who though needs to be confident that the EG service itself is correct). For now a false result is not a risk, the predictive impact of genetics on heart disease is too low for any serious consequences. A false positive may lead to extra attention to diet & lifestyle and false negative will leave the situation unchanged – as long as it is clear that typical or reduced risk does not mean NO risk, and that in fact for these common diseases, with what we know now, the lifetime risk for anyone is always going to be significant. (If the impact of the genes is so low and uncertain, what’s the point of the service? Doesn’t matter, that’s for the buyer to decide and has no bearing on risk or FDA classification).

Serious diseases where genetics have v. high impact, e,g. cancer involving BRCA mutations:

WebMDHaving one first-degree relative with breast cancer approximately doubles a woman's risk, and having two first-degree relatives increases her risk 5-fold [this is DTC genetic risk information!]… An estimated one in 800 women carry the BRCA1 gene (the number of carriers of BRCA2 remain unknown). Women with inherited changes in either of these genes have up to an 80% chance of developing breast cancer in their lifetime.

23andMe: (it looks like you need to create a demo account and log in to get this info) “Carrier for the 185delAG BRCA1 mutation. Lifetime risk of breast cancer for women is increased from 13% to 81% and risk of ovarian cancer is increased from less than 2% to 54%. May significantly increase risk of prostate cancer in men. There is also an increased risk for breast cancer in men”. Of course it also tells me if I have the mutations or not. [Note that this section is “locked” the results are released only after reading some specific warnings and giving consent]. They also state “Please remember that the BRCA mutations covered by this report are only three of hundreds in the BRCA1 and BRCA2 genes that can cause cancer. Their absence does not rule out the possibility that you may carry another cancer-causing variation in one of those genes.”

EG: no info available

What are the risks? What action would be taken for each of the different possible results?

WebMD may make a person go and take the genetic test, especially if there is a first degree relative.


a. Positive: I don’t know what it says as I have not seen a positive result, but it should say something like “this service is for information only, go to your doctor and order an FDA approved clinical test for BRCA1 and BRCA2 mutations”. Even if it doesn’t the most likely outcome is a trip to the doctor who should then decide to order the approved clinical test. Would a doctor take action based on the 23andMe report? He/she should not and would not if competent – the FDA risk assessment does not take into consideration actions of incompetent professionals – there is nothing in the 23andMe report, as far as I can see, that would make the doctor think that this is an approved clinical test. Outcomes along the lines of demanding an immediate mastectomy and not stopping until finding a surgeon prepared to perform it are inconceivable, and anyway would still require the intervention of an incompetent or unscrupulous professional.

b. Negative: I think we have to base our assessment on the assumption that the customer does read the whole report so it should be clear that the customer realises that there is still a slight chance of having some other mutation and if worried should order full clinical test especially if family history. On the web page (need to log in) they say at the top: “Please remember that the BRCA mutations covered by this report are only three of hundreds in the BRCA1 and BRCA2 genes that can cause cancer. Their absence does not rule out the possibility that you may carry another cancer-causing variation in one of those genes”.

c. False positive: This would obviously cause a lot of anxiety until the result is revealed as false by the follow up clinical test. Do the FDA incorporate the likelihood of a false result, I don’t know? Genotyping is very accurate as far as we know - it would be helpful to have this controlled and published. The interpretation algorithm is very unlikely to be wrong. Overall there is a very low possibility of a false positive, and given that the benefits of detecting the mutation, when it is present, include saving lives, I would say that they outweigh the risks of a few weeks of extreme worry for some individuals.

d. False negative. This is tricky. A false negative could be catastrophic and much worse than a false positive. Negative is the most common result by far and it is not likely to be questioned (unless there is a lot of family history, in which case the clinical test should be taken anyway). A false negative for cancer is particularly bad because the disease is often not detected until it is too late for simple therapy. The consequences would be severe – this is the same for any provider of this test. What are the steps taken with Myriad to ensure that false negatives almost never occur?

For carrier status of other serious diseases (cystic fibrosis etc) I would describe the consequences more or less the same as for the BRCA mutations.

I would classify as low risk this part of the 23andMe service because I cannot see any serious consequences as happening more than very infrequently based on any of the outcomes (except (d), come to that in a minute) and one reason is precisely because it is a DTC test and clearly not to be used as the basis for clinical decisions, in my opinion, reduces the risk, not the opposite. I would definitely classify the Existence Genetics service as medium or high risk (as I would the Myriad test) – because these tests are giving information to clinicians that is designed influence serious clinical decisions, for this reason I would want to be very confident of the quality of the services (and paradoxically, until now, the physician only services have been far less transparent and far less subjected to scrutiny (see here).

Much has been said about the inclusion of BRCA mutations and other serious diseases being a game changer and that now they do need extra regulation. I don’t agree (we need confidence and transparency, but that’s a different issue). Not only are DTC services low risk, they will also be life savers. How often are BRCA mutations found in women after they have been diagnosed with cancer, i.e. too late? That’s the potential number that could be saved by encouraging screening – even though DTC would only be a “first draft” to be confirmed by clinical screening. If FDA involvement is any more than a “light touch”, if it involves anything that would reduce, restrict or remove DTC access then that action would have a high risk of having life threatening consequences for the public – the risks involved in over-regulating are much higher than those of proceeding slowly and with extreme caution. It’s not the other way around.

The false negative worries me somewhat, whether for BRCA or for carrier status of very serious genetic diseases like cystic fibrosis. However it needs putting in perspective. Genotyping as far as we seen so far, at least among the transparent companies, is very accurate. I would like to have a formal public measure of that though, an ongoing and public quality control on any genetics tests for serious mutations, this include Myriad, Counsyl etc. How accurate is Myriad, how many false negatives are they likely to have produced? Some FDA or other office should be set up that regularly sends blind control samples to all the companies (the companies can pay for it with an annual fee – wouldn’t cost much, 24 samples per year?)


Overall, with a few caveats, I think that the service offered by 23andMe is comparable to that of WedMD – I think it is very low risk and should not be more than Class I, there should be almost no time, effort or expense required to get it cleared by the FDA. Notification should be enough. The FDA may have bound themselves now by saying that they are medical devices, but they have not yet decided on things like Class, PMA etc, so there is still the opportunity to regulate this area in a sensible way, and even, ideally, to develop a novel future-proof framework for this new and disruptive field.

Caveats: The obvious differences are that the 23andMe services have more steps, compared to WebMD, where errors are possible (genotyping, software algorithms, and the actual interpretations). The FDA is not concerned with genotyping quality, that is CLIA’s domain (although the FDA, at the same time, are pressuring Illumina, and still at the same time are saying genotype data on it’s own is OK - contradictory). The quality control of the software is equally important – making sure that the data from the lab is not jumbled, that each result is assigned to the correct gene and the correct person. That each interpretation and each calculation based on each SNP, is working correctly. I don’t know what quality standards are required here, but some should be. In my opinion (and experience) there is more chance for error here than in the genotyping itself, much more. We also have to remember that here we are talking about 23andMe, deCODE, etc – they are transparent and have been scrutinised a lot by many experts. We have no idea who is going to come along next though. Companies will come along that will simply be bad, incompetent, unscrupulous or whatever. The regulatory net needs to be fine enough to catch them but without tangling up the good players. If all of the elements of the calls for transparency, the NIH GTR and the HGC Code of Practice were put together into an effective system then that would satisfy the needs to guarantee that DTC personal genetics services are at the lowest risk – basically do what at is proposed by Dan Vorhaus and stir in the HGC.

The situation with Existence Genetics, Counsyl and may be Pathway & Navigenics, and all others who sell to medical practitioners is different. They are all selling a service to a medical professional. A genetic service delivered to clinicians who will take clinical action is something that has higher risk simply because decisions will be made and actions will be taken. The risks of serious consequences are higher, especially when results are in error. DTMD needs tighter controls than a DTC test that is clearly marked “for information only”. If a DTC service did appear that actually did claim that the results can be used to make clinical decisions then that would be need similar controls, in addition to the decision as to whether it should be available DTC or not.

I can get legal or financial information from a website, newspaper, journal etc, and also from lawyers and financial professionals. The difference is clear as are the differences in regulation and accountability.

The debate is the wrong way around – deal with the DTC via the transparency etc methods and deal with genetics services to clinicians as medical devices (I believe that the Roche Amplichip is class II). Once you allow genotyping you cannot block the information flow (SNPedia, Promethease, 23andMe, etc) and if the information is marked as “not for clinical” that has to be accepted.

Maybe not everyone will agree with these conclusions, maybe some of the analysis is mistaken – but regarding personal genetics, I think it is safe.

NB: I need to add something about pharmacogenetics:

The 23andMe result for Greg Mendel is Slightly increased warfarin sensitivity. May require decreased warfarin dose.

It does say at the top…”Only a medical professional can determine whether warfarin is the right medication for a particular patient. The information contained in this report should not be used to independently establish a warfarin regimen, or abolish or adjust an existing course of treatment”.

I bolded independently because I interpret it as meaning that the user should not make a decision alone but can make a decision after consulting a doctor.

I think this is or is close to saying that this information can be used for clinical decisions and I really think that 23andme need to modify the advice. They should say that it needs confirming via a class II regulated medical device and until then NO change in medication should be made.

I agree that the chances of the advice being wrong are small, but we have to think beyond 23andMe and the other high profile well managed companies.

Other recent comments on the FDA re personal genetics:

GeneticFuture: The FDA doesn't plan to regulate access to raw genetic data;

FDA Blog: FDA states cost 'not considered in any of our reviews' and In wake of FDA offensive, genomics entrepreneurs look overseas

Friday, August 6, 2010

DNA Dilemma, dissecting the FDA

Mary Carmichael (General Editor, Newsweek) has this week been running a very interesting series of articles about the “dilemma” of whether or not to undergo a 23andMe type DTC genetic test. As part of that she interviewed the FDA and with journalistic cunning recorded the whole interview and published the full transcript (here). It was a very good thing too that she has the tapes because the interview was in some places quite revealing.

Mary asked me to comment/dissect the interview and I have done so below – I look forward to seeing the comments of others that Mary has asked on their blogs as well.

In a sentence the interview came over to me as unnecessarily vague and evasive (unless for some reason it was necessary!) – certainly lacking in clarity and not answering some of the questions asked but giving up some useful information.

DNA Dilemma: The Full Interview With the FDA on DTC Genetic Tests

NEWSWEEK: During the recent congressional hearing, 23andMe’s general counsel said that the FDA “encouraged them to proceed” freely in 2008. What specifically changed about their test panel between 2008 and now that caused the FDA to start to take some regulatory action? Alberto, I think you’ve touched on this a little with me before, that the concern was mutations with powerful effects on breast-cancer risk or drug metabolism—
tests that might influence medical decisions. Is that correct?

Alberto Gutierrez: They met with [former FDA commissioner Andrew von Eschenbach]. We were not there. We did meet with them on several occasions, but their claims of what they’re offering have been changing constantly. What was there in 2008 I’m sure looks very different from what is there now. The drug-metabolizing claims were not there previously, and even some of the claims that directly touch on disease states, cardiovascular disease, breast cancer were actually not originally—they’re much bigger now.

KG: “we were not there” – so what?! Is he really saying that anything that went on before has no relevance now he is in the seat? He can’t say “I’m sure it looks different”, it either does or it doesn’t and he should know. The cardio stuff was there, there has always been health related content. It’s true though that it has changed with the addition of the “locked” more serious disease associations like BRCA1 and the pharmacogenetics, which are significant additions

That’s interesting. I think before when we talked, the cardio and breast-cancer stuff was not brought up. So even some of the common disease claims—what is the specific concern with them? My understanding was that a lot of those genes really don’t nudge the risk up or down very much.
AG: The concern actually is with everything. The law requires us to clear devices or approve devices before they go into the marketplace when they make medical claims. The question with 23andMe has been whether their claims were medical claims or not. The original claims they were making really were very much on the edge. We actually told them that we thought they were medical claims, but it was at least possible that you could argue that they were not. Now clearly the claims they are making are medical claims. And to show you this is an issue we actually have been dealing with, we actually brought it up in SACGHS, that this was a point of discussion in 2009, because we kept hearing from 23andMe claiming that they were not making medical claims. We actually thought to have the Secretary’s Advisory Committee at least state whether they considered the claims that they were making medical claims or not.

KG: So confirmation that for the FDA the test IS a medical device that needs to be cleared or approved.

In our previous interview, and this is a direct quote, you said those health claims were “what we would consider to be moderate risk.” I’ve recently seen [the FDA’s director of the Center for Devices and Radiological Health] Jeffrey Shuren quoted as saying these claims are high risk, which I assume could mean stricter regulation. Which is it, moderate or high risk, and has something changed since our initial interview? 
It always depends on what medications they can have, how they’re offering it, we always look at what is the risk of an undetected false result, what would be the medical action taken, and whether there’s high risk there or not. I don’t think Dr. Shuren meant to imply that this would be [pre-market approval, or PMA], but that there might be some claims there that we may consider to be PMA-like, or PMA per encounter ...
Elizabeth Mansfield: In general, though, they have a variety of claims that span our normal risk classification. On a claim-by-claim basis, there are some that might be low, there are some that might be moderate, there are some that might be high. I think Dr. Shuren’s assessment that these were “high risk” was as opposed to “no risk.”

KG: Well that cleared that up then! I’m still totally confused about Shuren’s high risk and Manfield’s low, moderate and high. Why is AG talking about “medications they can have” etc? What’s the relevance? What exactly is an “undetected false result”? If it’s undetected a) how can it be false and b) how can any action be taken?!

Maybe I am being mean, let’s assume he means false negative or false positive, that I can understand is an issue worth evaluating - at least if they are actually referring to a false lab result, a miscalled SNP, rather than a false interpretation. (One point though, the FDA will need also include in the equation the likelihood of a false positive or negative – the analysis that Daniel MacArthur carried out on Mark Henderson’s DNA suggested and amazingly low error rate of “less than one in every 14,000 markers”):

  • For most of what 23andMe do (and the others) the false negative/positive is going to be low risk or no risk – this cannot be opposed by most critics who apart from saying that it should be heavily regulated also say that the risk predictions are small and imprecise. Yes, that’s accepted, which means a false result would be low risk or even no risk.
  • For the pharmacogenetics aspects this is different. If the false result is used to modify a drug prescription then it could be medium or high risk (I’m not qualified or knowledgeable enough about potential adverse consequences to choose which). A false positive would be the same. If the companies want to continue to claim that the information is for education only and is not intended for clinical use they need to make that very clear next to these results – they need to say that before any clinical decisions are made that these particular gene variants need to be retested by a lab that offers the test as a true regulated medical device. This I can agree with – the 23andMe pharmacogenetic interpretations are correct but that will not be the case with all companies in the future, and it applies to both physician prescribed and DTC tests.
  • For serious diseases, the “locked in” sections – where you need to give specific approval via the website before the results are revealed to you I would assume would be medium/high risk. To be wrongly informed of cystic fibrosis carrier status, high risk BRCA variants, etc, would be worrying for the customer. Not life threatening though, as long as explained correctly. I have not seen how they present a positive (as in carrier of risk allele) but again they would have caution that the result is provisional and any further action will require testing via a properly regulated device. Medium or High though, I am not sure, although I would say medium (but I realise a lot would say high) – If I got the information that I had a very high risk for prostate cancer I would be worried, I would straight away get retested by another lab, if that confirmed it, I would want to repeat it again (I have worked in both research and diagnostic labs for many years, I would repeat it!). If it was confirmed then it’s possible 23andEtc would have saved my life – if it was not confirmed, I would have gone through a week or two of anxiety, but I would not have thrown myself off a bridge.

Having just typed the above (“it’s possible 23andEtc would have saved my life”) – I am thinking now that it would be more risky than beneficial if the FDA took action which reduced access to these tests. They are potentially lifesaving and I am thinking that perhaps the “high risk of serious disease” tests are actually more important to go DTC (not as a free for all though, this does not mean NO regulation) and should be encouraged. Let people decide and pay for something if they want to.

If a medical decision is going to be made then I would want the interpretation to be as bullet proof as possible, therefore approved. I cannot see a serious (medium or high risk) medical decision being made without the use of a doctor though so I would only require approval of tests marketed to doctors as tests on which clinical decisions will be made. It’s a sort of paradox here, if it’s DTC it should be clear that it is not for medical use – a doctor should not make a clinical decision based on a 23andMe test. A customer can make a lifestyle decision, that’s different. A paradox because the decision of Pathway to go only via medical doctors should logically make it more liable for regulation. I would assess the 23andMe website as just like any other healthcare site, like WebMD, is that a medical device? WedMD has risk factor calculators, it makes medical interpretations of personal information, I really don’t see the difference as far as regulation is concerned. No doctor would make a clinical decision based on a patients interpretation of what he or she has read on WebMD, no reliable doctor that is.

So the FDA would actually be going through these panels test by test, and looking at each one and seeing, Is this one low risk or moderate risk or high risk, and approving them specifically?
: Yes, in a general sense we would go claim by claim. Whether each claim would require a different submission or not—we’d be looking at each of their claims and looking to see that they would have data to credibly back the claims they are making.

KG: good luck to them – Sisyphus

So if one of these companies wanted to add a new test to its panel, it would have to get that specifically approved? 
No. I want to make a distinction between what we would be doing now, and maybe what we would do in the future, and the reason here is that—Dr. Shuren put this on the table [unintelligible]—with these devices maybe it can collect a lot of data that may not be relevant at the moment. If later on data becomes available that shows that the company may be able to make a claim that doesn’t exist now, is there a way for us to do this that wouldn’t require necessarily a submission for each of them, or that we could bring together experts that could make the call that there is now enough evidence to make this kind of claim? That is something that we are discussing, that we are trying to figure out how we’re going to deal with devices that generate a lot of data for which there is no use now but may be in the future, like with whole-genome sequencing …

KG: Again very clear! It will be “claim by claim” (which is a specific thing, it can’t be a “general sense”) – or maybe it won’t be. All he should have said to the previous question is “that we are trying to figure out how”. Also Shuren in Congress seemed to recognise that this would be impossible and suggested a different way, something like overall approval of a company with regular reviews (like CLIA perhaps?)

[When we spoke last time], you had just sent five letters out and Illumina got one. Your concern in that letter, I believe, was that Illumina was selling a “research use only” chip and it was being used for nonresearch purposes by DTC firms. There were several reports I saw this morning indicating that the FDA has “requested” that Illumina stop selling chips to DTC genetic-test providers. Is that true in whole or in part?
We have asked Illumina to work with us to bring their devices into compliance with our regulations. We have not specifically requested that they stop selling them to anybody.

KG: OK… can you “non-specifically” make such a request, with a nudge and a wink perhaps?

So they can continue supplying 23andMe and deCODEme with chips right now?
We’re working with them on that …

KG: Ah yes, looks like you might be able to…

Illumina obviously isn’t the only chip maker out there. Why is it the only one that is getting these letters? Are the other ones going to be called in as well?
You can bet that with the original set of letters in 2009, Illumina was not the only one to receive a letter. 
I’m sorry, I’m not familiar with the 2009 letters. What were those? 
We sent letters to several companies with direct-to-consumer chip arrays to come in and discuss the issue.
And did those companies come in? 
: Yes, and I think that’s part of what you saw with the public meeting on June 30 on the array issue and why several companies want to figure out a way to [unintelligible] …
[The chips are labeled “research use only.”] The DTC providers, when they sell these tests, aren’t [necessarily] doing research. Doesn’t that imply that Illumina does have to stop selling the chips to them?
If they continue to label them that way, yes, that would be something that we would most likely take a little further action on. At the moment, they’re working with us and we will see what they can bring to us in a reasonable amount of time without completely blowing up their business or the market or anything …

KG: I really hope that is the case

So I guess if it’s not being used for research only it needs a different type of [label or] approval?
That’s correct…

KG: Label OR approval, what do they want, what would Illumina have to do, how long, how much, what about when new SNPs are added, etc?

Until that happens, though, is it correct that the Illumina chips will remain on the market for direct-to-consumer companies to buy?
[Pause] I think Illumina needs to figure out how they’re going to move forward.

KG: Oh dear, back to the “non-specific” – horses head in the bed?

Okay … I want to move on to whether the issue with direct-to-consumer is actually providing data to people, or is it the interpretation algorithms these companies are using? So, would a company need to be approved just to provide a raw SNP list to people?
They would if they made medical claims about that data. If they don’t make any medical claims about that data, then they’re free to provide information as far as we’re concerned.

KG: Then the FDA need to give clear guidelines about what constitutes a medical claim. But one good thing – there will be NO obstruction to me getting my raw genotype data DTC. So then what happens if I go and read about my SNPs on, or pubmed, or OMIM or the excellent “HuGE Navigator”? What if I put my data through Promethease ? What about reading about SNPs and disease risk on WebMD and on and on

What exactly would constitute a “medical claim?” Would pointing people to medical research papers [qualify]?
It depends. There are rules as to how one can do that … Those rules are actually worked out pretty well, and they just would need to make sure they’re staying within the rules.
Are those rules on the Web?
I don’t know where the policy is. I would have to get it for you. It’s an agencywide policy. I would have to find it for you. And it won’t be that easy for people to follow it…

KG: The rules are worked out pretty well, now dammit, if only I could find them… What? “I don’t know”, “I wasn’t there” (see above), “it won’t be easy to follow” – terrible admissions to make

Alberto, I had heard a story, I believe it was in Rob Stein’s article in [The Washington Post], about these women who were having prophylactic oophorectomies on the basis of direct-to-consumer genetic results that were  “questionable.”
That’s a little bit taken out of context in the sense that it’s not based on the direct-to-consumer [model]. The cases we heard of were based on lab-developed tests that were not properly validated. What I did say was that even though 23andMe doesn’t make a direct claim as for ovarian cancer, they do provide information that links what they are giving the consumer to ovarian cancer.

KG: A little bit? It would have to be a lot out of context to have made such a shocking claim. It was shameful (actually it was also unbelievable, for good reason, it wasn’t true)

So that’s an example of where the interpretation is about pointing to a medical paper. 
: It’s more than that. They interpret the medical paper for them … Ovarian cancer is called the “silent killer.” People actually don’t know they have it until it’s too late. And women that are high risk a lot of times choose to have their ovaries removed because they are afraid that it may not be caught in time. There’s no diagnostic test for it. So the tests that we have seen—there was one that was taken off the market because of FDA action … We knew that in those cases most women who were being operated on actually were likely to be negative for ovarian cancer … The one we heard most about originally, it was a test called OvaCheck that was going to go to the market, and we believed it was not well characterized and it was going to create problems. It never made it to the market, and then in 2008 another one made it to the market called OvaSure, and the FDA wrote LabCorp a warning letter …

KG: His answer is not at all relevant to your question but is interesting just the same. The “silent killer” – that makes me think, again, that anything that makes people more aware of their health, their possible risks, possible diseases, especially symptom free diseases, etc, should be encouraged, and should be DTC. I was not aware of this information about ovarian cancer, nor was my wife. If it silent then you are not going to go to a doctor about it until it is too late and of course AG knows very very very well that nobody would undergo serious surgery without medical guidance, I suppose it’s impossible unless you really want to do it in an illegal fashion, as in the days of the back street abortion.

Thank you, that’s very helpful to clarify. In general, when the FDA makes decisions about potentially fairly serious new regulation of a device or a test or a drug for that matter, does it do any sort of analysis and take into account the costs that regulation may impose on companies?
This is a more complicated question than you actually think. We have rules we’re supposed to follow that we’re supposed to apply evenhandedly to everybody about what exactly is a new product. The law requires new medical devices to come into the FDA to get approval …That’s what the law is, so we don’t actually get to make a decision that’s based on the economic issues for that new device.

KG: It shouldn’t be complicated – surely they can say to a company “you will need PMA and you can expect it to cost $xyz”? Or do they just enjoy helping to proliferate the “FDA regulations consultant” industry?

I’m a little bit confused. Can we talk about that in the specific context of direct-to-consumer tests? What kind of thinking was there about the costs? 
[Direct-to-consumer tests] are a new medical device, and under the rules of the 1976 law, new medical devices require FDA clearance or approval before they go onto the market.

KG: No doubt again, they ARE a medical device

Right. But as part of that deliberation process, when you guys are considering whether to approve a device or not, do you take into account the cost that would impose on companies or the general impact on the industry?
No. No. Our review does not, no, we don’t take into account cost.
EM: And Mary, that cuts agencywide. That is not considered in any of our reviews.
Okay. Thank you. I thought I had seen some obscure law saying there was a
requirement to take into [account] costs … Yes. Here it is. This is actually on the FDA Web site: “As part of its mission to supply economic analysis to decision makers, the Economics Staff in the Office of Planning conducts economic analyses of all important proposed and final regulations issued by the Food and Drug Administration. Each economic analysis includes an assessment of the costs, benefits, and cost-effectiveness of the action.” So this is right on the Web. How can you not take— 
So that is just, if you look at the first part, that is for all new guidances, for new rules and regulations.
EM: Which this is not.
AG: This is neither a rule nor a regulation.

KG: But once upon a time it was a new rule so an economic assessment should have been done, it should exist and can be adjusted for inflation – what’s the problem??

I see. You’re saying that because this is just like in 1976, these are medical devices, therefore they need approval— you don’t consider this new regulation, it’s just that the old regulations need to apply to these companies?
That’s correct.

KG: Do they actually know what they are talking about?

Okay, thank you. One more question: … We’ve had the hearing and there were obviously some fairly troubling things that came out of the GAO report. I’m wondering what impact the hearing is having on the FDA right now. Are you guys taking those findings into account at all? Is that influencing your decisions?
The findings of the GAO?
And other things that came out of the hearing. 
No, we believed that these devices [should be] under regulation. It hasn’t influenced us. Dr. Shuren did say that we should have been moving faster, and that we need to make these companies come in and get cleared for moving forward and selling their tests on the market. We will proceed with due diligence and try to help the companies come into compliance.   
EM: The GAO report clearly did not undermine our feelings that these tests could present some risks to patients.

KG: I hope AG is right and that the GAO report is not used. EM’s answer though suggest it will be influential and I bet it will be. It should not be because as even the GAO admit, it was not scientific, the GAO did not make all of their information available for scrutiny, it was not independently reviewed, only selected sections and edited conversations were presented in a very theatrical way. I would bet that if all the information was made available it would show that overall the industry, as far as the reputable companies are concerned, is working very well and may even undermine EM’s feelings. It was very obvious that the GAO was making a “case for the prosecution” it was not an even handed survey, it was presented in a way to maximise the impression that the whole field is misleading and unethical.

[Given] Dr. Shuren’s comments that there should have been faster movement, are you guys feeling pressured to move fast right now?
: I think we are doing what we are supposed to be doing. We are obviously trying to get the companies to come into compliance and working with them to do so.

KG: OK we’ll see what’s next. Will it be like 2006 where the FDA got a similar rap on the knuckles by the Senate? If so then I suppose we will be back here after 4 more years of limbo to discuss the GAO/FDA/ETC findings on the DTC whole genome sequencing industry.

Overall the interview does nothing to clarify much at all, except that the FDA decided long ago that they are medical devices, even though they don’t know where the rules are (did he really say that? Thank goodness you got the tapes!). It’s all very murky – I was rather cynical in thinking that there are some murky politics behind it but after talking to Mary last night who had actually spoken to them, I think it is more that they simply do not know what to do yet, they don’t know how to regulate it. That is fine, it’s a very complex new situation, it’s not a simple laboratory test – the interpretation is a constantly changing river of information. There is no shame in not knowing what to do but the best response is not to pretend you are in control, it is better to consult widely (and openly, in public) and work closely with the industry you are trying to regulate. It is an opportunity to develop new innovative regulatory methods and not try to apply old rules, designed for simple tests, to this new, complex area.

One thing is clear though – if it makes a medical claim is it a medical device. I believe there should be clear guidelines for medical claims. Look – the nutrition and supplement industry are desperate to make health claims, they have to spend $millions to get them accepted and most of their applications are rejected. Here with DTC the situation seems to be the opposite, the FDA seem to have decided quite easily that these are valid medical claims. A medical claim should be something that would influence a clinical decision. Or a medical claim would be something like “if you take this test and follow our advice, or take our supplements, it will reduce your risk for disease” – if the DTC companies are quite clear that they are providing information that should not be used to make clinical decisions, and are not claiming to actually reduce your risk, are they really making health claims, are they really any different to any of the websites I have mentioned, including many run by the government?

Other recent comments on the FDA re personal genetics:

GeneticFuture: The FDA doesn't plan to regulate access to raw genetic data;

FDA Blog: FDA states cost 'not considered in any of our reviews' and In wake of FDA offensive, genomics entrepreneurs look overseas