Thanks for another’s great episode. I have two comments/questions:
1. Are you familiar with the psychologist Irving Kirsch? He wrote a book about the placebo effect a decade ago titled, The Emperor’s New Drugs. In his book, he argues that antidepressant medications were no better than placebo. I remember he notes in his book that the placebo effect is so powerful that it induced skin rashes in participants of one study.
2. In the podcast, both of you expressed skepticism when describing studies that relied on brain scans/FMRIs. I was curious as to what this was about.
Hi Linotte! Am rushing so I won't answer 1) in any great detail except to say I know Hirsch's name but little more. Re 2) my scepticism as others say is very much about the small sample size but also the inherent noisiness of brain scan studies - they are always presented as clear "this bit of your brain lights up" but it's actually just statistical tendencies for more activity in a given region, and with the invariably tiny sample sizes, it's very easy to get misleading results and false positives and so on. Thanks!
The Emperor's New Drugs thing is interesting - he finds the same effect size for antidepressants as another antidepressant meta-analysis that came out some years later (it is above zero). But he says "this shows the drugs don't have an appreciable/clinically useful effect beyond placebo" whereas the other study says "this shows they work beyond placebo and are clinically useful"! Always my go-to example of how different scientists can interpret the same result completely differently.
Yes! Small studies (due to very expensive MRI machines) are a major part of why I'm generally sceptical, but also various statistical issues with the analyses, and generally the interpretation of what it even means that A B C area of the brain "lights up" when you do X Y Z task.
I'm less sceptical of *structural* MRI studies by the way - that is, the ones that try and measure various static aspects of the brain and don't look at what "lights up" when - they look at questions like how big this or that brain region is. But that's just relative and I'm still pretty sceptical, especially of the smaller studies.
Having said that, we're getting to the point now where much bigger samples are available, and thus to the point where many more of the results are reliable. We will definitely do an episode on this!
Thanks for another’s great episode. I have two comments/questions:
1. Are you familiar with the psychologist Irving Kirsch? He wrote a book about the placebo effect a decade ago titled, The Emperor’s New Drugs. In his book, he argues that antidepressant medications were no better than placebo. I remember he notes in his book that the placebo effect is so powerful that it induced skin rashes in participants of one study.
2. In the podcast, both of you expressed skepticism when describing studies that relied on brain scans/FMRIs. I was curious as to what this was about.
Hi Linotte! Am rushing so I won't answer 1) in any great detail except to say I know Hirsch's name but little more. Re 2) my scepticism as others say is very much about the small sample size but also the inherent noisiness of brain scan studies - they are always presented as clear "this bit of your brain lights up" but it's actually just statistical tendencies for more activity in a given region, and with the invariably tiny sample sizes, it's very easy to get misleading results and false positives and so on. Thanks!
The Emperor's New Drugs thing is interesting - he finds the same effect size for antidepressants as another antidepressant meta-analysis that came out some years later (it is above zero). But he says "this shows the drugs don't have an appreciable/clinically useful effect beyond placebo" whereas the other study says "this shows they work beyond placebo and are clinically useful"! Always my go-to example of how different scientists can interpret the same result completely differently.
Here's on [2]:
https://www.astralcodexten.com/p/against-that-poverty-and-infant-eegs
Which comes with a bunch of links in it to Stuart's various stuff too.
Might even be worth a whole episode (the list grows longer)
You might be interested in next week's one...!
I’m sure I will be!
I could be wrong, but I got the impression that the brain scan/FMRI comment was about the tiny sample sizes inherent with that methodology.
Yes! Small studies (due to very expensive MRI machines) are a major part of why I'm generally sceptical, but also various statistical issues with the analyses, and generally the interpretation of what it even means that A B C area of the brain "lights up" when you do X Y Z task.
I'm less sceptical of *structural* MRI studies by the way - that is, the ones that try and measure various static aspects of the brain and don't look at what "lights up" when - they look at questions like how big this or that brain region is. But that's just relative and I'm still pretty sceptical, especially of the smaller studies.
Having said that, we're getting to the point now where much bigger samples are available, and thus to the point where many more of the results are reliable. We will definitely do an episode on this!