Statistical Modeling, Causal Inference, and Social Science

Harry Selker and Alastair Wood say yes.

Jeroen Ooms writes:

Here's a first version of a new web application for exploratory graphical analysis. It attempts to implement the layered graphics from the R package ggplot2 in a user-friendly way. This two-minute demo video demonstrates a quick how-to.

He asks for feedback, so if you have any, feel free to comment. I don't know ggplot2 but my impression is that I should really be using it. Maybe Yu-Sung and Daniel should consider using it for mrp.

Christopher Nelson tries his hand at being a graphics curmudgeon.

Mark Blumenthal links to Rush Limbaugh accusing Gallup of "upping the sample to black Americans to keep [Obama] up at 50%" in the polls. (For the context, see the last paragraph of the transcript.)

Frank Newport of Gallup responds here. Newport denies it all, but he would, wouldn't he?

Seriously, though, it's hard to believe that Limbaugh really believes that Gallup is fudging the numbers. As a big-time radio host, he's gotta know all about marketing surveys, right? I'm just assuming he said that "upping the sample" bit as more of a joke or an off-the-wall speculation. It did raise two interesting questions in my mind, though:

1. The assumption behind Limbaugh's argument--as with many arguments about polls--is that the published poll results have an effect of their own, beyond he president's underlying popularity. For example, maybe some senator would vote for the health care bill if he read that Obama's approval rating was 51% but would vote no if he read that Obama only had 49% approval. This might very well be true--it makes sense--I just don't really know.

2. What if you were a pollster and really did want to cheat and overrepresent Democrats? How would you do it? Contra Limbaugh's suggestion, I don't think you'd oversample blacks. I'm assuming Gallup does telephone surveys, and it's not like there's a separate telephone directory for blacks. Also, as several commenters to Newport noted, the percentage of blacks among the survey respondents is easy enough to check. And, for that matter, many survey organizations (possibly including Gallup) do post-sampling weighting adjustments for race, anyway, in which case oversampling blacks won't do anything for you at all.

If you're doing a telephone poll and want to oversample Democrats, you can just call states and area codes where more Democrats live. Call New York, LA, Chicago, etc. You can even call people in Democratic-leaning white areas if you want to mix things up a bit. That'll do the trick. Bury it deep enough in the sampling algorithm and maybe nobody will notice!

P.S. I looked at Gallup's home page and was surprised not to see any link to a description of their sampling methods. Or maybe it's somewhere and I didn't see it.

P.P.S. Blumenthal sent me this helpful link.

Jimmy points me to this article, "Why most discovered true associations are inflated," by J. P. Ioannidis. As Jimmy pointed out, this is exactly what we call type M (for magnitude) errors. I completely agree with Ioannidis's point, which he seems to be making more systematically than David Weakliem and I did in our recent article on the topic.

My only suggestion beyond what Ioannidis wrote has to do with potential solutions to the problem. His ideas include: "being cautious about newly discovered effect sizes, considering some rational down-adjustment, using analytical methods that correct for the anticipated inflation, ignoring the magnitude of the effect (if not necessary), conducting large studies in the discovery phase, using strict protocols for analyses, pursuing complete and transparent reporting of all results, placing emphasis on replication, and being fair with interpretation of results."

These are all good ideas. Here are two more suggestions:

1. Retrospective power calculations. See page 312 of our article for the classical version or page 313 for the Bayesian version. I think these can be considered as implementations of Iaonnides's ideas of caution, adjustment, and correction.

2. Hierarchical modeling, which partially pools estimated effects and reduces Type M errors as well as handling many multiple comparisons issues. Fuller discussion here (or see here for the soon-to-go-viral video version).

P.S. Here's the first mention of Type M errors that I know of. The problem is important enough, though, that I suspect there are articles on the topic going back to the 1950s or earlier in the psychometric literature.

Postdoc opportunities working with Prof. Andrew Gelman in the Department of Statistics on problems related to hierarchical modeling and statistical computing, with projects including high-dimensional modeling, missing-data imputation, and parallel computing. Application areas include public opinion and voting, social networks, international development, dendrochronology, and models of cancer and drug abuse. Applicants should have experience with Bayesian methods, a willingness to program, and an interest in learning. Applications will be considered as they arrive. The application consisting of cover letter, cv and a selection of published or unpublished articles should be emailed to asc.coordinator@stat.columbia.edu. Please also arrange for three letters of recommendation to be sent to the same email address. This is an exciting place to work: our research group involves several faculty, postdocs, graduate students, and undergraduates working on a wide range of interesting applied problems. We also have strong links to the Earth Institute, the Center for Computational Learning Systems, and the Columbia Population Research Center, as well as to Statistics, Political Science, and other academic departments at Columbia. As a postdoc here, you will have an opportunity to work on collaborative projects on theory, application, computation, and graphics. You can talk to our current and former postdocs if you want to hear how great it is to work here. Positions are usually for two years. Columbia University is an Equal Opportunity/Affirmative Action employer.

Also, if you're finishing up your Ph.D. in statistics, have interest in public health and international development, and would like to work with me, please contact me regarding the Earth Institute postdoc. Application deadline is 1 Dec, so time to get moving on this!

Seth writes:

Is this a fair statement, do you think?

Science revolves around the discovery of new cause-effect relationships but the entire statistics literature says almost nothing about how to do this.

It's part of an abstract for a talk I [Seth] will give at the ASA conference next July. Haven't submitted the abstract yet so can revise it or leave it out.

My reply: This seems reasonable to me.

You could clarify that the EDA literature is all about discovery of new relationships but with nothing about causality, while the identification literature is all about causality but nothing about the discovery of something new.

Nate, Daniel, and I have an op-ed in the Times today, about senators' positions and state-level opinion on health care. We write:

Lawmakers' support for or opposition to reform generally has less to do with the views of their constituents and more to do with the issue of presidential popularity. . . .

For instance, Senator Blanche Lincoln, a Democrat who has been a less-than-strong supporter of the present health care bill, recently told The Times, "I am responsible to the people of Arkansas, and that is where I will take my direction." But where does she look for her cue? Hers is a poor state whose voters support health care subsidies six percentage points more than the national average. On the other hand, Mr. Obama got just 40 percent of the vote there.

Likewise, in Louisiana, where the Annenberg surveys showed health care reform to be popular but where Mr. Obama is not, the Democrats are not assured of Mary Landrieu's vote. . . .

Here's our graph that makes this point:

Fun stuff.

Daniel Corsi writes:

I am a PhD student in epidemiology at McMaster University and I am interested in exploring how characteristics of communities are related to child health in developing countries.

I have been using multilevel models to relate physical characteristics of communities such as the number of schools, health clinics, sanitation facilities etc to child height for age and weight for age using observational/survey data.

I have several questions with regards to the group (community-level) level predictors in these models.

I was checking out the comments at my bloggingheads conversation with Eliezer Yudkowsky, and I noticed the following, from commenter bbbeard:

My sense is that there is a fundamental sickness at the heart of Bayesianism. Bayes' theorem is an uncontroversial proposition in both frequentist and Bayesian camps, since it can be formulated precisely in terms of event ensembles. However, the fundamental belief of the Bayesian interpretation, that all probabilities are subjective, is problematic -- for its lack of rigor. . . .

Andrew Roberts writes:

I teach political science at Northwestern. I have a book coming out with U of Chicago Press called "The Thinking Student's Guide to College" and I wanted to ask you a question about one part.

I have a section where I advocate a few "neglected majors". One of them is statistics. I wrote the following (see below) about statistics, but it seems a little dull to me. I'd be curious if you would add anything that would make the major seem more attractive. (FYI, the other neglected majors are linguistics, regional studies, and sociology).

To fully understand just about any phenomenon in the world, from atoms to people to countries, you need a grasp of statistics. Statistics teaches you how to measure quantities, collect data, and then draw inferences from that information. Though this might sound boring, these tasks are necessary to explain most of the forces affecting our lives, whether the workings of markets, the movement of public opinion, or the spread of disease. Not only does a statistics major give you the skills to answer these questions, it is also extremely marketable. There is hardly a firm which could not benefit from a trained statistician, and statisticians are just as desirable for public interest groups hoping to help the disadvantaged. And if you worry that you are not the math type, statistics is considerably less demanding than a pure math major and does more to help you understand the real world in all its complexities.

"Considerably less demanding than a pure math major," huh? OK, OK . . .

My main suggestion would be to be less apologetic. No need to say "Though this might sound boring"!

Perhaps some of you have specific suggestions for Andrew Roberts for his book?

In the spirit of Christian Robert, I'd like to link to my own adaptive Metropolis paper (with Cristian Pasarica):

A good choice of the proposal distribution is crucial for the rapid convergence of the Metropolis algorithm. In this paper, given a family of parametric Markovian kernels, we develop an adaptive algorithm for selecting the best kernel that maximizes the expected squared jumped distance, an objective function that characterizes the Markov chain. We demonstrate the effectiveness of our method in several examples.

The key idea is to use an importance-weighted calculation to home in on a jumping kernel that maximizes expected squared jumped distance (and thus minimizes first-order correlations). We have a bunch of examples to show how it works and to show how it outperforms the more traditional approach of tuning the acceptance rate:

Regarding the adaptivity issue, our tack is to recognize that the adaptation will be done in stages, along with convergence monitoring. We stop adapting once approximate convergence has been reached and consider the earlier iterations as burn-in. Given what is standard practice here anyway, I don't think we're really losing anything in efficiency by doing things this way.

Completely adaptive algorithms are cool too, but you can do a lot of useful adaptation in this semi-static way, adapting every 100 iterations or so and then stopping the adaptation when you've reached a stable point.

The article will appear in Statistica Sinica.

In response to my note on the limited ideological constraints faced by legislators running for reelection, Alan Abramowitz writes:

I [Abramowitz] agree--although they probably have less leeway now than in the past due to growing pressure toward ideological conformity within parties, especially GOP. But one thing that struck me as very interesting in your graph is that it looks like the advantage of a moderate voting record is considerably smaller now than it used to be, down from over 4 percentage points in the 1980s to maybe 1.5 points on average now. It suggests to me that the electorate has become increasingly partisan and that fewer voters are going to defect to an incumbent from the opposing party regardless of voting record. This could reflect more concern among voters with party control of Congress itself. Along these lines, one thing I've found in the NES data is a growing correlation between presidential job evaluations and voting for both House and Senate candidates over time.

My reply: Yes, that makes sense. The trend is suggestive although (as you can see from the error bars) not statistically significant. Recently I have not had my thoughts organized enough to write any articles on this stuff, but it feels good to at least post these fragments for others to chew on.

A correspondent writes:

You may be interested in this article by Matthew Rabin which makes the point that you make in your article: if you are an expected utility maximizer then turning down small actuarially unfair bets (e.g. 50% win $120; 50% lose $100) implies that you would never accept a bet where could lose $1000 (even if you might win an infinite amount of money). (But proved in more generality).

This was taught to me in the first year of my econ phd program (which I'm currently in!) as why you probably don't want to extrapolate from decisions over small bets to risk aversion in general, not as why we should throw out risk aversion and expected utility maximization completely. Of course, decision theorists do all kinds of things to try to "fix" this problem.

My reply: Yitzhak (as we called him in high school) wrote his paper after mine had appeared; unfortunately my article was in a statistics journal and he had not heard about it. (This was before I could publicize everything on the blog. And, even now, I think a few papers of mine manage to get out there without being noticed.)

I'm glad they teach this stuff in grad schools now--although, in a way, this still proves my point, in that the nonlinear-utility-function-for-money model is still considered such a standard that they feel the need to debunk it.

My correspondent replied: "I wouldn't call it a debunking....we still go on to use it as the workhorse model in everything we do...."

I think there are good and bad things about this "workhorse model":

Here.

The other day I commented on an article by Peter Bancel and Roger Nelson that reported evidence that "the coherent attention or emotional response of large populations" can affect the output of quantum-mechanical random number generators.

I was pretty dismissive of the article; in fact elsewhere I gave my post the title, "Some ESP-bashing red meat for you ScienceBlogs readers out there."

Dr. Bancel was pointed to my blog and felt I wasn't giving the full story. I'll give his comments and then at the end add some thoughts of my own. Bancel wrote:

1. From what I read, 2012 is a big-budget, low-brains remake of Miracle Mile, while completely missing the point of the original. So sad.

2. Meryl Streep was totally wasted in Fantastic Mr. Fox. And I don't mean she was drunk--well, maybe she was, who knows?--but her talent went largely unused. Seems like a crime to have Meryl Streep in a movie and not make more use of what she can do.

On the other hand, everyone deserves to relax now and then. If Streep is going to be taking a break, there's no harm in her doing it in the context of a movie.

And, on the plus side, they didn't let Gilbert Gottfried or anyone who sounds like him get anywhere near the place.

Somebody in Nigeria wants to send me money. I can't give you the details, but let me say that if this does work out, I'll never have to worry about book royalties again. I feel a little guilty about dealing in blood diamonds, but if this deal works out, I can make it all right by donating a lot to charity.

I've been ranting lately about how I don't like the term "risk aversion," and I was thinking it might help to bring up this post from last year:

Jonathan Rodden and Jowei Chen sent me this article:

When one of the major parties in the United States wins a substantially larger share of the seats than its vote share would seem to warrant, the conventional explanation lies in manipulation of maps by the party that controls the redistricting process. Yet this paper uses a unique data set from Florida to demonstrate a common mechanism through which substantial partisan bias can emerge purely from residential patterns. When partisan preferences are spatially dependent and partisanship is highly correlated with population density, any districting scheme that generates relatively compact, contiguous districts will tend to produce bias against the urban party. In order to demonstrate this empirically, we apply automated districting algorithms driven solely by compactness and contiguity parameters, building winner-take-all districts out of the precinct-level results of the tied Florida presidential election of 2000. The simulation results demonstrate that with 50 percent of the votes statewide, the Republicans can expect to win around 59 percent of the seats without any "intentional" gerrymandering. This is because urban districts tend to be homogeneous and Democratic while suburban and rural districts tend to be moderately Republican. Thus in Florida and other states where Democrats are highly concentrated in cities, the seemingly apolitical practice of requiring compact, contiguous districts will produce systematic pro-Republican electoral bias.

My thoughts:

I had mixed feelings about this one.

My first reaction was disappointment that Ellis Weiner, the author of the great book, Decade of the Year, is reduced to writing one-page bits where he's flopping around like a seal on a beach trying to squeeze a few laughs out of . . . whatever. You get the idea.

My second reaction, after reading the piece--I never read this sort of thing but I was curious what Weiner's been up to lately--was how scarily accurate his description was of the book-marketing process, the way they basically want you, the author, to do all the promotion, and also the bit where the publicist says, "I sort of have my hands full, promoting twenty-three new releases this fall, but I'm really excited about working on your book..."

My third reaction was: hey, this story really wasn't very good at all. Basically, he was taking a so-so idea (I mean, really, how many readers can really relate to complaints about book publicity) and then not taking it enough over the top. How did this end up getting published in the New Yorker, a place where I assume the best humor writers in America would kill to be in? Personal connections are part of the story, possibly. Beyond this, I suppose that magazine editors are the sort of people who would be particularly amused by a joke about the publishing industry. It's a real step down from the days of Veronica Geng, that's for sure.

P.S. I hate to be so negative, but I'm pretty sure that New Yorker writers don't spend their time reading statistics blogs. So I think I'm safe in writing this without worrying that I've hurt his feelings.

Matthew Yglesias remarks that, when staking out positions, congressmembers are not very strongly constrained by the ideologies of their constituents.

Wow, that was a lot of big words. What I meant to say was: Congressmembers and Senators can pretty much vote how they want on most issues, whatever their constituents happen to believe. Not always, of course, but a representative can take a much more liberal or conservative line than the voters in his or her district or state, and still do fine when election time comes.

Yglesias gives some examples from the U.S. Senate, and I just wanted to back him up by citing some research from the House of Representatives.

First, here's a graph (based on research with Jonathan Katz) showing that, when running for reelection, it helps for a congressmember to be a moderate--but not by much:

Being a moderate is worth about 2% of the vote in a congressional election: it ain't nuthin, but it certainly is not a paramount concern for most representatives.

This is one I'd never thought about . . . Daniel Rubenson writes:

I'm an assistant professor in the Politics Department at Ryerson University in Toronto. I will be teaching an intro statistics course soon and I wanted to ask your advice about it. The course is taught to fire fighters in Ontario as part of a certificate program in public administration that they can take. The group is relatively small (15-20 students) and the course is delivered over an intensive 5 day period. It is not entirely clear yet whether we will have access to computers with any statistical software; the course is taught off campus at a training facility run by the Ontario Fire College.

A reporter contacted me to ask my impression of this article by Peter Bancel and Roger Nelson, which reports evidence that "the coherent attention or emotional response of large populations" can affect the output of quantum-mechanical random number generators.

I spent a few minutes looking at the article, and, well, it's about what you might expect. Very professionally done, close to zero connection between their data and whatever they actually think they're studying.

Masanao points me to this. Incidentally, Don Rubin was a psychology grad student for awhile.

In the discussion of the attention-grabbing "global cooling" chapter of the new Freakonomics book,some critics have asked how it is that free-market advocates such as Levitt and Dubner can recommend climate engineering (a "controlled injection of sulfur dioxide into the stratosphere"), which seems like the ultimate in big-government solutions? True, the Freakonomics recommendation comes from a private firm, but it's hard to imagine it would just be implemented in somebody's backyard--I think you'd have to assume that some major government involvement would be necessary.

So what gives?

Ben Highton writes:

One of my colleagues thinks he remembers an essay your wrote in response to the Cox/Katz argument about using "involuntary exits" from the House (due to death, etc.) as a means to get leverage on the incumbency advantage as distinct from strategic retirement in their gerrymandering book. Would you mind sending me a copy?

My reply:

It's in our rejoinder to my article with Zaiying Huang, Estimating incumbency advantage and its variation, as an example of a before/after study (with discussion), JASA (2008). See page 450. Steve Ansolabehere assisted me in discussing this point.

P.S. There was a question about how this relates to David Lee's work on estimating incumbency advantage using discontinuities in the vote. My short answer is that Lee's work is interesting, but he's not measuring the effect of politicians' incumbency status. He's measuring the effect of being in the incumbent party, which in a country without strong candidate effects (India, perhaps, according to Leigh Linden) can make sense but doesn't correspond to what we think of as incumbency effects in the United States. Identification strategies are all well and good, but you have to look carefully at what you're actually identifying!

David Afshartous writes:

Regarding why one should not control for post-treatment variables (p.189, Data Analysis Using Regression and Multilevel/Hierarchical Models), the argument is very clear as shown in Figure 9.13, i.e., we would be comparing units that are not comparable as can be seen by looking at potential outcomes z^0 and z^1 which can never both be observed. How would you respond to someone that says "well, what about a cross-over experiment", wouldn't it be okay for that case?" I suppose one could reply that in a cross-over we do not have z^0 and z^1 in a strict sense, since we observe the effect of T=0 and T=1 on z at different times rather than the counterfactual for an identical time point, etc. Would you add anything further?

My reply: it could be ok, it depends on the context. One point that Rubin has made repeatedly over the past few decades is that inference depends on a model. With a clean, completely randomized design, you don't need much model to get inferences. A crossover design is more complicated. If you make some assumptions about how the treatment at time 1 affects the outcome after time 2, then you can go from there.

To put it another way, the full Bayesian analysis always conditions on all information. Whether this looks like "controlling" for an x-variable, in a regression sense, depends on the model that you're using.

Tyler Cowen links to a blog by Paul Kedrosky that asks why winning times in the Boston marathon have been more variable, in recent years, than winning times in New York. This particular question isn't so interesting--when I saw the title of the post, my first thought was "the weather," and, in fact, that and "the wind" are the most common responses of the blog commenters--but it reminded me of a more general question that we discussed the other day, which is how to think about Why questions.

Many years ago, Don Rubin convinced me that it's a lot easier to think about "effects of causes" than "causes of effects." For example, why did my cat die? Because she ran into the street, because a car was going too fast, because the driver wasn't paying attention, because a bird distracted the cat, because the rain stopped so the cat went outside, etc. When you look at it this way, the question of "why" is pretty meaningless.

Similarly, if you ask a question such as, What caused World War 1, the best sort of answers can take the form of potential-outcomes analyses. I don't think it makes sense to expect any sort of true causal answer here.

But, now let's get back to the "volatility of the Boston marathon" problem. Unlike the question of "why did my cat die" or "why did World War 1 start," the question, "Why have the winning times in the Boston marathon been so variable" does seem answerable.

What happens if we try to apply some statistical principles here?

Principle #1: Compared to what? We can't try to answer "why" without knowing what we are comparing to. This principle seems to work in the marathon-times example. The only way to talk about the Boston times as being unexpectedly variable is to know what "expectedly variable" is. Or, conversely, the New York times are unexpectedly stable compared to what was happening in Boston those same years. Either way, the principle holds that we are comparing to some model or another.

Principle #2: Look at effects of causes, rather than causes of effects. This principle seems to break down in marathon example, where it seems very natural to try to understand why an observed phenomenon is occurring.

What's going on? Perhaps we can understand in the context of another example, something that came up a couple years ago in some of my consulting work. The New York City Department of Health had a survey of rodent infestation, and they found that African Americans and Latinos were more likely than whites to have rodents in their apartments. This difference persisted (albeit at a lesser magnitude) after controlling for some individual and neighborhood-level predictors. Why does this gap remain? What other average differences are there among the dwellings of different ethnic groups?

OK, so now maybe we're getting somewhere. The question on deck now is, how do the "Boston vs. NY marathon" and "too many rodents" problems differ from the "dead cat" problem.

One difference is that we have data on lots of marathons and lots of rodents in apartments, but only one dead cat. But that doesn't quite work as a demarcation criterion (sorry, forgive me for working under the influence of Popper): even if there were only one running of each marathon, we could still quite reasonably answer questions such as, "Why was the winning time so much lower in NY than in Boston?" And, conversely, if we had lots of dead cats, we could start asking questions about attributable risks, but it still wouldn't quite make sense to ask why the cats are dying.

Another difference is that the marathon question and the roach question are comparisons (NY vs. Boston and blacks/hispanics vs. whites), while the dead cat stands alone (or swings alone, I guess I should say). Maybe this is closer to the demarcation we're looking for, the idea being that a "cause" (in this sense) is something that takes you away from some default model. In these examples, it's a model of zero differences between groups, but more generally it could be any model that gives predictions for data.

In this model-checking sense, the search for a cause is motivated by an itch--a disagreement with a default model--which has to be scratched and scratched until the discomfort goes away, by constructing a model that fits the data. Said model can then be interpreted causally in a Rubin-like, "effects of causes," forward-thinking way.

Is this the resolution I'm seeking? I'm not sure. But I need to figure this out, because I'm planning on basing my new intro stat course (and book) on the idea of statistics as comparisons.

P.S. I remain completely uninterested in questions such as, What is the cause? Is it A or is it B? (For example, what caused the differences in marathon-time variations in Boston and New York--is it the temperature, the precipitation, the wind, or something else? Of course if it can be any of these factors, it can be all of them. I remain firm in my belief that any statistical method that claims to distinguish between hypotheses in this way is really just using sampling variation as a way to draw artificial distinctions, fundamentally in a way no different from the notorious comparisons of statistical significance to non-significance.

This last point has nothing to do with causal inference and everything to do with my preference for continuous over discrete models in applications in which I've worked in social science, environmental science, and public health.

So go read the stuff on the main page now before it scrolls off your screens.

No, not me. It's somebody else. Story here.

Joshua Clover's fun new book, 1989, features a blurb from Luc Sante, author some years back of the instant-classic, Low Life. 1989 has some similarities to Low Life--both are about culture and politics--but Clover is much more explicit in making his connections, whereas Sante left most of his implications unsaid. I read Low Life when it came out, and I immediately felt: This-book-is-awesome-and-I get-it-but-nobody-else-will. I think, actually, that just about everybody who read Low Life had that same reaction, which is what being a "cult classic" is all about.

1989 was a bit of a nostalgia-fest for me, at least in the chapter about rap. (I'm not really familiar enough with the other musical styles of that era, so the other chapters were harder for me to follow. I read Cauty and Drummond's "The Manual" (another cult classic, I think) several years ago but only had a vague sense of them and I've never thought of taking their music seriously as some sort of cultural indicator.)

I remember when I first heard Straight Outta Compton, how dense and echoing the sound was, an intensity comparable (in my view) to the movie Salvador.

Judea Pearl sends along this article and writes:

This research note was triggered by a discussant on your blog, who called my attention to Wooldridge's paper, in response to my provocative question: "Has anyone seen a proof that adjusting for intemediary would introduce bias?"

It led to some interesting observations which I am now glad to share with your bloggers.

As Pearl writes, it is standard advice to adjust for all pre-treatment variables in an experiment or observational study--and I think everyone would agree on the point--but exactly how to "adjust" is not always clear. For example, you wouldn't want to just throw an instrumental variable as another regression predictor. And this then leads to tricky questions of what to do with variables that are sort-of instruments and sort-of covariates. I don't really know what to do in such situations, and maybe Pearl and Woolridge are pointing toward a useful way forward.

I'm curious what Rosenbaum and Rubin (both of whom are cited in Pearl's article) have to say about this paper. And, of course, WWJD.

Helen DeWitt links to a blog by John Scalzi pointing out that today's science fiction magazine writers get the same rate--6 cents per word--at F. Scott Fitzgerald did tor his short stories in 1920. After correcting for inflation, this means Fitzgerald was paid 20 times as much.

Scalzi writes that this "basically sucks." But I'd frame this somewhat differently. After all, this is F. Scott Fitzgerald we're talking about. I'd guess he really is worth at least 20 times per word what the authors of articles in Fantasy & Science Fiction etc.

P.S. As a blogger, my word rate is 0 cents, of course.

1. Understanding the 'Russian Mortality Paradox' in Central Asia: Evidence from Kyrgyzstan

Short answer: alcohol and suicide.

2. Lumberjacks as a counterexample to the idea of a "risk premium"

They take lots of risks and don't get paid well for it.

3. Cell size and scale

This is a visualization you won't want to miss.

4. Three guys named Matt

5. The political philosophy of the private eye

A genre that was rendered obsolete in 1961 (but nobody realizes it).

Tyler Cowen writes:

Andrew Gelman will have a second blog. I don't yet understand the forthcoming principle of individuation across the two blogs.

I don't like the term "risk aversion" (see here and here). For a long time I've been meaning to write something longer and more systematic on the topic, but every once in awhile I see something that reminds me of the slipperiness of the topic.

For example, Alex Tabarrok asks, "Why are Americans more risk averse about medicine than Europeans?" It's a good question, and it's something I've wondered about myself. But I don't know what he's talking about when he says that "the stereotype is that Americans are more risk-loving" than Europeans. Huh? Americans are notorious for worrying about risks, with car seats, bike helmets, high railings on any possible place where someone could fall, Purell bottles everywhere, etc etc. The commenters on Alex's blog are all talking about drug company regulations, but it seems like a broader cultural thing to me.

But I'm bothered by the term "risk aversion." Why exactly is it appropriate to refer to strict rules on drug approvals as "risk averse"? In a general English-language use of the words, I understand it, but it gets slippery when you try to express it more formally.

Asa writes:

I took your class on multilevel models last year and have since found myself applying them in several different contexts. I am about to start a new project with a dataset in the tens of millions of observations. In my experience, multilevel modeling has been most important when the number of observations in at least one subgroup of interest is small. Getting started on this project, I have two questions:

1) Do multilevel models still have the potential to add much accuracy to predictions when n is very large in all subgroups of interest?

2) Do you find SAS, STATA, or R to be more efficient at handling multilevel/"mixed effects" models with such a large dataset (wont be needing any logit/poisson/glm models)?

My reply:

Regarding software, I'm not sure, but my guess is that Stata might be best with large datasets. Stata also has an active user community that can help with such questions.

For your second question, if n is large in all subgroups, then multilevel modeling is typically not needed. But if n is large in all subgroups, you can simply fit a separate model in each group. That is equivalent to a full-interaction model. At that point you might be interested in details within subgroups, and then you might want a multilevel model.

Asa then wrote:

Yes, a "full interaction" model was the alternative I was thinking of. And yes, I can imagine the results from that model raising further questions about whats going on within groups as well.

My previous guess was that SAS would be the most efficient for multilevel modeling with big data. But I just completely wrecked my (albeit early 2000's era) laptop looping proc mixed a bunch of times with a much smaller dataset.

I don't really know on the SAS vs. Stata issue. In general, I have warmer feelings toward Stata than SAS, but, on any particular problem, who knows? I'm pretty sure that R would choke on any of these problems.

On the other hand, if you end up breaking the problem into smaller pieces anyway, maybe the slowness of R wouldn't be so much of a problem. R does have the advantage of flexibility.

Aleks sends along this amusing news article by Jennifer Levitz:

A new study found that rates of marriage outside the faith were sharply curbed among young Jews who have taken "birthright" trips to Israel . . . Over the past decade, Taglit-Birthright Israel, a U.S. nonprofit founded by Jewish businessmen, has sponsored nearly 225,000 young Jewish adults for free 10-day educational tours of Israel as a way to foster Jewish identity. . . .

A study [by Brandeis University researcher Leonard Saxe and partly funded by Taglit-Birthright] showed that 72% of those who went on the trip married within the faith, compared with 46% of people who applied for the trip but weren't selected in a lottery. . . . The Brandeis study looked at 1,500 non-Orthodox Jewish adults who took Taglit trips or applied for one between 2001 and 2004. . . . The Brandeis study looked at 1,500 non-Orthodox Jewish adults who took Taglit trips or applied for one between 2001 and 2004.

The article also said that 10,000 people participated in these trips last summer, which suggests that the 1,500 people in the research study represent a very small fraction of the participants from 2001-2004. I have no idea if this is a random sample, or what. Also I wonder about the people who participated in the lottery, were selected, but didn't go on the trip. Excluding these people (if there are many of them) could bias the results. The news article unfortunately doesn't link to any research report.

Philip Stark sent along this set of calculations on the probability that the hidden message in Gov. Schwartzenegger's message could've occurred by chance. The message, if you haven't heard, is:

The questions are no big deal, but what I find interesting is that medical school do personal interviews at all. No place where I've ever worked has interviewed grad school applicants. It's hard for me to see what you get from it, that it would be worth the cost. I guess there must be quite a bit of psychology literature on this question.

Christiaan de Leeuw writes:

I write to you with a question about the construction of informative priors in Bayesian analysis. Since most Bayesians at the statistics department here are more of the 'Objective' Bayes persuasion, I wanted some outside opinions as well.

Jay Kaufman writes:

I received the following email:

Hello, my name is Lauren Schmidt, and I recently graduated from the Brain & Cognitive Sciences graduate program at MIT, where I spent a lot of time doing online research using human subjects. I also spent a lot of time being frustrated with the limitations of various existing online research tools. So now I am co-founding a start-up, HeadLamp Research, with the goal of making online experimental design and data collection as fast, easy, powerful, and painless as can be. But we need your help to come up with an online research tool that is as useful as possible!

We have a short survey (5-10 min) on your research practices and needs, and we would really appreciate your input if you are interested in online data collection.

I imagine they're planning to make money off this start-up and so I think it would be only fair if they pay their survey participants. Perhaps they can give them a share of the profits, if any exist?

Why we (usually) don't worry about multiple comparisons

Culture wars, voting and polarization: divisions and unities in modern American politics

Guilherme Rocha writes:

Here. Official opening is Monday but youall get to see it earlier.

Matt Stephenson writes:

Tuesday 3 Nov, 4-5:30pm in Room R505, Department of Government, LSE.

Culture wars, voting and polarization: divisions and unities in modern American politics

On the night of the 2000 presidential election, Americans sat riveted in front of their televisions as polling results divided the nation's map into red and blue states. Since then the color divide has become a symbol of a culture war that thrives on stereotypes--pickup-driving red-state Republicans who vote based on God, guns, and gays; and elitist, latte-sipping blue-state Democrats who are woefully out of touch with heartland values. But how does this fit into other ideas about America being divided between the haves and the have-nots? Is political polarization real, or is the real concern the perception of polarization?

This work is joint with David Park, Boris Shor, Joseph Bafumi, Jeronimo Cortina, and Delia Baldassarri.

(Here's a video version of the talk, from when I gave it at Google.)

I'll be interested to see if people can explain to me the relevance (or lack thereof) of this work to politics in Britain and other countries.

P.S. I'm speaking at LSE on Monday also (on a different topic).

P.P.S. I'll be speaking again a couple times in London later in the academic year, but on other topics. All my talks there will be different.

Monday 2 Nov, 5-6:30pm at the Methodology Institute, LSE. No link to the seminar on the webpage, so I'll give you the information here:

Why we (usually) don't worry about multiple comparisons

Applied researchers often find themselves making statistical inferences in settings that would seem to require multiple comparisons adjustments. We challenge the Type I error paradigm that underlies these corrections. Moreover we posit that the problem of multiple comparisons can disappear entirely when viewed from a hierarchical Bayesian perspective. We propose building multilevel models in the settings where multiple comparisons arise.

Multilevel models perform partial pooling (shifting estimates toward each other), whereas classical procedures typically keep the centers of intervals stationary, adjusting for multiple comparisons by making the intervals wider (or, equivalently, adjusting the $p$-values corresponding to intervals of fixed width). Thus, multilevel models address the multiple comparisons problem and also yield more efficient estimates, especially in settings with low group-level variation, which is where multiple comparisons are a particular concern.

This work is joint with Jennifer Hill and Masanao Yajima.

(Here's a video version of a related talk that I gave at a meeting on statistics and neuroscience.)

P.S. My talk briefly touches upon some work done by a researcher at the London School of Economics!

P.P.S. I'm speaking at LSE on Tuesday also (on a different topic).

P.P.P.S. I'll be speaking again a couple times in London later in the academic year, but on other topics. All my talks there will be different.

I just today learned about an organization called SourceWatch--they have an article on the tobacco connections of the well-known sociologist Peter Berger. Beyond the inherent interest of the topic, I was fascinated by the way that the Sourcewatch webpage mimicked Wikipedia:

This is a smart move, I think: for better or worse, Wikipedia is generally considered to be authoritative.

But then I was thinking . . . is this the beginning of the end for Wikipedia. I don't know anything about Sourcewatch, if they're good guys or bad guys or whatever--but if they can mimic Wikipedia, I'm sure lots of other organizations could do so too. And, when they do it, all of a sudden there will be a lot of authoritative-looking Wikipedia-like pages floating around, a sort of counterfeit money devaluing the "real" 'pedia, which will then have to respond by branding itself--"100% real Wikipedia, accept no imitations"--and so on. Not a bad thing, perhaps, but not what we have now.

From Aaron Swartz, a link stating that famous sociologist Peter L. Berger was a big-time consultant for the Tobacco Insitute:

Peter L. Berger is an academic social philosopher and sociologist who served as a consultant to the tobacco industry starting with the industry's original 1979 Social Costs/Social Values Project (SC/SV). According to a 1980 International Committee on Smoking Issues/Social Acceptability Working Party (International Committee on Smoking Issues/SAWP) progress report, Berger's primary assignment was "to demonstrate clearly that anti-smoking activists have a special agenda which serves their own purposes, but not necessarily the majority of nonsmokers."

Masanao sends in this.

(The oven that Masanao was referring to is described here.)

Not any more.

Kavalier and Clay impressed me right away, even before I ever even held a copy in my hand, because for a year or so after it came out, I kept seeing people reading it. In the subway, in the park, everywhere. This was a book that people really wanted to read. So I bought it and read it and was duly impressed. It's a great book: DeLillo without the irony, if you will. The book Don DeLillo might have written had he been Michael Chabon and had the interests Chabon had instead of the interests that DeLillo has. Whatever.

More recently, I read (most of) Chabon's first book of stories, A Model World. I was into it for about a story and a half, and then I realized that these were all John Updike stories. Don't get me wrong here--Updike is my hero, and it's pretty impressive to me that he continues to write stories even after he's no longer around. And i don't really hold it against him (Updike) that he, like Gore Vidal, couldn't come up with good book titles. Chabon, though, he's a good titler. But I couldn't get into his book. It was just too weird that he was writing Updike stories.

But then I read Chabon's second book of stories, Werewolves in their Youth. Much better. Good to see someone getting better at what he does.

Lane Kenworthy, Yu-Sung Su, and I write:

Income inequality in the United States has risen during the past several decades. Has this produced an increase in partisan voting differences between rich and poor? We examine trends from the 1940s through the 2000s in the country as a whole and in the states. We find no clear relation between income inequality and class-based voting.

This article will appear in a special issue of Social Science Quarterly on the topic of "Inequality and Poverty: American and International Perspectives." We have some pretty graphs, some of which appeared in the Red State, Blue State book and some of which didn't.

P.S. "We find no clear relation . . .": That works great in an academic article but I don't think we'll be grabbing the headlines anytime soon.

After reading what Christian wrote about the solutions manual that he and his collaborator wrote for their Bayesian Core book, I'm reminded of my own efforts with the Bayesian Data Analysis solutions. Not long after the first edition of the book came out, nearly fifteen years ago, I wrote up solutions to fifty of the homework problems (out of a total of about 150). People sometimes ask for more solutions, but the difficulty is that, once you have official solutions, you want them to be correct, you want them to be clear, and you want them to illustrate good statistical practice. It's a lot of work. Somehow I was able to write up those fifty, back when I had more time on my hands, but, really, writing up another fifty would almost be the equivalent of writing a (short) book! Originally I thought I could quickly put together a complete or nearly complete set by gathering solutions from students, or people just emailing them in, but I quickly realized that this wouldn't work. I think it would be ok to post scanned-in versions of student solutions, but once I start typing them up, I need them to be cleaner, and that takes work. That's one reason I didn't even try to write a solution set for ARM.

In suggesting "a socially responsible method of announcing associations," AT points out that, as much as we try to be rigorous about causal inference, assumptions slip in through our language:

The trouble is, causal claims have an order to them (like "aliens cause cancer"), and so do most if not all human sentences ("I like ice cream"). It's all too tempting to read a non-directional association claim as if it were so -- my (least) favourite was a radio blowhard who said that in teens, cellphone use was linked with sexual activity, and without skipping a beat angrily proclaimed that giving kids a cell phone was tantamount to exposing them to STDs. . . . So here's a modest proposal: when possible, beat back the causal assumption by presenting an associational idea in the order least likely to be given a causal interpretation by a layperson or radio host.

Here's AT's example:

A random Google News headline reads: "Prolonged Use of Pacifier Linked to Speech Problems" and strongly implies a cause and effect relationship, despite the (weak) disclaimer from the quoted authors. Reverse that and you've got "Speech Problems linked to Prolonged Use of Pacifier" which is less insinuating.

It's an interesting idea, and it reminds me of something that really bugs me.

Phil Turk wrote: