This is the complete 'working lunch' talk given Jan 14, 2005. I'm marking it up for emphasis and organization; here's how far I've got on my second pass.
Remarks at NBER Conference on Diversifying the Science & Engineering Workforce
Lawrence H. Summers
January 14, 2005
[ Emphasis mine. – fsteele ]
I asked Richard, when he invited me to come here and speak, whether he wanted an institutional talk about Harvard's policies toward diversity or whether he wanted some questions asked and some attempts at provocation, because I was willing to do the second and didn't feel like doing the first. And so we have agreed that I am speaking unofficially and not using this as an occasion to lay out the many things we're doing at Harvard to promote the crucial objective of diversity. There are many aspects of the problems you're discussing and it seems to me they're all very important from a national point of view. I'm going to confine myself to addressing one portion of the problem, or of the challenge we're discussing, which is the issue of women's representation in tenured positions in science and engineering at top universities and research institutions, not because that's necessarily the most important problem or the most interesting problem, but because it's the only one of these problems that I've made an effort to think in a very serious way about. The other prefatory comment that I would make is that I am going to, until most of the way through, attempt to adopt an entirely positive, rather than normative approach, and just try to think about and offer some hypotheses as to why we observe what we observe without seeing this through the kind of judgmental tendency that inevitably is connected with all our common goals of equality. It is after all not the case that the role of women in science is the only example of a group that is significantly underrepresented in an important activity and whose underrepresentation contributes to a shortage of role models for others who are considering being in that group. To take a set of diverse examples, the data will, I am confident, reveal that Catholics are substantially underrepresented in investment banking, which is an enormously high-paying profession in our society; that white men are very substantially underrepresented in the National Basketball Association; and that Jews are very substantially underrepresented in farming and in agriculture. These are all phenomena in which one observes underrepresentation, and I think it's important to try to think systematically and clinically about the reasons for underrepresentation.
There are three broad hypotheses about the sources of the very substantial disparities that this conference's papers document and have been documented before with respect to the presence of women in high-end scientific professions. One is what I would call the-I'll explain each of these in a few moments and comment on how important I think they are-the first is what I call the high-powered job hypothesis. The second is what I would call different availability of aptitude at the high end, and the third is what I would call different socialization and patterns of discrimination in a search. And in my own view, their importance probably ranks in exactly the order that I just described.
[ Re Hypothesis 1, ‘high-powered job’ ]
Maybe it would be helpful to just, for a moment, broaden the problem, or the issue, beyond science and engineering. I've had the opportunity to discuss questions like this with chief executive officers at major corporations, the managing partners of large law firms, the directors of prominent teaching hospitals, and with the leaders of other prominent professional service organizations, as well as with colleagues in higher education. In all of those groups, the story is fundamentally the same. Twenty or twenty-five years ago, we started to see very substantial increases in the number of women who were in graduate school in this field. Now the people who went to graduate school when that started are forty, forty-five, fifty years old. If you look at the top cohort in our activity, it is not only nothing like fifty-fifty, it is nothing like what we thought it was when we started having a third of the women, a third of the law school class being female, twenty or twenty-five years ago. And the relatively few women who are in the highest ranking places are disproportionately either unmarried or without children, with the emphasis differing depending on just who you talk to. And that is a reality that is present and that one has exactly the same conversation in almost any high-powered profession. What does one make of that? I think it is hard-and again, I am speaking completely descriptively and non-normatively-to say that there are many professions and many activities, and the most prestigious activities in our society expect of people who are going to rise to leadership positions in their forties near total commitments to their work. They expect a large number of hours in the office, they expect a flexibility of schedules to respond to contingency, they expect a continuity of effort through the life cycle, and they expect-and this is harder to measure-but they expect that the mind is always working on the problems that are in the job, even when the job is not taking place. And it is a fact about our society that that is a level of commitment that a much higher fraction of married men have been historically prepared to make than of married women. That's not a judgment about how it should be, not a judgment about what they should expect. But it seems to me that it is very hard to look at the data and escape the conclusion that that expectation is meeting with the choices that people make and is contributing substantially to the outcomes that we observe. One can put it differently. Of a class, and the work that Claudia Goldin and Larry Katz are doing will, I'm sure, over time, contribute greatly to our understanding of these issues and for all I know may prove my conjectures completely wrong. Another way to put the point is to say, what fraction of young women in their mid-twenties make a decision that they don't want to have a job that they think about eighty hours a week. What fraction of young men make a decision that they're unwilling to have a job that they think about eighty hours a week, and to observe what the difference is. And that has got to be a large part of what is observed. Now that begs entirely the normative questions-which I'll get to a little later-of, is our society right to expect that level of effort from people who hold the most prominent jobs? Is our society right to have familial arrangements in which women are asked to make that choice and asked more to make that choice than men? Is our society right to ask of anybody to have a prominent job at this level of intensity, and I think those are all questions that I want to come back to. But it seems to me that it is impossible to look at this pattern and look at its pervasiveness and not conclude that something of the sort that I am describing has to be of significant importance. To buttress conviction and theory with anecdote, a young woman who worked very closely with me at the Treasury and who has subsequently gone on to work at Google highly successfully, is a 1994 graduate of
[ Re Hypothesis 2, Availability ]
The second thing that I think one has to recognize is present is what I would call the combination of, and here, I'm focusing on something that would seek to answer the question of why is the pattern different in science and engineering, and why is the representation even lower and more problematic in science and engineering than it is in other fields. And here, you can get a fair distance, it seems to me, looking at a relatively simple hypothesis. It does appear that on many, many different human attributes-height, weight, propensity for criminality, overall IQ, mathematical ability, scientific ability-there is relatively clear evidence that whatever the difference in means-which can be debated-there is a difference in the standard deviation, and variability of a male and a female population. And that is true with respect to attributes that are and are not plausibly, culturally determined. If one supposes, as I think is reasonable, that if one is talking about physicists at a top twenty-five research university, one is not talking about people who are two standard deviations above the mean. And perhaps it's not even talking about somebody who is three standard deviations above the mean. But it's talking about people who are three and a half, four standard deviations above the mean in the one in 5,000, one in 10,000 class. Even small differences in the standard deviation will translate into very large differences in the available pool substantially out. I did a very crude calculation, which I'm sure was wrong and certainly was unsubtle, twenty different ways. I looked at the Xie and Shauman paper-looked at the book, rather-looked at the evidence on the sex ratios in the top 5% of twelfth graders. If you look at those-they're all over the map, depends on which test, whether it's math, or science, and so forth-but 50% women, one woman for every two men, would be a high-end estimate from their estimates. From that, you can back out a difference in the implied standard deviations that works out to be about 20%. And from that, you can work out the difference out several standard deviations. If you do that calculation-and I have no reason to think that it couldn't be refined in a hundred ways-you get five to one, at the high end. Now, it's pointed out by one of the papers at this conference that these tests are not a very good measure and are not highly predictive with respect to people's ability to do that. And that's absolutely right. But I don't think that resolves the issue at all. Because if my reading of the data is right-it's something people can argue about-that there are some systematic differences in variability in different populations, then whatever the set of attributes are that are precisely defined to correlate with being an aeronautical engineer at MIT or being a chemist at Berkeley, those are probably different in their standard deviations as well. So my sense is that the unfortunate truth-I would far prefer to believe something else, because it would be easier to address what is surely a serious social problem if something else were true-is that the combination of the high-powered job hypothesis and the differing variances probably explains a fair amount of this problem.
There may also be elements, by the way, of differing, there is some, particularly in some attributes, that bear on engineering, there is reasonably strong evidence of taste differences between little girls and little boys that are not easy to attribute to socialization.
I just returned from Israel, where we had the opportunity to visit a kibbutz, and to spend some time talking about the history of the kibbutz movement, and it is really very striking to hear how the movement started with an absolute commitment, of a kind one doesn't encounter in other places, that everybody was going to do the same jobs. Sometimes the women were going to fix the tractors, and the men were going to work in the nurseries, sometimes the men were going to fix the tractors and the women were going to work in the nurseries, and just under the pressure of what everyone wanted, in a hundred different kibbutzes, each one of which evolved, it all moved in the same direction. So, I think, while I would prefer to believe otherwise, I guess my experience with my two and a half year old twin daughters who were not given dolls and who were given trucks, and found themselves saying to each other, look, daddy truck is carrying the baby truck, tells me something. And I think it's just something that you probably have to recognize
. There are two other hypotheses that are all over. One is socialization. Somehow little girls are all socialized towards nursing and little boys are socialized towards building bridges. No doubt there is some truth in that. I would be hesitant about assigning too much weight to that hypothesis for two reasons.
First, most of what we've learned from empirical psychology in the last fifteen years has been that people naturally attribute things to socialization that are in fact not attributable to socialization. We've been astounded by the results of separated twins studies. The confident assertions that autism was a reflection of parental characteristics that were absolutely supported and that people knew from years of observational evidence have now been proven to be wrong. And so, the human mind has a tendency to grab to the socialization hypothesis when you can see it, and it often turns out not to be true.
The second empirical problem is that girls are persisting longer and longer. When there were no girls majoring in chemistry, when there were no girls majoring in biology, it was much easier to blame parental socialization. Then, as we are increasingly finding today, the problem is what's happening when people are twenty, or when people are twenty-five, in terms of their patterns, with which they drop out. Again, to the extent it can be addressed, it's a terrific thing to address.
[ Re Hypothesis 3, Discrimination in hiring ]
The most controversial in a way, question, and the most difficult question to judge, is what is the role of discrimination? To what extent is there overt discrimination? Surely there is some. Much more tellingly, to what extent are there pervasive patterns of passive discrimination and stereotyping in which people like to choose people like themselves, and the people in the previous group are disproportionately white male, and so they choose people who are like themselves, who are disproportionately white male. No one who's been in a university department or who has been involved in personnel processes can deny that this kind of taste does go on, and it is something that happens, and it is something that absolutely, vigorously needs to be combated. On the other hand, I think before regarding it as pervasive, and as the dominant explanation of the patterns we observe, there are two points that should make one hesitate.
The first is the fallacy of composition. No doubt it is true that if any one institution makes a major effort to focus on reducing stereotyping, on achieving diversity, on hiring more people, no doubt it can succeed in hiring more. But each person it hires will come from a different institution, and so everyone observes that when an institution works very hard at this, to some extent they are able to produce better results. If I stand up at a football game and everybody else is sitting down, I can see much better, but if everybody stands up, the views may get a little better, but they don't get a lot better. And there's a real question as to how plausible it is to believe that there is anything like half as many people who are qualified to be scientists at top ten schools and who are now not at top ten schools, and that's the argument that one has to make in thinking about this as a national problem rather than an individual institutional problem.
The second problem is the one that Gary Becker very powerfully pointed out in addressing racial discrimination many years ago. If it was really the case that everybody was discriminating, there would be very substantial opportunities for a limited number of people who were not prepared to discriminate to assemble remarkable departments of high quality people at relatively limited cost simply by the act of their not discriminating, because of what it would mean for the pool that was available. And there are certainly examples of institutions that have focused on increasing their diversity to their substantial benefit, but if there was really a pervasive pattern of discrimination that was leaving an extraordinary number of high-quality potential candidates behind, one suspects that in the highly competitive academic marketplace, there would be more examples of institutions that succeeded substantially by working to fill the gap. And I think one sees relatively little evidence of that.
[ Re relative importance ]
So my best guess, to provoke you, of what's behind all of this is that the largest phenomenon, by far, is the general clash between people's legitimate family desires and employers' current desire for high power and high intensity, that in the special case of science and engineering, there are issues of intrinsic aptitude, and particularly of the variability of aptitude, and that those considerations are reinforced by what are in fact lesser factors involving socialization and continuing discrimination. I would like nothing better than to be proved wrong, because I would like nothing better than for these problems to be addressable simply by everybody understanding what they are, and working very hard to address them.
What's to be done? And what further questions should one know the answers to? Let me take a second, first to just remark on a few questions that it seems to me are ripe for research, and for all I know, some of them have been researched.
First, it would be very useful to know, with hard data, what the quality of marginal hires are when major diversity efforts are mounted. When major diversity efforts are mounted, and consciousness is raised, and special efforts are made, and you look five years later at the quality of the people who have been hired during that period, how many are there who have turned out to be much better than the institutional norm who wouldn't have been found without a greater search. And how many of them are plausible compromises that aren't unreasonable, and how many of them are what the right-wing critics of all of this suppose represent clear abandonments of quality standards. I don't know the answer, but I think if people want to move the world on this question, they have to be willing to ask the question in ways that could face any possible answer that came out.
Second, and by the way, I think a more systematic effort to look at citation records of male and female scholars in disciplines where citations are relatively well-correlated with academic rank and with people's judgments of quality would be very valuable. Of course, most of the critiques of citations go to reasons why they should not be useful in judging an individual scholar. Most of them are not reasons why they would not be useful in comparing two large groups of scholars and so there is significant potential, it seems to me, for citation analysis in this regard. Second, what about objective versus subjective factors in hiring? I've been exposed, by those who want to see the university hiring practices changed to favor women more and to assure more diversity, to two very different views. One group has urged that we make the processes consistently more clear-cut and objective, based on papers, numbers of papers published, numbers of articles cited, objectivity, measurement of performance, no judgments of potential, no reference to other things, because if it's made more objective, the subjectivity that is associated with discrimination and which invariably works to the disadvantage of minority groups will not be present. I've also been exposed to exactly the opposite view, that those criteria and those objective criteria systematically bias the comparisons away from many attributes that those who contribute to the diversity have: a greater sense of collegiality, a greater sense of institutional responsibility. Somebody ought to be able to figure out the answer to the question of, if you did it more objectively versus less objectively, what would happen. Then you can debate whether you should or whether you shouldn't, if objective or subjective is better. But that question ought to be a question that has an answer, that people can find. Third, the third kind of question is, what do we know about search procedures in universities? Is it the case that more systematic comprehensive search processes lead to minority group members who otherwise would have not been noticed being noticed? Or does fetishizing the search procedure make it very difficult to pursue the targets of opportunity that are often available arising out of particular family situations or particular moments, and does fetishizing and formalizing search procedures further actually work to the disadvantage of minority group members. Again, everybody's got an opinion; I don't think anybody actually has a clue as to what the answer is. Fourth, what do we actually know about the incidence of financial incentives and other support for child care in terms of what happens to people's career patterns. I've been struck at Harvard that there's something unfortunate and ironic about the fact that if you're a faculty member and you have a kid who's 18 who goes to college, we in effect, through an interest-free loan, give you about $9,000. If you have a six-year-old, we give you nothing. And I don't think we're very different from most other universities in this regard, but there is something odd about that strategic choice, if the goal is to recruit people to come to the university. But I don't think we know much about the child care issue. The fifth question-which it seems to me would be useful to study and to actually learn the answer to-is what do we know, or what can we learn, about the costs of career interruptions. There is something we would like to believe. We would like to believe that you can take a year off, or two years off, or three years off, or be half-time for five years, and it affects your productivity during the time, but that it really doesn't have any fundamental effect on the career path. And a whole set of conclusions would follow from that in terms of flexible work arrangements and so forth. And the question is, in what areas of academic life and in what ways is it actually true. Somebody reported to me on a study that they found, I don't remember who had told me about this-maybe it was you, Richard-that there was a very clear correlation between the average length of time, from the time a paper was cited. That is, in fields where the average papers cited had been written nine months ago, women had a much harder time than in fields where the average thing cited had been written ten years ago. And that is suggestive in this regard. On the discouraging side of it, someone remarked once that no economist who had gone to work at the President's Council of Economic Advisors for two years had done highly important academic work after they returned. Now, I'm sure there are counterexamples to that, and I'm sure people are kind of processing that Tobin's Q is the best-known counterexample to that proposition, and there are obviously different kinds of effects that happen from working in
Let me just conclude by saying that I've given you my best guesses after a fair amount of reading the literature and a lot of talking to people. They may be all wrong. I will have served my purpose if I have provoked thought on this question and provoked the marshalling of evidence to contradict what I have said. But I think we all need to be thinking very hard about how to do better on these issues and that they are too important to sentimentalize rather than to think about in as rigorous and careful ways as we can. That's why I think conferences like this are very, very valuable. Thank you.
Questions and Answers
Q: Well, I don't want to take up much time because I know other people have questions, so, first of all I'd like to say thank you for your input. It's very interesting-I noticed it's being recorded so I hope that we'll be able to have a copy of it. That would be nice.
LHS: We'll see. (LAUGHTER)
Q: Secondly, you make a point, which I very much agree with, that this is a wonderful opportunity for other universities to hire women and minorities, and you said you didn't have an example of an instance in which that is being done. The chemistry department at
LHS: I don't think that. I don't actually think that's the point at all. My point was a very different one. My point was simply that the field of behavioral genetics had a revolution in the last fifteen years, and the principal thrust of that revolution was the discovery that a large number of things that people thought were due to socialization weren't, and were in fact due to more intrinsic human nature, and that set of discoveries, it seemed to me, ought to influence the way one thought about other areas where there was a perception of the importance of socialization. I wasn't at all trying to connect those studies to the particular experiences of women and minorities who were thinking about academic careers.
Q: Raising that particular issue, as a biologist, I neither believe in all genetic or all environment, that in fact behavior in any other country actually develops [unintelligible] interaction of those aspects. And I agree with you, in fact, that it is wrong-headed to just dismiss the biology. But to put too much weight to it is also incredibly wrong-headed, given the fact that had people actually had different kinds of opportunities, and different opportunities for socialization, there is good evidence to indicate in fact that it would have had different outcomes. I cite by way of research the [unintelligible] project in North Carolina, which essentially shows that, where every indicator with regard to mother's education, socioeconomic status, et cetera, would have left a kid in a particular place educationally, that, essentially, they are seeing totally different outcomes with regard to performance, being referred to special education, et cetera, so I think that there is some evidence on that particular side. The other issue is this whole question about objective versus subjective. I think that it is very difficult to have anything that is basically objective, and the work of [unintelligible] I think point out that in a case where you are actually trying to-this case from the Swedish Medical Council, where they were trying to identify very high-powered research opportunities for, I guess it was post-docs by that point, that indicated that essentially that it ended up with larger numbers of men than women. Two of the women who were basically in the affected group were able to utilize the transparency rules that were in place in Sweden, get access to the data, get access to the issues, and in fact, discovered that it was not as objective as everyone claimed, and that in fact, different standards were actually being used for the women as well as for the men, including the men's presence in sort of a central network, the kinds of journals that they had to publish in to be considered at the same level, so I think that there are pieces of research that begin to actually relate to this-yes, there is the need to look more carefully at a lot of these areas. I would-in addition looking at this whole question of the quality of marginal hires-I would also like to look at the quality of class one hires, in terms of seeing who disappoints, and what it was that they happened to be looking at and making judgments on, and then what the people could not deliver. So I think that there is a real great need on both sides to begin to talk about whether or not we can predict. I hate to use a sports metaphor, but I will. This is drawn basically from an example from Claude Steele, where he says, he starts by using free throws as a way of actually determining, who should-you've got to field a basketball team, and you clearly want the people who make ten out of ten, and you say, "Well, I may not want the people who make zero out of ten," but what about the people who make four out of ten. If you use that as the measure, Shaq will be left on the sidelines.
LHS: I understand. I think you're obviously right that there's no absolute objectivity, and you're-there's no question about that. My own instincts actually are that you could go wrong in a number of respects fetishizing objectivity for exactly the reasons that you suggest. There is a very simple and straightforward methodology that was used many years ago in the case of baseball. Somebody wrote a very powerful article about baseball, probably in the seventies, in which they basically said, "Look, it is true that if you look at people's salaries, and you control for their batting averages and their fielding averages and whatnot, whites and blacks are in the same salary once you control. It is also true that there are no black .240 hitters in the major leagues, that the only blacks who are in the major leagues are people who bat over .300-I'm exaggerating-and that is exactly what you'd predict on a model of discrimination, that because there's a natural bias against. And there's an absolute and clear prediction. The prediction is that if there's a discriminated-against group, that if you measure subsequent performance, their subsequent performance will be stronger than that of the non-discriminated-against group. And that's a simple prediction of a theory of discrimination. And it's a testable prediction of a theory of discrimination, and it would be a revolution, and it would be an enormously powerful finding in this field, to demonstrate, and I suspect there are contexts in which that can be demonstrated, but there's a straightforward methodology, it seems to me, for testing exactly that idea. I'm going to run out of time. But, let me take-if people ask very short questions, I will give very short answers.
Q: What about the rest of the world. Are we keeping up?
LHS: Good question. Good question. I don't know much about it. My guess is that you'll find that in most of those places, the pressure to be high powered, to work eighty hours a week, is not the same as it is in the
Q: [unintelligible] because his book was referred to.
Q: I would like to make an on observation and then make a suggestion. The observation is that of the three. There is a contradiction in your three major observations that is the high-powered intensive need of scientific work-that's the first-and then the ability, and then the socialization, the social process. Would it be possible the first two result from the last one and that math ability could be a result of education, parenting, a lot of things. We only observe what happens, we don't know the reason for why there's a variance. I'll give you another thing, a suggestion. The suggestion is that one way to read your remarks is to say maybe those are not the things we can solve immediately. Especially as leaders of higher education because they are just so wide, so deep, and involves all aspects of society, institution, education, a lot of things, parenting, marriages are institutions, for example. We could have changed the institution of those things a lot of things we cannot change. Rather, it's not nature and nurture, it is really pre-college versus post-college. From your college point of view maybe those are things too late and too little you can do but a lot of things which are determined by sources outside the college you're in. Is that...
LHS: I think...
Q: That's a different read on your set of remarks.
LHS: I think your observation goes much more to my second point about the abilities and the variances than it does to the first point about what married woman....
LHS: Yeah, look anything could be social, ultimately in all of that. I think that if you look at the literature on behavioral genetics and you look at the impact, the changed view as to what difference parenting makes, the evidence is really quite striking and amazing. I mean, just read Judith Rich Harris's book. It is just very striking that people's-and her book is probably wrong and its probably more than she says it is, and I know there are thirteen critiques and you can argue about it and I am not certainly a leading expert on that-but there is a lot there. And I think what it surely establishes is that human intuition tends to substantially overestimate the role-just like teachers overestimate their impact on their students relative to fellow students on other students-I think we all have a tendency with our intuitions to do it. So, you may be right, but my guess is that there are some very deep forces here that are going to be with us for a long time.
Q: You know, in the spirit of speaking truth to power, I'm not an expert in this area but a lot of people in the room are, and they've written a lot of papers in here that address ....
LHS: I've read a lot of them.
Q: And, you know, a lot of us would disagree with your hypotheses and your premises...
LHS: Fair enough.
Q: So it's not so clear.
LHS: It's not clear at all. I think I said it wasn't clear. I was giving you my best guess but I hope we could argue on the basis of as much evidence as we can marshal.
Q: It's here.
LHS: No, no, no. Let me say. I have actually read that and I'm not saying there aren't rooms to debate this in, but if somebody, but with the greatest respect-I think there's an enormous amount one can learn from the papers in this conference and from those two books-but if somebody thinks that there is proof in these two books, that these phenomenon are caused by something else, I guess I would very respectfully have to disagree very very strongly with that. I don't presume to have proved any view that I expressed here, but if you think there is proof for an alternative theory, I'd want you to be hesitant about that.
Q: Just one quick question in terms of the data. We saw this morning lots of data showing the drop in white males entering science and engineering, and I'm having trouble squaring that with your model of who wants to work eighty hours a week. It's mostly people coming from other countries that have filled that gap in terms of men versus women.
LHS: I think there are two different things, frankly, actually, is my guess-I'm not an expert. Somebody reported to me that-someone who is knowledgeable-said that it is surprisingly hard to get Americans rather than immigrants or the children of immigrants to be cardiac surgeons. Cardiac surgeon is about prestigious, certain kind of prestige as you can be, fact is that people want control of their lifestyles, people want flexibility, they don't want to do it, and it's disproportionately immigrants that want to do some of the careers that are most demanding in terms of time and most interfering with your lifestyle. So I think that's exactly right and I think it's precisely the package of number of hours' work what it is, that's leading more Americans to choose to have careers of one kind or another in business that are less demanding of passionate thought all the time and that includes white males as well.
Q: That's my point, that social-psychological in nature [unintelligible].
LHS: I would actually much rather stay-yes, and then I'm on my way out.
Q: I have no idea how you would evaluate the productivity of the marginal hire if this person is coming into an environment where [unintelligible] is marginal and there's [unintelligible].
LHS: You're absolutely right. You're absolutely right. I used the term-I realized I had not spoken carefully-I used the term marginal in the economic sense to mean, only additional, to only mean...
LHS: No, to mean only the additional [unintelligible]. Yeah, obviously [unintelligible] going to identify X is the additional hire, is the marginal hire, the question you can ask is, you know, here is a time when, as a consequence of an effort, there was a very substantial increase in the number of people who were hired in a given group, what was the observed ex post quality? And what was the observed ex post performance? It's hard to believe that that's not a useful thing to try to know. It may well be that one will produce powerful evidence that the people are much better than the people who were there and that the institutions went up in quality and that made things much better. All I'm saying is one needs to ask the question. And as for the groping in the kitchen, and whatnot, look, it's absolutely important that in every university in America there be norms of civility and proper treatment of colleagues that be absolutely established and that that be true universally, and that's a hugely important part of this, and that's why at Harvard we're doing a whole set of things that are making junior faculty positions much more real faculty positions with real mentoring, real feedback, serious searches before the people are hired, and much greater prospects for tenure than there ever have been before because exactly that kind of collegiality is absolutely central to the academic enterprise.