Fiona Ingleby recently submitted a manuscript to a PLoS journal and received one terrible and sexist review rejecting it (interestingly the manuscript investigated the progression of PhD students to post docs in biology and found evidence for gender bias in their data). She and her co-author appealed three weeks ago with no response from the journal at which point she posted excerpts from the review to twitter, unleashing a storm of outrage towards the reviewer and PLoS. I'll let everyone else comment on the problems of sexism that this raises (here, here, here, and here) and focus on the problems of the current peer-review system that this situation highlights.
Objectivity
One of the first problems was that the reviewer was not blind to the authors which allowed them to bring their sexist attitudes to the review. Even more inappropriately the reviewer took time to research the authors and made other comments about them including that they were too young and inexperienced. Introducing double-blind reviews may help prevent or slow down such easy efforts to undermine particular researchers. However, in practice, especially when publishing in specialized journals and when you are firmly planted into your research line, it is fairly easy to figure out who the senior author is on a manuscript, blinded or not. Generally, more than one person serves as a gatekeeper to publication with at least two, sometimes three and other times even more. By having more than one reviewer you can get different perspectives and improve the reach and impact of the paper. It can also be good for editors where consensus among reviewers indicates the potential high-impact quality of publications. But by having more than one reviewer, a pragmatic issue may be to help the editor screen out any bad reviews. Two reviewers with completely opposite opinions may indicate that the paper needs clarification or that one of the reviewers is trying to tank a paper (oh no scientists are people with emotions and egos too!) or is not qualified to review the paper (oh no sometimes reviews are passed off to under qualified trainees). This issue of objectivity and double-blind review leads into the next point of pre-registration.
Pre-registration
The review had no critiques that could be used to improve the manuscript. The closest that the review came to actually commenting on the science was that the manuscript was “methodologically weak” and “has fundamental flaws and weaknesses that cannot be adequately addressed by mere revision of the manuscript, however extensive." Anytime that a review attempts to go after the methodology, it irks me a little. If we think back to the origin of any particular study, it has to go through at least one if not more reviews before you can even carry out the research. Every study gets a once over from IRB and other institutions to make sure that the study crosses some bar of ethics and usually includes that it is being carried out with good science. Next, many, if not most studies are carried out with funding. Funding success at federal levels is hovering around 10% and only slightly better in other realms meaning that any research that has funding has potential implications and is being carried out in a way that may achieve those implications that it beats 9 other solid research proposals. So it always seems odd that its not until after the study is complete that apparent methodological flaws make a study unworthy of publishing as if every published paper is perfect in its formulation and the way that it was carried out. Pre-registration, in which a study's methodology is submitted and if it is accepted is published regardless of the results, allows us to avoid these issues and provides early feedback in the publishing process.
Slow
Three weeks! Three weeks of sitting on an appeal that justifiably points out that the review did not actually comment on the science of the article and was sexist seems like a poor move for PLoS. Now the outrage from the science community is reiterating that sentiment. The circumstances of this review raises so many questions, what was the editor thinking sending that review out, why was there only one reviewer, why would you appeal after a terrible review process and not go somewhere else? Post-hoc review is slow with quick decisions on the order of a month and slow decisions that more than a year (not including shopping to multiple journals or going down the prestige ladder). Before even submitting a manuscript for review, researchers are faced with questions that often pit time to publication against ease of publication. Many researchers always go down the prestige ladder just because the turn around (i.e. rejection) can be fast and if something hits, its worth the time. Other researchers would rather publish fast and will choose specialized journals with reputations for turning around manuscripts quickly.
Could we improve the quality and speed of peer-review by giving more credit or reward (some people think this option removes objectivity) for peer-review? Perhaps peer-review could improve if we shift how we approach it. Instead of thinking of ourselves as the gate-keepers of the sanctity of science we could see ourselves as collaborators in trying to improve science and disseminate science, messy though it may be.
Thursday, April 30, 2015
Sunday, April 26, 2015
Consortium for Faculty Diversity
I was recently hired as a Consortium for Faculty Diversity Fellow at Haverford College in Philadelphia, PA in the Psychology Department. The Consortium is an association of liberal arts colleges committed to enhancing diversity among members institutions. Last year when I was searching for tenure track positions, teaching positions and post-docs at liberal arts institutions, I found the CFD application with opportunities to work as a post-doctoral fellow at a liberal arts institution. When I saw this application, I knew that my experiences in community engagement may qualify me. I was so excited for the opportunity to apply for this program because my goal for the past 10 years has been to be a scholar and teacher at a liberal arts institution.
The CFD Fellowship program offers the liberal arts faculty experience described as follows:
At Haverford I'll be teaching three courses including a Seminar in Embodied Cognition, Introduction to Psychology, and Cognitive Neuroscience. Also, I'll be conducting research with undergraduate students and looking for engagement opportunities in the Philadelphia area. I'll have more updates about the experience in the future.
- The residential liberal arts colleges of the Consortium are committed to promoting excellent teaching for undergraduate learning.
- As part of their intellectual vitality, interactive teaching and active learning environments, Consortium member schools seek to build rich intellectual communities of students and faculty members.
- At our schools, a passion for teaching is a "must," as is significant effectiveness in a variety of pedagogical settings.
- Faculty members at Consortium member schools can expect to devote a substantial amount of time to teaching and advising undergraduate students.
- Because our campuses and departments are small, faculty members are encouraged to undertake inter-departmental and inter-disciplinary curricular projects, including the design and teaching of interdisciplinary courses, first-year seminars or courses for non-majors.
- Use of innovative pedagogy that supports student learning, such as collaborative group work or inquiry learning, may also be expected.
Continued Misunderstanding of the Function of Higher Education
In my free time my favorite thing to do run. I think I was drawn to research for many of the same reasons that I love running. You need to be somewhat crazy, you have to put in long hours of work for only small personal victories, only people who do the same thing as you, in order to excel you have to good at a number of seeming unrelated activities, and finally at the professional ranks you have to answer to people who have little understanding of the inner workings. Earlier this week I went to the Boston Marathon to watch my wife run (a PR in 3:17:23). As a spectator of the marathon, even though I was only watching people run 26.2 miles in 2 to 5 hours, I was actually watching the culmination of hours, months and even years of hard work and preparation. At the professional level, the men and women race winners were each awarded $100,000 for 2 hours of effort and similar to other professional sports, many mistakenly believe that they are only compensated for their time in the limelight. In run training, you have to combine hours of volume training, hours of race pace training, hours of speed work, hours of strength and form training as well as cross training, active recovery, proper diet and sleep. So instead of $50,000 an hour, if we only count the days elapsed so far this year, the athletes likely only made $76 an hour.
In my other field, in the past week, two examples of people with little understanding of higher education made a similar mistake in only paying attention to the limelight and forgetting everything else behind the scenes. In Iowa, a legislator introduced a bill that would have the professor with the lowest ratings in teaching fired while in North Carolina an education bill would mandate 4-4 teaching for all faculty. While the bill in Iowa died in committee the idea behind it shows almost no understanding of the role of faculty. Just as the two hours on the marathon course are just a sliver of what makes a marathoner, the three hours per week in one particular course are just a sliver of what makes a professor a professor. I value the importance teaching and have worked hard to gather feedback from my students throughout my career teaching but I am sympathetic to professors who struggle in the classroom. However while student evaluations are one of the only ways that teaching ability is assessed, this article discusses some issues with student evaluations and suggests that they may not be the best way (check out the comments as well).
In North Carolina, Senate Bill 593—“Improve Professor Quality/UNC System” would “ensure that students attending UNC system schools actually have professors, rather than student assistants, teaching their classes.” The bill asks professors at the research institutions of North Carolina to teach 4 courses a semester and ignores the fact that if tenured faculty are not teaching courses, its likely adjunct professors. Again while 12 hours in the classroom per week may not seem like much, its just the tip of the iceberg with lecture preparation, grading, and meeting with students. But this focus on teaching at Research I institutions ignores the fact that professors at these institutions are supposed to be creating knowledge in order to pass on that knowledge in the classroom. In my six years at research focused universities (Iowa, York and Toronto) I have met very few professors that were openly hostile towards teaching with most enjoying teaching (in the broader sense, including time outside the classroom). However, almost all of the professors that I know teaching 1-2 courses a semester have almost no time as it is. Few take time off, few sleep regular hours and few have hobbies outside teaching. Asking them to teach 4 courses but place the majority of their evaluation on their research will lead to research professors in North Carolina to leave in droves with no one looking to take their place (despite the huge oversupply of candidates).
If anything I think these two bills suggest that higher education needs to do a better job of communicating its role in society. College is not just a place for job training, so in order to avoid further commodification of higher education we need to not only communicate the importance of our research, but our teaching and engagement.
Edit: 4/27/15 One day after I write this, this article is published on the future of the American Research University. This passage echo's my sentiments above:
In my other field, in the past week, two examples of people with little understanding of higher education made a similar mistake in only paying attention to the limelight and forgetting everything else behind the scenes. In Iowa, a legislator introduced a bill that would have the professor with the lowest ratings in teaching fired while in North Carolina an education bill would mandate 4-4 teaching for all faculty. While the bill in Iowa died in committee the idea behind it shows almost no understanding of the role of faculty. Just as the two hours on the marathon course are just a sliver of what makes a marathoner, the three hours per week in one particular course are just a sliver of what makes a professor a professor. I value the importance teaching and have worked hard to gather feedback from my students throughout my career teaching but I am sympathetic to professors who struggle in the classroom. However while student evaluations are one of the only ways that teaching ability is assessed, this article discusses some issues with student evaluations and suggests that they may not be the best way (check out the comments as well).
In North Carolina, Senate Bill 593—“Improve Professor Quality/UNC System” would “ensure that students attending UNC system schools actually have professors, rather than student assistants, teaching their classes.” The bill asks professors at the research institutions of North Carolina to teach 4 courses a semester and ignores the fact that if tenured faculty are not teaching courses, its likely adjunct professors. Again while 12 hours in the classroom per week may not seem like much, its just the tip of the iceberg with lecture preparation, grading, and meeting with students. But this focus on teaching at Research I institutions ignores the fact that professors at these institutions are supposed to be creating knowledge in order to pass on that knowledge in the classroom. In my six years at research focused universities (Iowa, York and Toronto) I have met very few professors that were openly hostile towards teaching with most enjoying teaching (in the broader sense, including time outside the classroom). However, almost all of the professors that I know teaching 1-2 courses a semester have almost no time as it is. Few take time off, few sleep regular hours and few have hobbies outside teaching. Asking them to teach 4 courses but place the majority of their evaluation on their research will lead to research professors in North Carolina to leave in droves with no one looking to take their place (despite the huge oversupply of candidates).
If anything I think these two bills suggest that higher education needs to do a better job of communicating its role in society. College is not just a place for job training, so in order to avoid further commodification of higher education we need to not only communicate the importance of our research, but our teaching and engagement.
Edit: 4/27/15 One day after I write this, this article is published on the future of the American Research University. This passage echo's my sentiments above:
"Those of us leading or working in research universities, especially public ones, face the urgent imperative to articulate and give full-throated explanations of the extent to which university research not only brings economic and social betterment (through new medicines, policies, products, jobs, etc.) but also is crucial to the educational mission. It drives discoveries that can be commercialized to enrich innovators and their backers, and it ensures that those innovations will be deployed to sustain the vitality of our economy, our society, and our human values. Research is also a good in itself across the full set of disciplines and fields that constitute university life; it is an aptitude and skill that students, both undergraduate and graduate, learn in college that can be of lifelong value; and it is a force that generates new knowledge — and new modes of teaching and learning."
Monday, April 13, 2015
The "real reason" college tuition costs so much
In higher education, there many be no other refrain spoken more often than the costs of tuition raising at a pace that far exceeds inflation (or any other good and service). Two articles in the New York Times in the past few months have attempted to explain why tuition is so high and has increased so much. The earlier published article appears to be a couple page ad for a forthcoming book about the future of education and the author's assumption that it will be revolutionized by on-line education. Of the many problems in higher education he identifies, the one discussed at present is the profile and rankings arms race that he sees national and regional research institutions chasing. The article can be summarized by a quote from Mr. Trachtenberg, the former president of George Washington University, the case study of little brother universities trying to punch above their weight by raising their costs so they can spend that money on the "wrong" things:
In the second article, the real reason why tuition costs so much, the author names two different reasons for the increasing cost of tuition, the increasing number of students attending universities (requiring more services, facilities, etc.) and the increasing number of university administrators. Surprisingly, the author vehemently denies the most commonly cited cause of raising tuition costs, decreases in state and federal funding. He states that higher education's federal budget is 10 times higher than the 1960s and state appropriates are 4 times higher than the 1960s. Almost everywhere else I looked I couldn't find data supporting the idea that funding hasn't decreased (this was a very difficult search to do given the motivations on both sides of the argument). In 1960, 2.9 million students were enrolled in colleges and universities while in 2014, 7.24 times that many were enrolled. This piece from the Chronicle does a great job summarizing state funding trends with their findings pointing to decreases in funds.
Combining the suggestions from the two articles and reanalyzing one of their points I think we find the reasons why tuition is increasing. There has been an explosion in the number of students attending college, paired with increased government oversight and an increase in the number of services that colleges provide causing an increase in the number of staff to oversee/audit those new services all while the schools receive less funding from the public (federal and state).
One thing that seems to be rarely discussed when pointing out the college tuition is skyrocketing is that college today is much different from the 1960s or some other time period that is pointed to for lower tuition costs. Technological advances has created the need for revolutionized infrastructure in order to support computer and internet abilities (that also comes along with specialized staff to make sure those services are provided safely and timely). Other support services have been created and expanded including alumni/career services and student support services that help students find employment, health services or extracurricular activities. Again, to beat an old drum, college is more than sitting in a lecture and taking a multiple choice exam (or watching a video on-line, writing in a forum and taking tests). If we envision college this way, then it also makes sense to extend it to all aspects of the college experience. Dorm rooms should be 9X7 concrete blocks, the cafeteria should serve gruel and there should be no extracurricular opportunities because all of the students' time outside of the classroom should be spent working and preparing for class.
Colleges are meant to provide for the public good and includes more than just the time students spend in the classroom. In short, the public good provided by colleges/universities include: collaborative relationships with the community, service to the community, supporting diversity, improving quality of life, providing economic benefits, training a workforce, regulating social change, transmitting culture, creating knowledge, disseminating knowledge, providing critical reflection, promoting diversity of thought, increasing problem solving abilities, promoting democratic citizenship, civic participation, and social responsibility. Practically these public goods are reflected in increased engagement by college graduates including more volunteer work and participation in elections. College graduates are also more likely to make more money thus increasing their contribution to tax revenues while also using less social services. So while we watch public funds drain away from colleges and universities, at what point does the public good become a private good?
"College is like vodka, he liked to explain. Vodka is by definition a flavorless beverage. It all tastes the same. But people will spend $30 for a bottle of Absolut because of the brand. A Timex watch costs $20, a Rolex $10,000. They both tell the same time."With this metaphor and case study in George Washington University, the author describes the amenities arms race taking place at universities across the US. Under the assumption that more (or higher priced) means better, the author suggests that the price of tuition is increasing because it can. Without the higher price, universities lose out on the lazy rivers and luxurious apartment-style dorms that all of the other universities are investing in. So in this author's view, the high and increasing price of tuition is due to the misguided, endless search and battle for supremacy in research prestige, branding and self-aggrandizement.
In the second article, the real reason why tuition costs so much, the author names two different reasons for the increasing cost of tuition, the increasing number of students attending universities (requiring more services, facilities, etc.) and the increasing number of university administrators. Surprisingly, the author vehemently denies the most commonly cited cause of raising tuition costs, decreases in state and federal funding. He states that higher education's federal budget is 10 times higher than the 1960s and state appropriates are 4 times higher than the 1960s. Almost everywhere else I looked I couldn't find data supporting the idea that funding hasn't decreased (this was a very difficult search to do given the motivations on both sides of the argument). In 1960, 2.9 million students were enrolled in colleges and universities while in 2014, 7.24 times that many were enrolled. This piece from the Chronicle does a great job summarizing state funding trends with their findings pointing to decreases in funds.
Combining the suggestions from the two articles and reanalyzing one of their points I think we find the reasons why tuition is increasing. There has been an explosion in the number of students attending college, paired with increased government oversight and an increase in the number of services that colleges provide causing an increase in the number of staff to oversee/audit those new services all while the schools receive less funding from the public (federal and state).
One thing that seems to be rarely discussed when pointing out the college tuition is skyrocketing is that college today is much different from the 1960s or some other time period that is pointed to for lower tuition costs. Technological advances has created the need for revolutionized infrastructure in order to support computer and internet abilities (that also comes along with specialized staff to make sure those services are provided safely and timely). Other support services have been created and expanded including alumni/career services and student support services that help students find employment, health services or extracurricular activities. Again, to beat an old drum, college is more than sitting in a lecture and taking a multiple choice exam (or watching a video on-line, writing in a forum and taking tests). If we envision college this way, then it also makes sense to extend it to all aspects of the college experience. Dorm rooms should be 9X7 concrete blocks, the cafeteria should serve gruel and there should be no extracurricular opportunities because all of the students' time outside of the classroom should be spent working and preparing for class.
Colleges are meant to provide for the public good and includes more than just the time students spend in the classroom. In short, the public good provided by colleges/universities include: collaborative relationships with the community, service to the community, supporting diversity, improving quality of life, providing economic benefits, training a workforce, regulating social change, transmitting culture, creating knowledge, disseminating knowledge, providing critical reflection, promoting diversity of thought, increasing problem solving abilities, promoting democratic citizenship, civic participation, and social responsibility. Practically these public goods are reflected in increased engagement by college graduates including more volunteer work and participation in elections. College graduates are also more likely to make more money thus increasing their contribution to tax revenues while also using less social services. So while we watch public funds drain away from colleges and universities, at what point does the public good become a private good?
Subscribe to:
Posts (Atom)