2011 Poets & Writers Magazine Ranking of MFA Programs: A Guide to the Methodology

Data Sources
For those program measures not subject to applicant polling, such as rankings and recitations of admissions, curricular, and funding data, only data publicly released by the programs—either to individual applicants, to groups of applicants, in a program's promotional literature, or via a program Web site—have been included in the rankings chart. All data were updated regularly to reflect programs' most recent public disclosures.

Many of the nation's full- and low-residency MFA programs decline to publicly release internal data. In 2007, between 40% and 60% of the nation's MFA programs declined to answer questions on an AWP questionnaire seeking admissions and funding data from member programs. Specifically, 47% of programs declined to reveal how many assistantships they offered annually to incoming students; 61% declined to reveal the stipend offered to teaching assistants; 56% declined to reveal whether they offered a full tuition waiver to teaching assistants; 49% declined to reveal how many scholarships were offered to incoming students; 55% declined to reveal their annual number of applicants; and 52% declined to reveal the size of their annual matriculating class. Compounding the incompleteness of the AWP survey was the fact that the Association did not distinguish between low-residency and full-residency programs. Given that low-residency programs do not offer teaching assistantships (as low-residency students are only on campus during brief residencies), this omission was a critical one. Likewise, because AWP surveys are only sent to AWP members, and AWP has previously indicated in public disclosures that 33% of U.S. creative writing programs are not AWP members, the 2007 survey's polling cohort (142 MFA programs) was missing as many as 71 potential respondents.

Programs unable or unwilling to release data regarding their funding and admissions processes are necessarily disadvantaged by a ranking system that promotes and rewards transparency. Yet no program that fails to release this data for applicants' consideration can avoid being judged, by applicants and other observers, through the lens of such nondisclosures. As research for these rankings is based entirely on publicly-available, publicly-verifiable data, (1) the accuracy of the data upon which the rankings are based can be readily confirmed by any party, and (2) programs can easily optimize their involvement in the rankings by ensuring their applicants have access to all of the data prospective students generally require in making application and matriculation decisions.

Programs were not contacted directly for these rankings for a variety of reasons: (1) As indicated above, past attempts by AWP, the national trade organization for creative writing programs, to secure even bare-majority participation by its member programs via a nationwide data-disclosure project were unsuccessful (and AWP member programs presumably owe more, not less, institutional fealty to AWP than to any independent nonprofit or freelance journalist); (2) the human resources required to track down internal admissions data for nearly two hundred MFA programs, many of which do not wish to release such data, would likely be prohibitive for any independent nonprofit organization or freelance investigative journalist; (3) to the extent the present rankings seek to actively promote program transparency, it would be counterintuitive for the rankings to reward programs willing to selectively leak data to members of the media through private channels, but not, via publicly-accessible channels, to the public-at-large; (4) unless 100% compliance with a nationwide data-disclosure project could be ensured, any attempt to reach programs individually—rather than place the responsibility for disclosure of admissions, curricular, and funding data on the programs themselves—will necessarily favor those programs researchers are able to successfully contact. This places the onus for proof of "equivalent due diligence" (as to each program) on researchers rather than where it belongs, on the programs themselves. The programs, not their assessors, are the "bearers of least burden" with respect to due diligence in the release of these data, as they only stand to benefit from increased transparency and are entirely in control of their internal data and program Web sites at all times.

Structure
Low-residency programs were measured in eight categories, six of which are rankings—four employing unscientific but probative polling of the sort described above, and two based upon publicly-available hard data. Low-residency programs have not been assessed with respect to their funding packages because these programs generally offer no or very little financial aid to incoming students. The reason for this is that low-residency programs presume their applicants will continue in their present employment during the course of their studies.

Cohort
Over the course of three successive application cycles, a total of 195 low-residency applicants were polled as to their program preferences, with these preferences exhibited in the form of application lists. The locus for this polling was the Poets & Writers online discussion board, The Speakeasy, widely considered the highest-trafficked low-residency community on the Internet. The relatively small cohort used for this polling accounts for the following: (1) The annual applicant pool for low-residency programs is approximately one-eighth the size of the full-residency applicant pool (see below); (2) low-residency applicants do not congregate online in the same way or in the same numbers that full-residency applicants do; and (3) low-residency programs are subject to a "bunching" phenomenon not evident with full-residency programs, with only eight programs nationally appearing on even 10% of poll respondents' application lists, and only three appearing on 20% or more. For this reason only the top ten low-residency programs have been included in the rankings (also available in the September/October 2010 print edition of Poets & Writers Magazine); below this level it is difficult to draw distinctions between programs, as none received a significant number of votes over the three years polling was conducted.

One explanation for the bunching phenomenon described above may be that low-residency programs are less susceptible to comparison than full-residency programs, as many of the major considerations for full-residency applicants, including location, funding, cohort quality, class size, duration, and cost of living, are not major considerations for low-residency applicants due to the structure and mission of low-residency programs. Generally speaking, low-residency programs are assessed on the basis of their faculty and pedagogy, neither of which are conducive to quantification and ranking. That three programs have such a clear advantage in the rankings on the other 43 operating in the United States, Canada, the United Kingdom, and China is a function of both the relatively recent development of the low-residency model (with older programs tending to be more highly regarded, though none dates to before 1976) and the consensus that appears to have existed for years that three programs in particular are strongest in terms of faculty, selectivity, and placement. It is worth noting, too, that a significant number of the world's 46 low-residency MFA programs were founded within the last eight to ten years; applicant familiarity with these programs may still be relatively low.

The three-year low-residency polling described above has been further broken down into year-by-year poll results. The cohort for the 2009–10 annual ranking was 88, for the 2008–09 ranking 55, and for the 2007–08 ranking 52. If and when individual account-users applied to programs in more than one admissions cycle, their application lists from each cycle were treated as separate slates of votes; repeat applicants accounted for less than 10% of the polling cohort, however. Full-residency applicants on The MFA Blog who applied to one or more low-residency programs as part of their overall slate of target programs (see "Structure" and "Cohort" under the header "Full-Residency Rankings," above) were also included in the low-residency voting; due to the exceedingly small number of such votes, these entries were manually compared both to one another and to existing low-residency application lists to ensure duplicate lists were avoided.

While polls with larger cohorts are, all other things being equal, more reliable than those with smaller ones, the fact that the annual applicant pool for low-residency programs is likely between 400 and 500 (see below) suggests that even the 2007–08 single-year low-residency rankings polled a substantial percentage of all applicants nationally during that application cycle. Moreover, as is the case with the full-residency rankings, cross-checking applicant vote totals across a period of three years reveals substantial consistency in the results and quickly unearths any significant anomalies or outliers. Of the ten low-residency programs listed in this year's print rankings, eight (80%) ranked in the top 10 in all three years of polling, while another was in the top 10 for two of the three application cycles studied. All of the programs in the top 10 achieved at least an Honorable Mention (a ranking between 11 and 15) for all three of the years in which low-residency applicants were polled.

An "N/A" notation signifies that a program has not released the requisite data. An asterisk indicates that the program is unranked in that category. Only five low-residency programs achieved a positive score in the national placement ranking, which considered placement data for full- and low-residency programs in a single assessment: Vermont College of Fine Arts in Montpelier (#17 nationally); Warren Wilson College in Swannanoa, North Carolina (#38); Bennington College in Vermont (#41); University of Alaska in Anchorage (#46); and Queens University of Charlotte, North Carolina (#53). In order to better acknowledge the achievement, in the placement category, of these five low-residency programs relative to their low-residency peers, and in recognition of the fact that low-residency graduates are substantially less likely to seek postgraduate fellowships (largely because they do not give up their present employment when they matriculate), the rankings above have been re-constituted as low-residency-only: Vermont College of Fine Arts, #1; Warren Wilson College, #2; Bennington College, #3; University of Alaska, Anchorage, #4; and Queens University of Charlotte, #5.

Due to the still relatively small number of low-residency programs in the United States and abroad, only programs receiving top 10 placement in any category of assessment have received a special notation in either the print or online editions of the rankings.

National Low-Residency Applicant Pool
A realistic estimate of the annual number of low-residency MFA applicants is 400. This estimate is based in part on the fact that the five most-applied-to low-residency programs receive an average of 144 total applications per year; in contrast, the five most-applied-to full-residency programs receive an average of 1,137 fiction and poetry only applications per year. If this comparison is any guide, approximately eight times as many individuals apply to full-residency programs as low-residency programs each year, suggesting a mean low-residency applicant pool, per year, of just over 400. This figure can then be cross-checked using the number of votes for Warren Wilson College in the present low-residency rankings (79), the total number of low-residency votes cast for the rankings (195), and Warren Wilson's publicly-released annual applicant pool size (200). Using these figures one would expect an annual national low-residency applicant pool of 494. The only other low-residency programs for which all these data are both available and may be considered reliable are Bennington College (whose data suggest an estimated 488 annual low-residency applicants) and Lesley College (598).

In view of the above, the three-year, 195-person sample used for this year's low-residency rankings likely represents between one-half and one-third of an annual applicant cohort for this type of residency program.

Added to the adjusted mean for annual poetry, fiction, and nonfiction applicants, the estimate for the annual number of low-residency applicants suggests a total annual applicant pool to creative writing MFA programs—across all genres and types of residency, and gauging discrete applicants only—of somewhere between 4,000 and 5,000.

Cohort
Between July 15, 2009, and April 15, 2010, 346 fiction applicants were polled for the fiction-genre rankings, 141 poetry applicants were polled for the poetry-genre rankings, and 101 nonfiction applicants were polled for the nonfiction-genre rankings. The reason for the disparity between the total number of fiction and poetry applicants in the genre-specific polls (487) and the total number of votes in the overall fiction and poetry poll (527) is that 40 applicants, or 7.6% of the cohort polled in fiction and poetry, did not specify their genre—though it was clear from their application lists that the genre in which they applied could not have been nonfiction (due to the fact that the majority of MFA programs do not offer nonfiction tracks, an applicant specifying that he or she has applied in only genre, but who lists certain programs on his or her application list, can be precluded from consideration as a nonfiction applicant). One consequence of this 7.6% nongenre-reporting population is that certain programs are tied in the overall rankings even though, by virtue of their rankings in the two major genres, this would seem to be a statistical impossibility.

The cohort sizes used in this polling are roughly consistent with the national distribution of MFA applicants by genre, as revealed by those few programs which both (1) accept applicants in all three genres, and (2) release their internal admissions data for all three genres. The national distribution of fiction, poetry, and nonfiction applicants is approximately 6 to 3 to 2, respectively.

Due to the still relatively small number of nonfiction programs in the United States and abroad, only programs receiving top 20 placement in the genre have received a special notation in either the print or online editions of the rankings. No Honorable Mentions have been awarded, for the following reasons: (1) the relatively small number of votes for programs ranked beyond twentieth in the genre, all of which appeared on fewer than 10% of nonfiction applicants' application lists; (2) a bunching phenomenon in the nonfiction rankings, such that any presumptive Honorable Mention section of the nonfiction rankings (programs ranked between 21 and 25) would include nine programs, making the Honorable Mention section nearly half the size of the rankings proper; and (3) there would be little statistical distinction, that is, two votes or less, between the nine presumptive Honorable Mention programs and the six programs ranked behind them—a smaller disparity, out of a cohort of 101, than the three-vote difference between the top 50 and Honorable Mention sections in the 527-cohort full-residency rankings.

Programs without a nonfiction program are designated, in the top 50 rankings, with an em-dash (—).