Art Museum Network
"Leading where it Matters:
Measuring Success in the Art Museums of Minneapolis"April 1. 2008, 5:30-6:30 p.m.
Frederick R. Weisman Art MuseumMaxwell L. Anderson
The Melvin & Bren Simon Director and CEO
Indianapolis Museum of Art
_________________________________________________________________________________________________The boards of the three leading Minneapolis art museums are meeting jointly on April 1, 2008 to consider a key question facing non-profits in every community: how do we evaluate our performance? Max Anderson, director and CEO of the Indianapolis Museum of Art, and author of "Metrics of Success in Art Museums," commissioned by the Getty Leadership Institute, will address the group and facilitate a discussion about how best to craft performance metrics and consider ways of improving governance and management practices in light of these metrics.
_______________________________________________________________________________________________________________________________
The Outcomes of Measurement
The objective of measuring performance in an art museum is to have values and strategic goals drive governance and management priorities and practices. Business guru Jim Collins espouses the "Hedgehog Concept", whereby organizations flourish by looking for the intersection of "what you are deeply passionate about, what you can be best in the world at, and what drives your resource engine." (Good to Great and the Social Sectors, pp. 17-23.) 'Best in the world' may be a bar so high in the museum field that it is normally unattainable, but the apt premise behind Collins' formula is to look for passion and distinction in a field and see where it leads you.
Art museums occupy a privileged space in the non-profit sector: we attract contributed income from America's wealthiest donors. On the other hand, we cannot compete with social service and healthcare organizations in the urgency of our offer. So measuring our accomplishments is ever more essential as government retreats from maintaining safety nets from our nation's neediest, corporations reduce or restrict the scope of their charitable giving, and individuals with significant art collections build or consider building private museums.
We need to start by assessing our contributions from both ends of the spectrum: from a values and goals perspective, and from the granular measurement of productivity.
Much of what art museums do on a daily basis is to undertake tasks and obligations that are connected with appropriate professional conduct but are not necessarily values- or goals-driven. Each department or division of an art museum has a set of assignments specific to that business unit but which may not promote the overarching values and goals of the museum. Especially during times of economic turmoil, it is reasonable to review these ‘business-as-usual’ functions that may be substituting for real service to mission or for potentially transformative innovation. In so doing, you may arrive at the values and goals discussion from a less lofty, more practical point of departure.
An example might be the rental of your museum for external events. Few museums have undertaken the forensic accounting that reveals the true cost and benefit of external rentals. Once you factor in staff time, as well as complete energy and security costs, and wear and tear on your facilities, you may find that the earned income is not worth the true opportunity cost for certain kinds of event. And what is the potential cost of your service to mission by having activities which may be unrelated to or even at variance with your values?
Lower-level memberships are widely believed to be an important measure of a museum’s standing in a community. But in fact they are often a reflection of understandable bargain-hunting by those who would prefer not to purchase multiple full-price tickets throughout the year--and that's how we market memberships. More to the point, have you examined the true costs to your museum of a membership below $100? When factoring in staff expense, the opportunity cost of unsold tickets, and the true expense of membership events, promotional literature, mass mailings, and other activities, you may be losing money on this business unit without gaining audience or true loyalty.
Thus measuring the productivity of staff departments should not be done in a vacuum, but against the backdrop of other museum-wide concerns. If you orient business decisions towards the values and goals you have as a museum, you can gain a great deal by measuring performance against the furtherance of those goals. Having a significant membership base may be an important value to you--even if it is unlikely that lower-level memberships are important to your bottom line. Similarly, having external events that do not provide surpluses may still be defensible from a mission perspective. As long as you are aware of these facts, and don't govern or manage the museum with unwarranted fiscal optimism about them, there's no harm done. There is no shame in supporting your mission with activities that are not 'profitable.' In the end, your mission model is to put art into peoples lives, while your financial model is to break even--so even if a cost center in one area is considerable, it may be critical to your mission, and if it is paid for by another revenue stream, such as endowment or contributed income, then there is nothing to wring your hands about.
Attendance Hopes versus Reasonable Goals
As you become more comfortable with using metrics, it's important to insure that you measure what's important, not what's easy to measure. Let's deal up front with the mother of all metrics: museum attendance. It's the first statistic cited as evidence of a museum's stature. But pursuing attendance growth without a context makes no sense. Minneapolis is a large city, with 3,175,041 living in its metropolitan area, and 369,051 in the city proper, as estimated in 2006. The real question should be, not how do we grow attendance, but what would be the appropriate attendance for your specific institution relative to the size of your metro area or city, and where should you seek comparators?
To arrive at a reasonable target, it is instructive to measure your attendance and other performance metrics against institutions in other cities with which you would compare yourselves. Your closest comparators in pure size are Seattle-Tacoma-Bellevue, with 3.2 million, and San Diego County, with 2.9 million. They’re both affluent, educated, and aspirational. But they differ in their corporate culture, cultural heritage, politics, and demographics. So picking your comparators carefully will require more than a numbers game.
For the Walker, one instructive comparison might therefore be the San Diego Museum of Contemporary Art. Less demographically pertinent but still informative comparators would be SFMOMA, MCA Chicago, and the Whitney.
For the Weisman, the comparison might be the University of Washington's Henry Art Gallery. Less demographically pertinent would be the Michael C. Carlos Museum in Atlanta and the Nasher Museum of Art at Duke University.
And for the MIA, comparisons might include the Seattle Art Museum and the San Diego Museum of Art. Again, moving past comparable city sizes, you might choose to look at institutions with sizable budgets such as the Cleveland Museum of Art and the Detroit Institute of Arts.
But bear this in mind: according to the 2007 Statistical Survey of the Association of Art Museum Directors, your three museums are outperforming peers nationally, often by wide margins. This fact speaks of the enviable cultural vitality and visual literacy of the Twin Cities. So instead of taking the time to measure attendance, you have earned the right to look elsewhere in assessing performance.
Measuring the Museum’s Effectiveness
Attendance, membership levels, and external event income are symptoms of your effectiveness as an art museum, but do not reveal the extent to which you are serving goals and values. Simple-minded populist fare can drive the turnstile, but rarely enhances your reputation, scholarship, collections care, or educational impact. And big shows with big attendance make headlines but often lose money.
To measure your effectiveness as an art museum, the Board and CEO need to be in alignment in answering the question: effective at what? Becoming substitute for reduced arts education in the schools? A tourist destination? An economic driver? A crime-abating community center?
Earned income is often front and center in the deliberations of Boards of trustees. But across the Association of Art Museum Directors, the national average may startle you: less than 5% of the total revenue of AAMD museums comes from ticket sales (click here for supporting data).
Every stakeholder has a potentially different measurement of effectiveness. Earned income represents only a small fraction of an average museum's revenue. As a Board and a professional staff, you have the responsibility to choose from among a large supply of potential metrics to arrive at some that reflect the values and goals you espouse. Beginning with values and goals, rather than with financial metrics, will get your conversation at the Board table onto what matters most: how effective are you in fulfilling the potential of an art museum? If you focus too much on how much you resemble a for-profit attraction, you may end up drifting towards the behaviors typical of commercial destinations, and lose sight of your core educational mandate.
We'll take a few minutes together to test assumptions about what you would like your museum to be effective at doing. And for the sake of argument, let’s exclude the attendance, blockbusters, and number of memberships and events, since these get the lion’s share of media attention, but are extrinsic to the museum’s core functions, which include collecting, preserving, and interpreting works of art.
If you begin with the three top indicators of success for collecting, preserving, interpreting, and engaging, what might these be?
You might choose as a filter your museum’s impact on the field or impact on your community. Bear in mind that much of what you do cannot be easily measured, as is the case with the quality of the experience you offer. But in seeking to evaluate your museum, the outcome should be a way of assessing your effectiveness in promoting your mission, not simply the recitation of statistics that are more easily captured than others.
The Three Attributes of Success Metrics
In order to be worthy of adoption, metrics must have three attributes. They must be:
1. Directly connected with the core values and mission of the art museum;
2. Reliable indicators of long-term organizational and financial health; and
3. Easily verified and reported.Museum leaders must make a case for their institutions without turning first to the number of shows, visitors, events, and members—because these numbers may only seem to represent success. The job of museum administrators must be to diagnose the underlying health of their institutions and not simply recite statistics that may be ephemeral or unrelated to the overarching performance of an institution.
Defining Appropriate Metrics
The following aspects of a museum’s identity fit the three criteria for appropriate
metrics (mission-focused, long-term, and verifiable):1. Quality of Experience
2. Fulfillment of Educational Mandate
3. Institutional Reputation
4. Management Priorities and Achievements
5. Caliber and Diversity of Staff
6. Standards of Governance
7. Scope and Quality of Collection
8. Contributions to Scholarship
9. Contributions to Art Conservation
10. Quality of Exhibitions
11. Facilities' Contribution to Core Mission12: Ethics and Accountability
Arriving at Specific Metrics
You may choose to walk before running. When pursuing organizational change, it is often more effective to highlight a few key indicators that can spawn adoption by staff and lead naturally to their embrace of new working methods. The 12 categories above represent a holistic view of institutional performance. But you may find it preferable to review performance during a series of cycles, perhaps in alignment with a strategic plan in place or in formation. By choosing a handful of the 12 categories above, and focusing on improvement on these during a 3-year window, you may make more headway than by a scattershot approach to all of them.
Using Metrics in Management
Once you've agreed as Board and staff leadership on appropriate metrics, the next challenge is to make use of these in how decisions are made. At the Indianapolis Museum of Art, Board meetings and select staff meetings now begin with a brief review of our institutional Dashboard, which is a real-time, web-based, warts-and-all set of performance indicators that touch on various areas of institutional activity. It is built out of open source technology, and is easily replicable at other institutions. Have a look at it and feel free to comment on its usefulness; the link is here: IMA Dashboard.
Regardless of whether your metrics are real-time or open to all for perusal, they are only effective if staff embraces the responsibility of gathering, authoring, and updating the data, and if the Board concurs that these metrics line up with the museum they signed up to help lead.
Here again you may want to walk before you run. If you agree on even just 10 keys indicators of performance, you can begin to develop a culture of institutional evaluation that doesn't await the audit, the annual meeting, or the next financial reversal. Thoughtful year-round evaluation can help you weather turbulent times by keeping in mind not what's urgent, but what's important.
Using the relevant and agreed-upon metrics, you can schedule time at each meeting to consider how the outcomes of particular programs, events, exhibitions, publications, and other activities have improved your performance in particular areas. By keeping performance measures in sight as you plan, implement, and evaluate the museum's endeavors, you stand a better chance of matching institutional decision-making with values and goals.
Assignment for April Fool's Day
Please come equipped with three performance metrics that you would be interested in exploring with regard to your institution. We will make time to discuss several of these following a brief presentation about the general challenges of achieving a 'balanced scorecard' of evaluation. If you have questions or comments prior to or following the presentation, please feel free to email Max at: manderson@imamuseum.org.
_________________________________________________________________________________________________
Appendix I
36 Sample Metrics-Related Questions to Consider
Below you will find a sampling of three questions per category of metrics; these are simply presented to foster conversation at your board table or in your executive offices. Other metrics that assess performance in relation to values and goals may be more pertinent to your specific institution, the circumstances under which you are operating at present, or a planning process already underway that is emphasizing one aspect of institutional performance.
1. Quality of Experience
Market research should be the backbone of an effort to judge the quality of the experience offered by each of your museums. Collaboration with a business school or consulting firm could result in an intercept survey of visitors to each museum. Sample questions could result in data on the percentage of survey respondents who feel that a visit to the museum resulted in:
a) An intangible sense of elation—a feeling that a weight was lifted off their shoulders
b) A greater appreciation of specific works of art or a period or movement
c) Average length of time spent by visitors in front of ten significant works in the collection2. Fulfillment of Educational Mandate
a) Number of schoolchildren who visited the museum last year in organized tours
b) Number of audience-focused programs annually
c) Number of awards garnered by education initiatives
3. Institutional Reputation
a) Total number of volunteers working more than five hours a week
b) Total number of visitors from out-of-state tour groups
c) Level of buzz: Number of mentions of the museum on Google4. Management Priorities and Achievements
a) Percentage of budget directly dedicated to programs (labor and non-labor expenses)
b) Amount spent on curatorial travel paid out of non-exhibition budgets
c) Total dollar amount of unrestricted gifts directed to acquisitions or educational programs5. Caliber and Diversity of Staff
a) Number of full-time staff with Ph.D.s or M.A.s in art history
b) Number of qualified applicants for the most recent curatorial opening
c) Percentage of employees from minority groups managing two or more staff members6. Standards of Governance
a) Percentage of operating expenses paid from endowment proceeds
b) Number of museum-related conferences attended by trustees over the last 2 years
c) Number of trustees who have donated works of art to the museum in the last decade valued at $5,000 or more
7. Scope and Quality of Collection
a) Number of artworks in the museum's most significant collection (e.g. American paintings, European sculpture, European decorative arts, African art, etc.)
b) Number of accessioned works on display
c) Number of works requested for loan by other art museums last year
8. Contributions to Scholarship
a) Number of articles published by full-time museum staff in panel-edited scholarly journals
b) Number of collections catalogues in preparation and for how many years
c) Number of staff presentations at juried national meetings
9. Contributions to Art Conservation
a) Number of full-time conservators
b) Number of types of treatment conducted over the last year
c) Number of works treated in the last year
10. Quality of Exhibitions
a) Number of exhibitions with museum-published catalogues of 75 pages or longer staged over the last five years
b) Number of exhibitions presenting 30 or more objects of which more than 33% have not been shown together before
c) Percentage of total traveling presented that were organized by the museum11. Facilities' Contribution to Core Mission
a) Percentage of total building size dedicated to permanent collection galleries
b) Percentage of total building size dedicated to special exhibition galleriesc) Energy cost per square foot
12. Ethics and Accountability
a) Percentage of Board and staff leadership who sign an ethics policy
b) Number of policies with text reproduced in full on your museum website
c) Number of times your Compensation Committee meets annually
_________________________________________________________________________________________________
LINKS to publications by Max Anderson