COVES Frequently Asked Questions… Answered

We receive a lot of questions from curious museum professionals, organizations considering joining COVES, and professionals interested in improving their evaluation skills and knowledge. Regardless of where your curiosity comes from, we’d love to fuel it! Here are our answers to a number of Frequently Asked Questions of the COVES initiative.

What is the survey like?

Here is a version of the standardized survey that we use in the training process – it includes the questions and response options that all museums use, as well as a brief rationale for each. You’ll see that it’s not that revolutionary, but it ensures that museums are asking the exact same questions in the exact same way to enable comparison between sites.

This is what we call our “core” survey, because it is the base for all COVES institutional surveys. That said, each museum is able to tailor specific aspects of the survey, adding some questions that are of specific interest. Some Participating Sites ask about marketing, some ask about educational initiatives – the focus is up to you. Of course, we’ll help think through each question you want to add to ensure the data you gather meets your needs.

How many surveys do we need to collect? How do you decide on the sample size?

The sample size, the number of completed surveys each museum should try to gather each month, is based on a museum’s general visitor audience size. We look at your visitor patterns for the last two years, and we figure out (1) how many completed surveys you will seek to gather in a year, and (2) how many surveys you should try to collect each month. For example, if your museum sees about 250K visitors a year, you would look to collect about 800 surveys per year, and those 800 surveys would be distributed across the 12 months of the year in proportion your visitor patterns each month (as in, you’d collect more surveys in July than September).

When are we done collecting surveys? How many days a year do we need data collectors?

We hope NEVER! The “O” in COVES literally stands for “Ongoing,” so the data collection process occurs across the year.
Throughout the year, your data collectors will be handing out only one survey, but the information you collect through that survey will change over time as your museum offerings change. On average, museums spend 200 hours a year, collecting data. This time is best broken up into 2-4 hour segments.

How do we collect the data?

We will go through this in GREAT (too great? no, never) detail during the half-day training. The survey is completed on-site by visitors right before they leave your site. The data collector is stationed near the exit and requests visitors to complete the 7 minute survey on a tablet or computer. The data collector will either hand over the tablet to the agreeable visitor or make a note in a “Refusal Log” that is also loaded on the device.

What if we don’t have consistent, or any, wi-fi in our visitor spaces?

We know that not all museums have strong internet signals across their campus. We have two ways to work with that:

1. If you have a weak signal near your exit, we suggest having the data collector carry a Hot Spot with them during data collection. Paying a modest monthly rate is much more economical than boosting the signal by updating or installing a wireless signal!
2. We are working with Qualtrics, the survey platform and company we use for the COVES project, to set up the museum surveys in an offline fashion. We’ll keep you posted, but our hope is that you will soon be able to store your completed surveys on your tablet until you are back at your desk and able to connect it to the internet. Genius!

Where does the data go? Who does what for data cleaning, storage, and analysis?

All of the data is stored online, in a secure account within Qualtrics, which only the COVES research team has access to. Each month, Alex and Ryan download the completed surveys, clean the data and make sure everything is appropriately arranged for inclusion in the COVES dashboard, and upload each museum’s proprietary data to their dashboard. An aggregate set of data is also uploaded, so we can all compare our visitors’ responses with each other, but no one but the research team and the data recipient at each site gets to the the original data files.

What is the Online Dashboard?

One of the greatest aspects of the COVES project is the online dashboard! The dashboard is exactly what you think it will be – a series of charts and graphs that show the responses to your survey! Ok, it doesn’t look a TON like a car dashboard, but it is as quickly useful and informative as your speedometer and gas tank-meter!

The online dashboard is made available to museums once there are a sufficient number of completed surveys (usually 3 months), but you can check out this virtual walk-through or training to see what the dashboard looks like.

How will the data benefit us?

This is a GREAT question, and something we don’t have a specific answer for, as it truly does vary from museum to museum.

The most obvious answer is that you will have a much deeper understanding of who visits your museum rather than relying on assumptions or anecdotal data. Beyond that, you will have the ability to compare demographics to census data and experience ratings across a group of peer COVES institutions (including those of similar size, location, or content type).

The more complicated answer relies on why you would consider COVES a good fit in the first place – do you have specific visitor-facing questions already that these data would address? Or, do you need to first see your data and then ask questions of what you see to identify next steps for questions? A few broad examples of how others have used data include:

• Tailoring recommendations for visitors based on group composition
• Identifying audience segments
• Noting patterns that allow for targeted improvement (e.g., staff helpfulness, exhibit quality ratings, updating bathrooms or other building issues, etc.)

What does COVES cost for participating museums?

After a three-year grant from the Institute of Museum and Library Services, COVES has become a self-sustaining initiative, with over 25 museums paying for an annual membership to participate. The membership structure is based on institutional size (annual operating budget per IMLS standards), which are broken into the following categories:

• Very Small, $1,200 annually (annual budgets below $1 million)
• Small, $2,000 annually (annual budgets between $1 million and $3 million)
• Medium, $5,500 annually (annual budgets between $3 million and $10 million)
• Large, $9,000 annually (budgets of over $10 million)

The cost structure will evolve and change, and eventually drop for each member level, as more museums participate in COVES.

How is the Museum of Science involved? Does the museum make $ on COVES?

COVES is an initiative that is housed at the Museum of Science, Boston. Although the Museum oversees and manages the COVES initiative and is responsible for fiscal and project oversight, the Museum of Science, like all member museums, pays to participate. Annual fees from members cover COVES project costs only; any surplus revenue is reserved for use exclusively on COVES related expenses.

Is COVES right for my institution?

COVES is valuable for you institution if you want to…

1. Build internal capacity around understanding your visitors;
2. Use a set of common tools and measures developed by museum colleagues for the field; or
3. Benchmark or compare your museum to other institutions, regionally or by size.

There are many other reasons to consider using COVES as your ongoing visitor data gathering system, but these will be specific to you.

We sign up for COVES… then what happens?

You sign up, and then the sequence roughly goes like this:

1. We tailor elements of the COVES core survey to meet your institutional needs and then get everything set up on the back-end.
2. One of our team members comes to your institution to deliver a half-day training in which we cover as much as humanly possible (what the project is, why you’re starting to get involved, how to do data collection, and everything else)
3. And then, the data collection process begins!

We have created Evaluation Capacity Building materials (1-2 page documents and short videos) to assist with questions that may arise after the training, and we are always available through email and phone calls to discuss both data collection needs and data interpretation issues. Additionally, The COVES Community of Practice allows you to reach out to any other institution participating in COVES to discuss the questions that arise as you consider how to use your visitor data.