BulletinCover-NovDec2016
THE
BULLETIN
Volume 84 | Issue 6
November/December 2016

Assessing Usage Data for Better Decision Making

Emily Feuer and Jessica Krupski

Read the print version [pdf] of this article. 
 

With tuition and competition on the rise, students and families have increased pressure on colleges and universities to become more accountable. One of the ways student affairs professionals can quantify students’ experiences and efficiencies in the office is through assessment. Collecting quantitative data to be able to report on effectiveness has been around in the academic realm for decades (testing, accreditation, etc.), but is now becoming more and more relevant and important in the student affairs world as well. While the scary “A” word makes many professionals cringe, assessment can positively prove a program’s value, help with difficult decisions (especially as it pertains to budgeting), and help to prove a point with more than just words, but with numbers. When thinking about assessment, surveys are often the most obvious measurement tool. For student union professionals, usage data is critical not only for departmental assessments, but also for planning for the future. Such data may help justify for new construction and can be useful for developmental initiatives when speaking with donors.

What is usage? Many student unions consider usage as “how and when” the facility is utilized by students, staff, faculty, alumni, and community guests. This includes attendance, equipment, and room reservations. During the beginning stages of planning to gather, analyze, or report on usage data, it is important to begin at the end and ask, “What is the end goal?” Consider some basic questions to get started: What will make this assessment project successful? When will it need to be completed? Who is going to do it? Why is this important? When thinking about the ultimate goal remember to keep in mind departmental, divisional, and institutional missions and values to ensure that when it comes to reporting, the target audience is invested in the results.


Collecting Data

When thinking about which usage data to collect, the options are bountiful. Practitioners may want to consider looking at options in the categories of reservations, equipment usage, and attendance.

Reservations
For reservations, it may be helpful to analyze reservation numbers by space (in X space, there are X reservations per day), by day of the week (how active a space is on weekdays versus weekends), by time (are there more reservations before or after 5 p.m.), and by organization type (student organization versus university department). After gathering a baseline of all of the various usage indicators for reservations, it will be easier to identify trends, other areas that are stagnant, and which indicators are not useful and therefore do not need to be tracked going forward.

Furniture
To help better understand and predict wear and tear on tables, chairs, and other furniture, valuable usage data may include how many times these types of equipment are booked with reservations. Further, lounge furniture usage in the building may be estimated using traffic counts, room checks, or other markers. At SUNY–New Paltz, usage data has resulted in prioritization of furniture replacement and the repurposing of a room to increase usage.

Audio/Visual Equipment
Through various discussions with colleagues, anecdotally operations staff may realize that more and more reservations require complicated audio/visual needs such as livestreaming and video conferencing. Capturing data including how many reservations require audio/visual equipment and personnel through reservation software programs or simple tallies can help show this increase. This type of usage data can help tell the story of increased pressure on student unions, even if, perhaps reservation numbers as a whole have decreased. At the University at Albany, after tracking the number of times an outside sound company was contracted for the year (and how much money was spent), staff determined that it may be more effective and efficient to change a current (and active) job description to include more audio/visual responsibilities. This change would not have been made if not for the data on the frequency of use.

Attendance
Taking stock of how many people are in the building (at both events and daily/general use) can be a powerful tool. Beyond tracking attendance at events, it may be useful to document the number of people in certain areas at specific times, the number of people in certain lines, and/or possibly the number of people who come to general information desks for help. SUNY–New Paltz used visitor tracking data to develop student managers and other student staff schedules based on peak hour trends.

Other
There may be other types of equipment to account for or request. Does the union rent gaming equipment? What about movies or electronic equipment? At the University at Albany, the student union loans out phone chargers. With more than 850 rentals for the 2015–16 academic year, requesting funding for yearly replacements was an easy sell.


How to Collect and Organize Data

Now that assessment goals have determined what usage data to collect, consider how to actually collect it. What resources already exist? Are key cards used after hours? Do retail outlets track transaction counts? Are there already traffic or lighting sensors installed in key areas? Is ticketing used for any events in the building?

Most campuses have an electronic platform for reservations. By using query functions and/or creating reports, the number of reservations and the details of those reservations become available in aggregate form. When using these query functions, or when creating custom reports, make sure to be conscience about how data are categorized and communicate this categorization to others. For example, be clear with all staff on how “semester” reservations are defined. Does this include breaks within a semester? What about finals week? Consistency and common understanding are critical so that information can then be compared to prior years to gain a better understanding of how and why the building is being used. While the reservations system can track the reservations made, observation data (like rounds) will reveal how the reservations are being fulfilled during and after the actual event.


Figure 1: March Campus Center Room 375 Usage


At many student unions, student staff members conduct rounds throughout the building at designated times throughout the day. A student staff member will walk through the building taking count of the number of people in each space through simple paper counts, Excel sheets available on a tablet, or through online form platforms such as Collegiate Link or Google Sheets. Once that data is collected, then what? Before 2014 at the University at Albany, this data was stored in a binder to never be looked at again. Over the last two years, these counts have been compared with monthly and yearly reservation data collected through the Event Management Systems (EMS). The actual usage is contrasted with the requested/anticipated usage identified through the reservation. Figure 1 shows University at Albany attendance figures and reservation information in a simple chart comparing the current year to the past year. This information can then be used to understand building hours, staffing needs, as well as scheduling repairs and replacements. While rounds can be a powerful tool, it may be useful to ask building partners such as food vendors and offices if they have noticed any trends or specific usage of space in the building. For example, while a specific lounge may not be a reservable space, a food vendor next door to the lounge might notice that it is consistently being used as a book club meetup location.

When planning the collection of usage data, look at the academic calendar. Will students conduct rounds every hour, every other hour, or some other variation? Will this be the same for weekdays and weekends? Will the data be collected and analyzed every month or will some months be combined? Should August and September be combined since August is a short month according to the academic calendar? Be thoughtful in planning to facilitate the most accurate comparison from month to month and year to year.

Once the data have been collected, they will need to be organized. As a beginner, Excel can be an excellent tool. One worksheet with different tabs for different months, or possibly different spaces, may be a simple and effective approach. Using formulas to compile yearly data based on the monthly/space tabs can also be helpful. There are, of course, more advanced programs such as SPSS statistical analysis software and IFTT, which automates interactions between different applications using “recipes.” Within reservation programs like EMS, one powerful tool is the ability to have EMS “remember” a specific query, which makes it easier to compare data from one month or year to the next.


Interpreting Data

While gathering data is an important aspect of the assessment cycle, it does not end there. By starting with the end in mind, the data collected can help answer important questions about office initiatives or functions. Consider the original goal and how the results answer that initial question. Use these results to make meaningful decisions about space, equipment, and services in addition to planning for the future.

Data can be used to inform future decision making. For example, if certain areas are being used more frequently than others, maybe it is important to include more signage to those areas of the building. Or possibly, if certain areas are being used by specific groups over and over again, these groups should be part of the renovation/upgrade process for those areas, as was the case at the University at Albany. Usage data can also be helpful for furniture, fixture, and equipment (FF&E) planning. A piece of equipment that is used five times a week will most likely need to be replaced more frequently than a piece of equipment that is used once every other week. Also, in terms of capital planning, knowing which spaces are most frequently used can influence whether more of those spaces should be added to the building.

Further, consider what the data reveal that might require more investigation. Are certain organizations not using the building for certain reasons? Are groups renting certain kinds of equipment from other vendors that the union might be able to purchase and rent (generating revenue)? Is some service not being used because people don’t know it’s available? Usage data can help guide future marketing strategies for the department. Of course, usage data can also help determine new usage data to include in the next assessment cycle. The process just keeps on continuing!


Reporting Data

Audience
When thinking about how best to report data, it is helpful to first think about the audience. Is the data for coworkers or is it for the vice president? Is it for the general public? Depending on who will take action based on the data, it might be necessary to report in multiple ways. The data nerds may want to know all of the specifics of the data including methodology, response rate (as applicable), other less relevant findings, and trends between current data to past data. Others may only have the time to read a few quick highlights. It may be helpful to start with a full report of all of the data and then parse it into smaller infographics or mini-reports/charts based on the top three to five results or findings. When creating smaller reports, be sure to direct the audience to a contact for more information.

Sharing Data
After collecting and analyzing data and considering who it will serve, it’s time to start drafting the actual report and perhaps a suggested action. However, it will be necessary to build understanding and secure buy-in. While a natural first inclination may be to share the data upward to a departmental supervisor and above, also consider sharing the findings “downward.” Are there student employees who collected some of the data who might want to see how their hard work is being used? Did a specific population show interest in the process and by extension may want to see the findings? While it may be frightening to share “negative” data, doing so may inform an argument for increased resources or funding.

In addition, consider sharing data with other venues/campus partners that host events. This may inspire them to respond likewise as they conduct their own assessments. Being able to see the greater event trends on campus may help with future plans and explain micro-trends in the building. Perhaps, the union is seeing a decline in reservations for small meeting rooms, but a newer venue with enhanced meeting rooms is seeing a large increase. Thinking even larger, it may be beneficial to share annual data with ACUI. If many campuses started to do this, there would be a larger collection to use as a resource and trend tracker.

Limitations
A small disclaimer: When sharing data, make sure it is clear what the data indicates, but also what it doesn’t indicate. At the University at Albany, attendance data is collected by student staff members estimating the number of people in a room every hour. This data is a helpful estimate for attendance, but it is important to note the method could include human error or timing conflicts. Rounds may be completed at 6 p.m. for a space in which an event does not start until 6:30 p.m., causing the student-tracked number to be significantly less than the actual attendance. While this number is helpful when creating general and aggregated trend assumptions about the room, it may not be an accurate depiction for the event organizers. Keeping the data’s limitations in mind is an important part of being an ethical and responsible researcher, but these limitations should not be a barrier to reporting on or using the data. A simple disclosure such as “this data is aggregated from counts by student staff members, so please consider the possibility of human error when interpreting the results” on the report may suffice.

Key Performance Indicators
Many unions are already collecting usage data in one form or another although it might not be analyzed regularly or used for assessment purposes. Some institutions may track metrics related to their strategic plan. Often this is in the form of key performance indicators, of quantifiable outcomes related to broad priorities or goals. These may be a useful context in which to examine the union’s more specific data sets. For instance, in addition to the data types mentioned, the Maucker Union at the University of Northern Iowa has contact hours of students at social/recreational events as one of its key performance indicators for engagement goals. It calculates this number by multiplying the number of hours of each event by a tally of student attendees.

Additionally, assessment of usage data can be used as leverage or evidence to support a need. For example, based on safety data related to slip and fall incidents on its linoleum tiled ballroom, the Lowman Student Center at Sam Houston State University was able to make the case to install carpet squares in the space, which has had the added benefit of improved acoustics. Similarly, during the renovation and expansion of the union at the University at Albany, usage data have aided in understanding how relocating services has affected traffic patterns, reservations, and students’ use of the facility. For example, once the gaming lounge was closed, recreational equipment (billiards and table tennis tables) was moved to a new lounge. Data allowed for tracking, documentation, and reporting an unexpected spike in attendance in this space, which led to a better understanding of the value and need for recreation within the union. Without collecting data before and after the move, this information would not be available and a rather high increase would have been unknown to the larger campus community.

In these instances, and in potentially infinite others, collecting, interpreting, and reporting usage data using assessment methods offers the opportunity to make more informed and supported decisions. In addition, union professionals can better articulate the fulfillment of its mission and vision to students, colleagues, and the greater campus community.

 


 

Emily FeuerEmily Feuer, efeuer@albany.edu
Emily Feuer is the assistant director for reservations at the University at Albany Campus Center. She serves on the Division of Student Affairs Assessment Council and has worked on assessment projects for the division including welcome experience event outcomes, compiling data on select student populations, and original research on student retention. She also co-manages the assessment efforts for the Campus Center. In addition, Feuer is a doctoral student in the educational administration and policy studies department at the University at Albany. 

Jessica KrupskiJessica Krupski, jkrupski@albany.edu
Jessica Krupski started her career in higher education in 2011 and has held positions in admissions, student involvement, and leadership. She currently serves as the assistant director for evening events and operations for the University at Albany Campus Center. Through the Division of Student Affairs Assessment Council, Krupski co-chairs the student employment experience committee and co-manages assessment efforts for the Campus Center. Krupski is also a doctoral student in the educational administration and policy studies department at the University at Albany.