Forgetting, To Move Forward
The Internet never forgets but perhaps it would be best if we occasionally exercised our selective memory. When reviewing services in particular it would do us well to refresh our assessments. While discounting the past may lend recent impressions an overvalued status, there is an intuitive appeal in listening more attentively to what people who have just had an experience have to say about it. Times, places and tastes change. We need not keep in stone the ratings of several years ago when we have more recent inputs to consider. Studio 54 of recent times is not the Studio 54 of yore; it would be absurd to expect the orgies of yesteryear when going today. On a less dramatic note we can pursue this hunch when it comes to beer bars.
Over the past decade-plus craft beer has stepped out of a niche interest into a conversation topic of general interest, even among those who cannot distinguish a Pilsner from a pale ale. In that time bars, breweries, styles, and knowledgeable consumers have proliferated. What was once considered a hop bomb now appears quaint; that 30-tap bar selection, rotated seasonally, was a unicorn a couple of decades ago; and the light lager swigger has learned about a variety of beers. All of which is to say, perhaps it is time to consider discounting historical reviews so as to get closer to the pulse of the zeitgeist. A gradual removal of past reviews accomplishes two things, it captures a rating more in line with contemporary appraisals (quality) and it identifies the number of reviews within a selected time window (quantity).
Time is like a river
Once again we can leverage a toy example to investigate alternative angles on viewing things and remind ourselves that the answers we get often depend on the questions we ask. Those questions in turn are inspired by our interests. This is not a case for relativism but a re-emphasis on the bootstrapping nature of many metrics and views we take for granted, grounded in the beings we are.
Three key features to track: review counts, review scores, and time. The final element introduces flow, movement. Numbers carry about them a certain finality and certitude that does not always warrant the authority we bestow upon them. Be skeptical of precise numbers. Look for ranges, confidence intervals and other bits of wiggle that help provide a more complete picture of the situation. Speaking of pictures, charts can be of great help when there is not much to differentiate among summary statistics (e.g. Anscombe’s quartet).
Data Source: TripAdvisor (Accessed: 26 Jun 2019)
Caught Between Bar Stools
Selecting the top 30 beer bars in NYC as per TripAdvisor we can review past ratings for each. Grouping ratings by the months they were submitted in easily lends them to weighting by month, with decreasing significance to those occurring earlier on, eventually dropping altogether from consideration. To get started we set the window of interest at 24 months. This was a judgment call that seemed about right for allowing past input without overvaluing either what came before or more recent impressions. Setting aside specific numbers, we get what we expect globally: squiggly lines indicating patron preferences over time, general trends, and rates of change. There are also gaps in the ratings, sometimes of multiple months, resulting in rating lines breaking off on the chart.
Data Source: TripAdvisor (Accessed: 26 Jun 2019)
This breaking off of reviews raised a subsequent thought regarding how long to retain ratings in the absence of online customer engagement. Here too we resorted to personal discretion and set a cutoff of six months. Bars that had gone six or more months without a rating had their scores shelved. These visual gaps help to send a message to potential patrons about traffic, interest, and/or whether the platform is adequate for the review of beer bars generally and the impacted bar, specifically.
Data Source: TripAdvisor (Accessed: 26 Jun 2019)
Thus at minimum we are left with three ways of viewing bar appraisals: a simple rating average; a time-discounted rating, with accompanying time series to visualize changes; and the latter with the added wrinkle of suspending ratings in the absence of engagement. The question of which of these is the most informative relies on what one is looking for.1 Keeping in mind that all models are wrong but some are useful.
Data Source: TripAdvisor (Accessed: 26 Jun 2019)
Notes
Opening Photo by Pixabay on Pexels
1 Process and becoming; always moving, always changing, ever in flux; snapshots are informative as well as misinformative, dangerous to chase an image – a frozen representation of desire, which itself constantly changes; chasing shadows, of course never fulfilled when grasped, grasping at air; life is not fixicity and stasis can only mean death; even prominent goods are not guaranteed to meet expectations, as both production methods and tastes change. (Excuse me, too much coffee).
Below are graphical representations of the coefficient of variation for the opposing ways of measuring ratings touched on here. We can see that in the latter, 24-month weighted moving average, incoming reviews predictably continue to make an impression, while the cumulative average develops a lock-in situation:↩
Data Source: TripAdvisor (Accessed: 26 Jun 2019)
Data Source: TripAdvisor (Accessed: 26 Jun 2019)
Code @nbviewer