How Refinery Helps With Sampling Complex Event Data

Sampling is the practice of extracting a subset of data from a dataset to make conclusions about that larger dataset. It’s far from a perfect solution, but when it’s implemented with Refinery, Honeycomb’s trace-aware sampling proxy, sampling can help you manage very high volumes of complex event data.

GOAT, an e-commerce platform focused on designer sneakers, apparel, and accessories, is a great example of a company that needs an effective sampling solution. In his talk at the 2021 hnycon, Kevan Carstensen, Backend Engineer at GOAT, explained that GOAT’s small backend team is tasked with managing customer-facing services with an extremely high volume of requests, as well as internal tools with a handful of stakeholders and much fewer requests.

Sampling with Refinery helps Kevan and his team manage this huge event volume, cut through the noise, and resolve issues quickly. But sampling isn’t necessarily right for everyone and implementing it correctly at your organization will be a unique process that depends on your infrastructure and team.

Implementing sampling with Refinery is a complex decision and a unique process, so let’s follow GOAT’s journey for implementing sampling with Refinery and pull out important lessons you can use to do the same.