- Podcast
The Hopara Dataverse: The Vision That Will Redefine Data with Dr. Michael Stonebreaker
Imagine a world drowning—not in water, but in data. A relentless flood of information, pouring in from IoT devices, enterprise systems, sprawling data lakes. And in this world, the tools we use to navigate—dashboards, tables, static reports—are brittle rafts, barely keeping us afloat.
Now, imagine something different. A system not of numbers and charts, but of landscapes. A way of seeing, navigating, zooming—not unlike the way we explore the world itself.
Dr. Michael Stonebraker doesn’t just imagine this world. He’s building it.
Michael has spent a lifetime shaping how we think about data. A Turing Award-winning computer scientist, he helped define the very foundations of modern databases. But today, he’s here to talk about something else: a fundamental shift in how we access and visualize information.
In this episode, Pete Wright sits down with Mike to explore Hopara—the tool that might just change how we see data forever. They talk about factories in Brazil, the hidden language of vibration sensors, and why Google Maps might hold the secret to understanding a chaotic world. They discuss the peculiar habits of EKG machines, the logistical ballet of John Deere combines, and why hospitals can be some of the slowest adopters of life-saving technology. And they confront the elephant in the server room: the real reason data remains such an untamed beast in the enterprise.
If you think data visualization is about prettier charts, think again. This is about something much bigger.
Links & Notes
Transcript
Pete Wright:
Imagine a world drowning, not in water, but in data, a relentless flood of information pouring in from IOT devices, enterprise systems, sprawling data lakes. In this world, the tools we use to navigate, dashboards, tables, static reports, they’re brittle rafts barely keeping us afloat.
Now imagine something different, a system, not of numbers and charts, but of landscapes, a way of seeing, navigating, zooming, not unlike the way we explore the world itself. Dr. Michael Stonebreaker doesn’t just imagine this world, he’s helping to build it.
Michael has spent a lifetime shaping how we think about data. A Turing award-winning computer scientist, he helped define the very foundations of modern databases. But today, he’s here to talk about something else, a fundamental shift in how we access and visualize information, with Hopara. I’m Pete Wright. Welcome to Seeing Beyond the Dashboard.
Dr. Michael Stonebreaker. Mike, I’m so glad that you are here. I’m honored to talk to you about some of the things that are on the docket for today. But you’ve had a remarkable career shaping the way we think about information architecture. Before we get into the meat of our discussion today, give me a quick snapshot of your journey. What first drew you into this field? And how has the field evolved around you in ways that you did or maybe didn’t anticipate?
Dr. Michael Stonebreaker:
When I graduated from college, it was 1965 and the Vietnam War was going full blast. So I had the choice of going to Canada, going to jail, going to Vietnam, or going to graduate school. So I chose graduate school, and that’s the only reason I have a PhD.
When the Army didn’t want me anymore and then I turned 26, I could graduate, and I took a job at Berkeley. And I was taken under the wing of a senior faculty member. And at that point, Ted Codd had just published his pioneering paper on relational databases that came out in 1970. My mentor, Gene Wong, and I said, “Well, it’s an obvious to see if we can build one of these puppies.”
So I became a database person and I’ve been doing that ever since. I’ve been extremely fortunate to have had a bunch of commercial experience and gotten to talk to a lot of people who tell me why existing solutions don’t work, and I can then attempt to do better. So that’s been kind of a template for my career.
Pete Wright:
It’s a fascinating journey and that leads us to our conversation today. We’re talking about Hopara and the challenge of making sense of real-time data at scale. So we live in a world of organizations drowning in data, whether from IoT devices or sprawling data lakes or enterprise systems. What is the fundamental shift you believe needs to happen in how we access and visualize this information?
Dr. Michael Stonebreaker:
Well, as you say, we’re drowning in data. And the standard techniques are to write what are called dashboards, which provide you summary data about this sea of information. And the trouble with summary data is that it’s sort of down in the weeds and I completely lose the big picture. What I really want is a system that will give me the big picture and then allow me to drill down into the details.
And Ben Shneiderman, who’s a well-known vis guy from the University of Maryland, called this overview first, then detail on demand. That paradigm is, I think, extremely powerful, and I think the best example of a system that does that is Google Maps. In 21 clicks, you can go from a picture of the Earth to a plot map on your home street. And you simply move and zoom, that’s the fundamental paradigm in Google Maps, it’s fundamental paradigm in Hopara. The problem with Google Maps is that it’s hardwired for geographic data. So if you have floor plans or you want to draw pictures of statistics, Google Maps doesn’t do that.
So my favorite example of a Hopara application is a real-time world. It turns out to be a customer of a Brazilian company called IBBX. And IBBX makes vibration sensors and a bunch of other kinds of sensors also. But the nice thing about vibration sensors is that machines on the factory floor tend to vibrate right before they die. So this is an early warning system of problems. The same is true, by the way, of disk drives. They tend to vibrate-
Pete Wright:
Yeah, they just start to shake right before they fail.
Dr. Michael Stonebreaker:
Right before they fail. So IBBX sells vibration sensors for the factory floor. And one of their customers has 58 factories in Brazil with about 20,000 sensors spread across these 58 factories. And IBXX, before Hopara came along, could give you details. They could tell you how many machines were vibrating, but they couldn’t give you the big picture. And what their customer really wanted was to see a map of Brazil with 58 dots on it indicating the location of all of their factories color-coded red, green, or orange with sort of whether the vibration problems were extreme, marginal, or there weren’t any.
And then you could take a factory and zoom into that factory and see a floor plan of the factory with all the machines on the factory floor individually color-coded red, green, or yellow. And you could then pick a machine and hit the zoom into that machine and see the location on the machine of any sensors if that was of interest to you. Or you could zoom further and see a time series of vibration data of that individual sensor.
Pete Wright:
I just want you to describe what you’re seeing for those who might be struggling to visualize it.
Dr. Michael Stonebreaker:
So you start with a map of Brazil, which is a 2D, looks just like Google Maps. And on it are placed dots for all their factories. Then if you click on a factory, you get a 2D plot of the floor plan, if it’s a one-floor factory. And if it’s multiple floors, you can go up and down the floors. So it’s 2D, although we could easily do a 3D plot. But the trouble with 3D is it’s ridiculously expensive to both capture and also to render. So 2D floor plans are cheap and easy and make most everybody happy.
So you see a 2D floor plan with all the machines on the floor plan, either with icons or with the actual layout of the machine. And then if you zoom in on the machine, you get a bigger picture in 3D of where exactly the sensors are on that machine. And then if you zoom some more, you get a time series of the history of the vibrations on that particular machine.
So you can go from a picture of Brazil to a detail on an individual machine with four clicks. And that means that’s what’s called detail on demand. And that particular customer loves the fact that you can get an overview and then go into the details.
Pete Wright:
My mind is spinning a little bit with the implications of such a system. Can you share a little bit on the back end? After the customer has sort of integrated Hopara, what is the practical impact of this kind of visualization for operations efficiency and so on?
Dr. Michael Stonebreaker:
The ultimate customer wants to avoid downtime on his factories, and so he wants to know, for example, if all the factories with red, coded red are on the coast, then is there something about the sea air that is influencing stuff? So he wants to see things that are not easy to see without a visualization. He, of course, also wants to know machines that are expected to fail within the next day, within the next three days, so he can plan a repair strategy. So he wants to get both kinds of information, both things that are not expected and then things that are expected but which a visual representation can give you a much better idea of what’s going on.
Pete Wright:
I imagine that ties directly into some aspects of competitive advantage. I imagine being able to respond faster to environmental issues like the sea air gives you a leg up in terms of keeping factories online. How else are you seeing that apply?
Dr. Michael Stonebreaker:
There are a couple other places where Hopara is very popular. One is in laboratory settings. Labs, such as run by all the big pharma companies, have hundreds and hundreds and hundreds of devices. Many of them are portable and many of them need maintenance. So what you’d like to know is when you need maintenance on machines. If a machine is not where it’s supposed to be, where is it? If one of these devices is a refrigerator and all of a sudden you get a temperature alert, you want to, lickety-split, send somebody over there to close the door of the fridge since that’s the most likely culprit. So you want to be able to monitor your laboratory and be able to react to problems. Hopara is very, very good at doing that, and we have a bunch of laboratory customers who have exactly that application.
You can imagine, although this is not currently a focus of Hopara, if you consider a hospital. And the problem is hospitals are very, very conservative institutions, so we will get there over time. But if you get a code blue, then the whole world descends on a particular room. What you’d like to do is just send the closest person with the skills that you need. And that’s something Hopara could do by simply monitoring where everybody is. Also, machines have a tendency, like EKG machines have a tendency to walk away from where they’re supposed to be. So again, you can keep track of where stuff is, and if it’s not where it’s supposed to be, you can find it. So again, hospitals are, in time, a great application of this sort of technology.
Pete Wright:
For those listening who are wondering, how do I get from where I am now to where Hopara visualizes me to be, what’s the technology behind Hopara? And how do I integrate it with what I have currently?
Dr. Michael Stonebreaker:
Well, assuming that you have sensors that are connected into a sensor network, the way hopara works is that you simply record these streams and put them in a postgres database or, in fact, your favorite SQL database. And Hopara simply reads that database in real time and does its magic. So all you have to do is put your data in a database and then install Hopara, so it’s very, very easy to use.
The problem most people have is that they’re not an expert on how they want to visualize the data. So Hopara engineers who’ve written many, many, many of such visualizations can be very helpful in getting users going by helping them decide what sort of drill down they’d like to use. And then often, we help them program that visualization. We have a very easy to use, user-friendly toolkit that allows you to create custom visualizations, typically within one day. The biggest problem that most everybody faces is getting their data from where it is now to a postgres database.
Pete Wright:
It’s incredibly impressive, and I think for systems folks, saying that you can build a system like this in a day might sound like a fairy tale. What is it that makes the process so efficient compared to other solutions?
Dr. Michael Stonebreaker:
The template of starting with a map, drilling into a factory floor, drilling into the details, drilling into time series, that paradigm is found everywhere. So we’ve written quite a number of examples of that paradigm. The thing that makes it so quick is you start from the closest thing that you have to what you want, and then you just have to edit that into what the user wants instead of having to build stuff from scratch.
But again, I want to caution everybody that that’s not the biggest problem. That’s never the biggest problem. The biggest problem is invariably, you have this data and it’s often sensor A records it in kilograms, sensor B records it in pounds. So there’s invariably data conversion that you have to deal with. Often, there are errors, missing data. Cleaning up your data is the biggest problem our users invariably face. But if you have clean data, it goes very fast.
Pete Wright:
Let’s assume the most common scenario where data is dirty. What is your recommended approach for using Hopara? Do you recommend people get your data house in order before you approach a Hopara installation? Or is this the kind of thing where, let’s go ahead and get it in Hopara and clean up from there?
Dr. Michael Stonebreaker:
You can do it either way, but my general recommendation is there are a whole bunch of tools that specialize in data cleaning and data integration. Hopara specializes in making the most of your data once it’s clean. So I would recommend using your favorite tool to clean up your data.
And in general, one of the places where enterprises are especially remiss is just everywhere in the enterprise, the biggest problem is invariably getting your data into one place in a consistent schema. So this is a much, much bigger problem than just monitoring your lab data or monitoring your factory floor. So in my opinion, a serious enterprise should be chipping away at the deferred maintenance of doing this.
Pete Wright:
One of the things you said earlier, you mentioned, when we were talking about the Brazilian example, the visualizations. I can see getting very excited about key visualizations that take Hopara beyond the traditional static dashboard. Talk a little bit more detail about some of those key capabilities, things that customers are seeing that really excite them about how they can see their data more clearly. Something like jump to visualization or callback functions, how do these things change the way users interact with their data?
Dr. Michael Stonebreaker:
You can imagine a single visualization is a way to wander around a canvas and get all kinds of information. So let’s suppose I am visualizing the location of Pete’s, your office. So I’m on a floor plan of your office and I wander into the room that you’re in. And what I really want to do is now jump to your home address to see what your house looks like. So in Hopara, you can move from one context to another dynamically and in real time. So you can move to your home address, look around your house, move to your kid’s school and so forth. So you can move from context to context, and that’s a very powerful way to see multiple kinds of data in a single visualization.
Pete Wright:
What do you find, when you sit down with somebody who’s using Hopara for the first time, surprises them the most?
Dr. Michael Stonebreaker:
That they can do it at all.
Pete Wright:
Okay.
Dr. Michael Stonebreaker:
Most people have dashboards on the brain and have no concept that you can do a lot better, or they have built into their psyche that visualizations are too expensive. Hopara strikes a great balance between being very affordable at both, at runtime and be very, very powerful. So it usually blows people’s minds to realize that they can do what we can do.
Pete Wright:
I can imagine. I sort of sometimes regret the next question because I’m asking you to think about the future of a thing that is still pretty brand new. But I am interested in how you chart next steps for Hopara. What is next on the horizon of features and capabilities, managing and adapting to compute costs against traditionally expensive rendering, those sorts of things? What is the Hopara vision for expanding capabilities?
Dr. Michael Stonebreaker:
Well, we find that most visualizations are very domain specific. The lab people want to see their data a particular way. The factory people want to see their data a particular way. So our strategy so far has been to try and be very, very persuasive in fairly narrow domains and build development tools that will be able to create those visualizations very efficiently.
But in my opinion, most everybody needs what we have. So we have not, for example, focused on road networks, which is an area where we could certainly focus. The people who have large farms want to be able to see an overview of what’s happening and then drill into the details. The people who schedule, for example, John Deere combines. So individual farms rarely own their own combines. The combines sort of move from field to field and they’re owned by somebody else. So if I’m somebody who owns a bunch of combines, I want to be able to get a big picture of what’s happening and then be able to drill into the details, especially if my combine fails. So it’s hard to think of examples at scale that don’t need this capability.
Pete Wright:
Big agriculture, you just blew my mind a little bit. I come from an agriculture family and just realized, oh my goodness, I get the Brazil example. I want to be able to look at the state of Oklahoma and North Texas, and I want to see where my equipment is at any given point and drill into that. That is a bit of a mind-blowing example.
Dr. Michael Stonebreaker:
Yep. So it’s hard to imagine applications that have a scalable data problem that wouldn’t benefit from overview first, then detail on demand.
Pete Wright:
Sure. This is fascinating. I feel like we could go on and on, but I want to respect your time. I know you have a hard out. I’d like to put some resources in the notes to accompany this conversation. Where do you recommend people go to learn more? Obviously, the Hopara website. Are there other resources, technical resources you’d like to share?
Dr. Michael Stonebreaker:
Let me send you an email with some resources.
Pete Wright:
You got it. A fantastic introduction. Mike, thank you so much for your time, and thank you everybody for downloading and listening to this first episode of our show. We’re deeply thrilled to introduce you to what we have going on at Hopara. Can’t wait to show you what’s on deck. On behalf of Dr. Mike Stonebreaker, I’m Pete Wright. We’ll see you next time.
