Or listen in your favorite podcast app
Apple Podcasts / Google Podcasts / Spotify
More than likely you have uttered the phrase, “Yeah, I’ll just Dropbox that to you.” The turn of phrase has become so popular thanks to the capabilities Dropbox has offered to the masses. People around the world can share and store documents, pictures and information simply by dragging a file into a folder. But how does that file get stored, and how do those precious items live in a space that seems almost endless? Andrew Fong is the Vice President of Infrastructure at Dropbox, and on this episode of IT Visionaries, Andrew dives into the nitty-gritty of how he helps keep those mementos accessible anywhere. Plus he describes how a small task force formed in 2013 shaped the future of Dropbox forever and the importance of scale.
Key Takeaways
- Keep Your Data Close and Your Servers Closer: By building its own data servers, Dropbox was able to eliminate its reliance on third-party servers like AWS. When you control your servers, you’re able to manage the amount of data you intake and, in return, how you use that data.
- Core Strength: Think about the items that are core to the foundation of your business and then invest in those things. For Dropbox, one of those core functions is the need for its system to be reliable and durable. If you don’t understand what the core principles of your business are you can’t invest in the right things
- Time to Reload: Once a project is up and running, you need to make sure your team remains invested and energized. Don’t wait too long to switch things up. If team members are excited about other opportunities, give them those chances.
For a more in-depth look at this episode, check out the article below.
More than likely you have uttered the phrase, “Yeah, I’ll just Dropbox that to you.” The turn of phrase has become so popular thanks to the capabilities Dropbox has offered to the masses. People around the world can share and store documents, pictures and information simply by dragging a file into a folder. But how does that file get stored, and how do those precious items live in a space that seems almost endless? Andrew Fong is the Vice President of Infrastructure at Dropbox, and on this episode of IT Visionaries, Andrew dives into the nitty-gritty of how he helps keep those mementos accessible anywhere. Plus he describes how a small task force formed in 2013 shaped the future of Dropbox forever and the importance of scale.
Dropbox is a file sharing service best known for its ease of use, scale and reliability. But it’s the last of those three things that drives Fong every day. Prior to joining Dropbox, Fong worked at AOL and YouTube, where he focused on site reliability, and when he joined Dropbox in 2012, he was hired in a similar role.
“I think of reliability as a core feature that we have to have,” he said. “When we go into any type of planning, reliability has to be at the core, and durability as well, but reliability has to be the number one thing that we are worrying about as an organization.”
Fong stressed that the core reason Dropbox focuses its attention on reliability and durability is that its users depend on those core abilities, mentioning that if the servers are neglected or malfunction in any capacity, the amount of time it takes to recover from those missteps could be detrimental to the overall operation.
So in 2013, Fong and Dropbox constructed a team built around one simple goal: to build a file storage system that was both reliable and durable, while ending the company’s dependency on AWS. If the team built the product right, Dropbox would be able to increase its data storage capacity through this new technology. They nicknamed this task force Magic Pocket.
With a goal in mind and an idea in place, Fong and his team rolled up their sleeves and went to work. First by transferring data from their old system into Dropbox data centers, while operating under a tight deadline.
“The primary thing to understand about Dropbox and S3 Storage [Dropbox’s old storage system] is they solve fundamentally different problems,” he said. “Magic Pocket is a block storage system that’s item-potent. S3 is not an item-potent storage system from its interface and allows you to change, update, move, and do a lot of different actions to the blocks restored there.”
According to Fong, data that a consumer uploads to Dropbox’s system — whether it’s a file, picture, or a movie — is classified as “warm data.” Meaning the team at Dropbox knows the user is going to need access to that piece of data at some point. Fong acknowledged that the old storage system worked with those capabilities. But, if Dropbox wanted to scale at the rate the company thought it could, it needed to invest in its own storage capability.
“On a financial side, as well from a technology perspective, keeping that data close to us allows us to do other things with it,” Fong said. “Because we weren’t going to have to pay network transfer fees. We weren’t going to have to deal with what Amazon was dealing with, consistency. It gave us some different levers to pull. And we thought that was a very unique opportunity for us from a technology perspective to actually create a little bit more leverage for the product.”
While undertaking the Magic Pocket project was a stressful process, Fong reiterated that it’s not something that every company should do.
“Managing a system like this, you need to go into this with your eyes wide open,” he said. “If you want to build a system like Dropbox, especially at the scale that this operates at, because if you don’t have the scale, the unit economics don’t kick in. What I mean by unit economics, the cost per gigabyte, you don’t actually get that value from the vendors you’re going to work with until you start buying in volume. So you need to have a volume that actually makes sense before you even go down this journey.”
While Magic Pocket was incredibly successful and helped set the company down a path toward rapid growth, there are some things that Fong would do differently if he was afforded a second opportunity at the project. One of those alterations revolved around the team Dropbox constructed to build Magic Pocket.
“What ended up happening culturally is that we had created a team that was incredibly talented, incredibly deep, and was a team of the best engineers we could possibly put together,” he said. “If I did this again with the leadership we have, we would have reloaded that team in a different way and been a lot more intentional about finding the next thing so that we could get people excited immediately post-launch with a different set of people working on the next piece of it.”
The goal for Fong behind that mindset is to make sure that innovation is constantly being fostered and the core beliefs of the project are always being met. He related the idea to settlers and town developers. Where you build something to get it running, but you ultimately need individuals on the project who are in it to see it through to the next phase.
“We had people that were much more the go-figure-it-out [type],” Fong said. “Then let’s go look at this landscape. Let’s understand it. Let’s build something and let’s get it out the door, but we never moved to the next phase, which was let’s make sure we operate it and operate it well for an extended period of time. So we didn’t get that operator mindset into the team early enough.”
Once Dropbox made the decision to reload the team, and in return allow engineers to move onto new projects, Magic Pocket flourished. Going from a project that was just up-and-running, to something that helped change the landscape of Dropbox’s capabilities. It’s a lesson that Fong said if you’re with a company long enough and get to experience wins like this, the hours spent working on that project were well worth it.
—
To hear the entire discussion, tune into IT Visionaries here.
