Finding New Connections in Big Data

Dateline: May 23, 2014 Welcome to our Friday WRAP – one thought-provoking idea to think about over the weekend. Value from big data opportunities can be found in some of the most unlikely places.  Recently, Fast Company published an article, “How Facebook Predicts Who You Know Using Old Yahoo Code,” that highlighted a big data opportunity at Facebook.

Every computer science student learns the basics of graph theory–a set of mathematical abstractions for modeling networks and the connections within them. But few developers ever work with network data on the scale of Facebook’s social graph, with its more than 1 billion users and hundreds of billions of connections, or edges, between them. So in 2012, when Facebook engineers started thinking in earnest about ways to efficiently process all the network data they’d amassed, they found themselves in uncharted territory…. But Facebook still wanted to be able to measure basic statistics about their users’ connections, and efficiently do computations like measuring affinities between users to suggest potential friends, or finding mutual friends of sets of users.

The article goes on to explain how they were able to harness their data, use graph theory, connect with open source graph theory toolkit Apache Giraph, and ultimately to create an algorithm to make connections where none had been made before.  These connections were a key to extending the usefulness, and to some the ‘magic,’ of Facebook’s services.  But this idea, making connections where none were obvious before, is something every company can use. Where would discovering new connections in your data be of value in your organization? That’s a WRAP!  Have a nice weekend!

Speak Your Mind