bits on data

Big Data Hype

In the wake of the early social media boom and web 2.0, 'Big Data' was the belle of the tech buzzword ball. Google’s seminal Google File System and MapReduce papers spurred engineers at Yahoo to develop Apache Hadoop to handle a massive influx of data including user-generated content from cat videos to every single thought that entered their mind. The hype sold the idea that companies would be able to provide faster improvements and higher customer retention by leveraging this terabyte-to-petabyte-sized data to better understand their customers and how to personalize advertisements down to individuals. The tech industry was tripping over itself to jump on the Hadoop bandwagon.

Companies quickly built colossal Hadoop clusters of data on HDFS, resulting in the digital equivalent of a hoarder's house where data would rarely – if ever – see the light of day again. As this data accrued, companies threw money at software engineers, urging them to hastily retrain themselves to use a distributed MapReduce framework. They churned out code at a frantic pace, creating some of the buggiest, most convoluted software known to the data world. Data pipelines resembled a spaghetti junction, and despite reliability improvements of the storage and infrastructure layer, the output of these systems was rarely accurate and never timely. Questions would be asked of this system, and weeks or more likely months later, the answer would come with little to no interest in it by the time it returned.

Unfortunately, rather than catching on to the lack of understanding, companies leaned into their misunderstanding as something that would work itself out, and everyone spoke about hypothetical usage of data rather than the harsh realities everyone shared. In the span of 2010 to 2012, every company adopted a Hadoop cluster and was paying for a large distributed database that nobody could use efficiently. Being a part of this data craze has certainly never been dull, but has evolved so rapidly, that we’re still in a bit of a tailspin to understand the ever-evolving data landscape from this era.

New day, new hype

Since my days as a Software / Data Engineer, I moved to a new role that is as fantastic and novel as it is perplexing. Much like the early Big Data days, my role in Developer Relations (DevRel) is experiencing the same hype and confusion that leads to a lot of chaos for employees and employers alike. We’re witnessing a similar explosion of these jobs in companies hiring Developer Relations professionals without exactly understanding why they need a DevRel function. Being on the front end of the big data wave, I couldn’t help but notice the stark similarities between the lack of vision during the Big Data hype and the ongoing DevRel hype.

A handful of SaaS companies that create developer tools (aka developer-first companies), follow the lead of predecessors like Twilio, Confluent, Salesforce, and Google by growing and attracting large user bases with hopes that these communities will quickly become a wave of sales leads and see the same level of success. The problem with novel approaches like Big Data and Developer Relations is that they take time to mature. The companies that started these eras, did so organically with little to no need for culture shifts. They were effectively solving a problem and the conditions were right for a new approach to flourish in that given context.

When businesses see any kind of success, they often want to replicate it without doing the upfront work of understanding if the same formula addresses any of their company's problems. After various reports from other companies following the lead of the first company begins, a cargo cult behavior develops and the hype cycle takes off as it did with DevRel. With dollar bills in their eyes, companies posted positions for engineers that had domain knowledge, community trust, and the ability to sell ideas. Business leaders recognized they needed someone who represented their target market, but there was a key aspect they missed from the DevRel being practiced at the earlier developer-first companies. We’ll dive into that in a future post, for now, let me give a bit of history on Developer Relations before it was even called that.

Quick side note: Developer Relations is a subset of a much broader community relations trend. Each community has its own set of rules to follow so while some of what I’ll say applies to the broader trend, this series will dive into the unspoken rules of developer communities I’ve learned through Developer Relations.

History of DevRel

DevRel is a term that encompasses strategies and tactics aimed at building and nurturing a community of mutually beneficial relationships between people within a community, but also organizations and developers. The concept of DevRel can be traced back to the 1980s when Apple is considered to have created the first DevRel program. Mike Murray, who coined the term “software evangelist,” was instrumental in persuading third-party developers to develop software and applications for the Macintosh platform. Mike Boich was Apple's first Software Evangelist for the Macintosh project and hired Guy Kawasaki, who would become Apple's Chief Evangelist and popularize their DevRel program (again, probably not called that at the time).

DevRel continued to grow in popularity in the late 1990s and early 2000s but was primarily a role only seen in tech giants like Microsoft and Sun Microsystems. In the mid to late 2000s, with the rise of open-source software and web technologies, the role of DevRel evolved significantly. Companies like Google, Facebook, and Twitter began to see developers as partners who could help them grow their platforms. All of these companies recognized the importance of building strong relationships with developers. They understood that developers were not just consumers of their products, but also influencers and creators who could extend and enhance their platforms. These companies started to create programs to engage with developers, providing them with resources, tools, and support.

Despite its existence in big tech for nearly three decades, mainstream adoption of DevRel started around 2013, with companies like New Relic, Twilio, EngineYard, and SendGrid popularizing the developer-first approach. The term advocate started to replace the prophetic perception of the evangelist, signaling that developer advocates were there to listen and not just talk at you. The broader adoption of DevRel has been a turbulent ride fraught with the same issues were just now seeing for engineers in the AI space.

Now that you understand DevRel, in the next installment of this series, I will talk about the way that developers adopted tools before communities, open-source, or customer reviews. I’ll cover how long-established methods to sell products to developers diminished in efficacy based on how people were buying products outside of business. I’ll then dive into some hand-wavy game theory between community-driven marketing versus vendor-driven marketing to alter behaviors that produce positive-sum outcomes. I’ll finally cover my personal vantage point of how DevRel should be structured, particularly around open-source communities.

Stay tuned, Bits