It’s the time of year when a lot of teams have set their goals and plans for the upcoming year.
What does this mean for you? It can be a challenge to set goals, and I’ve always found it even more so for Developer Relations teams and activities. I think part of the reason for this is that the activities are hard to measure, or the way they can be measured don’t really fit to the ethos of Developer Relations. Take blog posts for example, one of them might be read by 50 people and solve a specific problem for 40 of them, and another might get 2000 views but not actually provide any real solutions. How do we measure the value of a piece of content like that?
Before I start I would recommend that you read the chapter in the Developer Relations Book by Caroline Lewko and James Parton on setting goals. It helps to understand how to think about this in the context of the organization you are part of. It doesn’t however go into lots of detail so I’m sharing this blog about how I am doing this myself in 2022.
To give some context on where I am, I joined my current company with a high level goal of bringing about a major change in the high performance developer community. In around 5 years we’ve come a long way, from a very small community of developers we have now brought our technology to some of the faster supercomputers in the world as well other areas like the automotive industry.
It was goal setting that helped me to focus each year for myself and my team’s activities to make this happen.
Each year I have made a plan that tried to align as much as possible with what I understood to be the goals of the company, and when you work in a smaller company (for me < 100) it’s common for these goals to change over time as the company pivots rapidly. I wanted to spend some time writing about how I go about the process of defining both high level goals, and then mapping that to more specific activities and metrics for the year.
Part of my team’s responsibilities are to help increase the adoption of the SYCL standard and look after its developer community, but it’s a little more complicated than it seems. SYCL, the technology we advocate for, is an open interface. In practice a group of experts from different organizations across the world define the API specification so that others can implement these APIs for different hardware targets (generally GPUs made by different vendors). What that means is that anyone can use SYCL, and in fact multiple organizations have implementations of this standard (including the company I work for). This means that most of the activities we do benefit the whole community and aren’t always directly channelling developers to our own implementation. Some might see this as helping the competition, but I have been working in this field long enough to understand that growing a healthy community involves building trust and healthy relationships and helping developers find the right solution for them. This for me is a crucial difference between Developer Relations and something like sales.
In my 15 years of experience in “Developer Relations,” adoption and community management are usually the main drivers for hiring, but the tricky part is figuring out the best ways to make these happen. The motivations for adoption and community management can also vary drastically depending on the organization too.
I like to take a step back and look at the bigger picture when setting goals for the coming year. Driving adoption can involve many different activities — support, training, presentations, tutorials, videos and conference presence to name a few. I also find that “community” can be even trickier to figure out. How do we know how healthy and happy our developer communities are?
I am a pragmatic type of person and have a tendency to jump into defining activities and start getting into too much detail too quickly; but I realise it’s important to stop and take stock of where we currently are and where we are trying to get to. Things change year to year, especially in the midst of a pandemic.
I’ve seen it referenced in lots of presentations, books and blogs about DevRel, but it really is important to figure out how your team’s goals map to the objectives of either the whole company, or at least the division you work in (if you work in a larger company). What this boils down to is, at a high level, why are we doing what we are doing and how does it directly or indirectly bring value to the organization?
So, let’s start getting to the fun part where we define the goals, then map these to metrics and activities. I’ll use some more generic examples to make them easier to map to your goals, and explain my process of mapping this down to lower level strategies and individual activities. The KPIs in particular help your organization understand the value you are bringing, if what you are doing is effective and brings a way for the dreaded measurement of performance (if you are lucky enough to receive a bonus).
Let’s start with some high level goals.
The company I work for has defined the following high level goals that align to our overall vision for the company.
- Drive adoption of our primary technology
- Help developers migrate from other technologies to our primary technology
- Grow and monitor the activity of the developer community that is using our primary technology
Sounds simple right?
Let’s break some of these down to get more granular.
Goal 1 — Drive adoption of our primary technology
This goal exists to help educate developers about SYCL, to promote it and demonstrate my company’s leadership with the technology.
The target for this goal is to educate a given number of developers through the year about SYCL at events and online activities.
At a high level we can say we want to reach x developers this year.
How do we do this?
I am choosing a combination of event activities and online educational content based on the team I have in place and the budget available.
Activities
- Organize training at major conferences
- Publish an online training course
- Make relevant presentations at developer conferences
- Write technical articles and blog posts
How will we measure this?
We can use some accurate, and inaccurate/rudimentary measures to get an indication of what our performance and reach is.
Metrics
The following metrics we know can be measured accurately, and reflect that we are performing the activities we said we would do. These are most useful to measure that we are doing the things we said we would. They are either done or not and we have full control over that, and we can assign numbers to these quite easily.
- Number of training sessions held
- Online training course published
- Number of presentations at conferences
- Number of blog posts
These next metrics will indicate whether we are getting close to that x developer reach target for the year. It’s important not to take these numbers too seriously because they are generally out of our control. For next year we can also at least, for example, get an idea of whether we need to target more events to reach our high level target. We can estimate these numbers and compare with the numbers we gather through the year.
- Number of attendees at training
- Registrations for an online training course
- Estimation of audience at presentations
- Views of blog posts
Goal 2 — Help developers migrate from other technologies to our primary technology
This goal exists to grow our developer community by specifically targetting developers using CUDA, and helping them to migrate to SYCL. We will try to do this by enabling developers to understand the benefits of migrating to SYCL and educating them on how to do that.
At a high level we can say we want to measure that y organizations indicate they will migrate from CUDA to SYCL based on our activities this year.
How do we do this?
I choose a combination of specific targetted event activities, online content and support to specific organizations. The online content is used to aid the migration path for developers so does contribute to the high level goal.
Activities
- Deliver workshops targetted at user groups
- Support developers migrating their code
- Develop porting and optimization guides
How will we measure this?
Again, we can use some accurate, and inaccurate/rudimentary measures to get an indication of what our performance and reach is.
Metrics
These metrics can be measured accurately, they are either done or not and can be assigned numbers.
- Number of workshops held
- Number of support requests responded to
- Number of guides created
These metrics we have more limited control over, but help us to understand if we are reaching the y organizations.
- Number of known organizations migrating
- Number of attendees at workshops
- Number of follow up contacts from workshops
- Page views for guides
Goal 3 — Grow and look after the developer community that is using this technology
This goal exists to ensure that we have a healthy and happy developer community.
At a high level we want to estimate the number of developers using SYCL is growing and that developers are getting the supprot they need in online channels.
How do we do this?
I choose to track some statistics about the developer community and to understand what organizations are using SYCL. I don’t want to put specific targets on these metrics as I don’t really have control over how that will happen and limited historical data. So this goal is more about gathering data and getting an understanding, this data can also be used to showcase developers from the community that are working on applications, research and open source projects.
Activities
- Define the organizations using technology
- Track open source projects using technology
- Track research papers published using technology
- Track discussions happening about technology online
- Track presentations made about technology at conferences
How will we measure this?
The measurements I am using can be compared to previous years information if it is available. For some of these I can do that, but for others we can use them next year to compare with if the goals stay the same.
Metrics
- Number of organizations using technology
- Number of open source projects using technology
- Number of research papers published using technology
- Number of discussions happening about technology online
- % of questions asked about SYCL online that are responded to
- Number of presentations made about technology at conferences
Building a roadmap
The next step from this I like to follow is to build a roadmap for the year that helps paint a picture of what the year looks like. If we are writing 8 blog posts a year that means 2 every 3 months for example, and if we are doing 4 developer workshops then that is 1 every 3 months. This will help us to map out how these activities will be done through the year. It also helps to visualize whether or not we have bitten off more than we can chew, is what we are targeting realistic?
So to finish things off, here’s a table showing how we might, at a high level, measure these activites through the year. Here’s an example for the first goal.
I am sure there are lots of different ideas and techniques for setting and tracking annual goals, so let me know how you do it in your team, and also how it scales when your team grows too.
