By Ted McEnroe, Head of Research, The Community Roundtable
What goes into the community research that The Community Roundtable does in the field of community management? Data, collaboration and experience.
It’s an exciting day for me. Today I shipped out the first draft of the Community Careers and Compensation 2015 reports (yes, there are two versions) to the graphic designer to get the layout done. That will be followed by more edits, more layout tweaks and more edits after that, leading up to the final report release in the middle of next month.
I’ve written before about the cooperative effort that goes into creating the questions for each of our core research offerings. But for those of you who took the time to complete the survey, it may seem like things get awfully quiet between the survey period and the release. And you’re right. It gets quiet – like library or research lab quiet.
Once the data is in and the survey period closes, we have a lot of data to go through. For the CCC, as an example, we end up with about 65,000 individual data points to review, and formulas to write off those bits of information. For the State of Community Management, the number is even higher. Data is entered, formulas are written, and then we do through the painful process of cleaning. That means the members of the research team begin going through line by line looking for red herrings, typos, misinterpreted replies, and data in improper formats. These range from respondents who may feel that all 20,000 members of their external community should be counted as volunteers, to cultural differences, like using 1,5 versus 1.5 for headcount in a community team.
Jillian Bejtlich from our team is our spreadsheet wiz, and we go through the data, the myriad ways we can parse it to test hypotheses, find patterns and answer questions. Rachel Happe, of course, joins the fray as we go through, and depending on the project, other members of the team take a look as well. We evaluate, look for those “Aha!” moments, and over a period of days and weeks, discussions and debates, settle on tentative key findings for the report.
The data supporting those key findings – and every finding has supporting data – are checked and rechecked, and the findings and their supporting evidence are presented to the team for approval. I love and hate this part, because already a lot of work has gone into it, and the challenging and wordsmithing are both valuable and a discussion of how ugly my baby is.
Key findings in place, we bring in additional data (if there is any), clean and recheck the data to make sure nothing new and weird has developed, and we begin to build the major graphics for the report. Each report has a handful of graphics we know will be central to the document – in the SOCM, it’s engagement profiles and variations on the Community Maturity Model. In the CCC, it’s the salary table and the Community Skills Framework. The design for these graphics begins even before we have data in place, and continues as data comes in, to make sure that the end graphics tell an effective story.
From there, we build out the boilerplate data sections – those we know we are going to have in each SOCM or CCC. It’s the same process. Examine, analyze, compare, lay out, write copy, and it gets repeated for each element. While we go through this, we are continuing to build two lists – one of data points that deserve further exploration and one of data points that can be used in other projects – since after all, not every piece of data and every observation can fit in a single report. We also make sure the information we gather is making it into other aspects of our work – questions we can answer on TheCR blog in the upcoming months, webinars and calls we can plan, and elements we can add to our online training and other assets. We also use this information in less formal ways – when you ask us a question on Facebook or Twitter, we are drawing from our experience and knowledge of the data – not just hypothesizing based on anecdote. Members of TheCR Network, too, are able to take advantage of our insights and those of their peers as they dig into the thorny issues of community.
From there, we fall into a more traditional editorial process. Write. Edit. Rewrite. Approve graphics. Rewrite. Edit. And so on. In the end, we end up with reports full of data and analysis that can answer your questions and help you plan your community or career strategy. And a whole lot more.
It’s a lot of work, but it’s a joy to do. And none of it would be possible without the input of hundreds of community management professionals like you – who take the time, share their data and help us create a whole that is far greater than the sum of its parts. Communities are complex organisms, and there are no magic formulas – many of us have learned that through experience. There’s also no one piece of research that will answer every question. But bit by bit, together we’ve created a body of work that can guide communities new and old, large and small, on the road to success.
Oh – and keep an eye out for the Community Careers and Compensation 2015 summary report next month. I can’t wait for you to meet our newest baby.
—
Interested in learning more about research at TheCR? Sign up to contribute to our research campaigns, or just receive updates on our new releases.
