Open-ended research questions typically are not included in surveys because they are expensive and time consuming. I am currently working with a group of researchers at UC Berkeley’s UCNets project to develop a workaround to this problem by using structural topic modeling to analyze open-ended survey research questions.
Topic modeling offers a couple of important advantages:
1. Since topics are automatically inferred from text, the speed of analysis increases dramatically.
2. Hand coding and training of coders is labor intensive. Furthermore, since social phenomena often have multiple interpretations, reliability is often low. Topic modeling helps resolve these problems because the coding (the topics generated) is done by a machine.
The open-ended research questions I am currently analyzing using topic modeling include questions about how people make new friends, and the quality of the relationships to friends and family for help during a crisis.
Data is currently being collected and analyzed. If the technique proves to be effective, it will be incorporated in the research agenda.