Yesterday I sent out the final version of the ESPM Graduate Student Survey Report to our student listserve, which I had been working on off and on over the last week with several of my classmates. Last spring, I coordinated the design and release of this survey of all ESPM graduate students as an effort to identify the key priorities and issues of students within the department. The survey includes both quantitative and qualitative questions assessing the quality and importance of 12 key areas relating to graduate student life, from advisor relationships to the financial support. It includes sections summarizing the data and comments for each of the department’s three divisions, and an Executive Summary that I authored explaining the survey’s purpose, methods, and results. We haven’t yet decided whether we will release the report publicly, but if we do, I’ll try to post a link to it here.
I just completed reviewing all of my coding data, which includes nearly 10,000 codes on over 2500 documents! My research assistant and I were very careful in our original coding of these texts and conducted intensive inter-rater reliability assessments throughout the coding process, but I wanted to double-check them nevertheless. We had both coded the texts of approximately 10% of the 245 cases, and reviewed, compared, and corrected our results for those cases as we completed them. I went back over those codes, and identified the codes for which the discordance was greatest. I then reviewed all of the texts identified with those codes across all 245 cases to ensure they were coded consistently.
I also did this using a large number of random spot-checks for all of the other codes we used, and compared them to similar codes for potential mistakes and overlap. I also reviewed all of the comments and memos within our coding software MaxQDA that we had generated about these codes during the coding process, and used them to further standardize the identified codes. This rigorous review process, which took the better part of a month, gives me increased confidence in the reliability and validity of my data, and all of my related analyses that will make use of them.