3/29/2023 0 Comments Melissa data ip locatorThat was a “eureka” moment for me: I realized that this threat to online research went far beyond my project and that to protect the integrity of online research, scientists must become aware of cyber threats. Census Bureau to discuss the issue of bots in online surveys. As a result of this experience, I have had phone calls with researchers from various academic institutions, industry, and even the U.S. Many had experienced bot infiltrations of their own. To my surprise, nearly 1 million people shared, read, or commented on this thread. I had hoped that sharing my story might save at least one person from going through what I had. Melissa Simone, PhD (She/Her) September 17, 2019 I'm putting my pride aside because I think #dataintegrity is and will be a growing issue in survey data and is not discussed enough (1/n) I haven't shared this story publicly because I felt a bit like it was my fault. My online #researchstudy was recently infiltrated by bots. Instead of being angered by this experience, I’m grateful I had it, since it has opened up a new avenue of research for me.Īfter I had paid the human participants, I took to social media to share my story and lessons learned in what became a viral Twitter thread. I ultimately paid only the human participants for their time, and plan to use the money I wasn’t hoodwinked out of to reopen the original study on LGBTQ+ communities and eating behaviors, with much improved bot-detection tools. Had I not done this analysis, I might have unwittingly paid bots about $6,000, a big chunk of my research funds, for unusable data. I flagged participants with two of more of these infractions as potential bots.Īmong the responses I received during the 12 hours the survey was live, only 11, or about 3%, were not flagged as bots. These included impossible timestamps not answering required questions or requests, such as consent identical survey or open-ended responses from different “participants” inconsistent responses to identical questions not completing the survey data where it shouldn’t be, like responses to questions about being a cisgender woman by participants who identified as cisgender men impossible data values illogical responses to open-ended questions and completing the survey materials impossibly fast. As I dug deeper into the data, it became clear that only bots could have completed the large number of false responses I received in such a short period of time.ĭuring several hundred hours of damage control and bot hunting, I developed several unique coding schemes to detect the electronic signatures of potential bots. I immediately froze the data collection to give myself some time to investigate further. I was incredibly upset and worried about the potential impacts this could have on my study results, funding, and reputation. I initially thought that the study was being trolled by an individual or a small group of people. My first clue was identical and somewhat bizarre responses to open-ended questions. I was interested in whether sexual orientation and gender influenced eating in ways that we have not yet conceptualized, so I developed a set of open-ended questions to learn more about the lived experiences of individuals who identify as LGBTQ+ and their eating behaviors. It consisted of validated self-report questions to measure eating disorder behaviors, exposure to discrimination, psychological well-being, social support, and a number of demographic variables. That threat wasn’t on my mind when my study went live. But these tools have become targets for bots and professional survey takers, which are flooding online research and threatening data integrity. With the growing availability of machine learning and other big-data methods, there are few easier and more cost-efficient ways to collect data. Online surveys and questionnaires let researchers like me collect large, geographically diverse samples relatively quickly. Little did I know I was actually launching a battle with bots. I launched an online study earlier this year aimed at understanding the processes that influence eating behaviors and eating disorders among individuals who identify as LGBTQ+. Exclusive analysis of biotech, pharma, and the life sciences Learn More
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |