Research Training - gothgirl.systems

*Thursday, 2nd July 2026 - 09:30 UTC+01:00* --- > [!note] > Break is around 12:30 for lunch, with an aim to finish the session by 15:30. --- # Overview - [[#What is research?|The Basics & Today's Brief]] - [[#Qualitative Research|Qualitative Research]] - [[#Quantitative Research]] # What is research? A deliberate way of finding things out, designed so what you learn is more likely to be true. ## The three things that make it research - Intent - You decide what it is you need to know before you look for answers. - Method - Distance ## Two main methods of research - Qualitative - Used to find out why and how. Smaller numbers but greater depth. - Quantitative - Used to find out what, how many, and how often. Bigger numbers, best suited for scale. ## What don't we know that that we would need to, before we could research this properly? Let's say it's for an event that is hosted in the community We'd need something like... - Relevant or suitable dates - What would be the best date that could be chosen such as to maximise the attendance to such an event? - Relevant of suitable times - What would be the best time of day on said date that could be chosen such as to maximise the attendance to such an event? - Subjects - What would be the subject of the event? - Suitability of contents for the target audience - Would the contents of the event, in respect to the event itself, be suitable for the target audience that any advertising would be aimed at? - Locations - What would be an amenable and appropriate location for such an event to take place? - Interactivity if applicable - Total cost - What would be the total cost of either: - Attending the event itself - Travelling to the event itself - Both Some can be answered through means of data. Some can be answered by asking someone. ## The Hypothesis There may be hypotheses as to the outcome for an objective. Possible hypotheses could include, using the example of an event's outcome: - Weather - It could be too hot for the attendees to be comfortable attending. - On a related point, there may be too little or no shaded areas at the event for the attendees or the organisers to cool down. - The risk of heatstroke and sunburn would be a factor. - On hot and sunny days, people might choose to do other things instead, such as: - Sitting in the shade - Sitting in a beer garden. - It could be too cold for the attendees to be comfortable attending. - In rare cases, the roads and pavements could be icy. - In rarer cases, there could be snow that may result in the cancellation of the event altogether. - There could be heavy rain that may result - This leads into the possibility of flooding, which would necessitate the cancellation of an event altogether. - Insufficient or Ineffective Advertisement - Insufficient Advertisement - Only reaching people organically would be insufficient, meaning that paid advertisement on behalf of the event organisers would be necessary. - Ineffective Advertisement - Inappropriate day of the week / time of the day - Both of these factors would affect attendance to the event, individually and collectively. - Day of the week - Hybrid work patterns may be a factor, as many choose Friday as their working from home day. - Time of Day - The time of day may make it difficult for people to attend due to work commitments or other pre-existing events. - For example, someone may not attend the event if they had a pre-existing commitment in the form of a Spanish lesson halfway across the city either at the same time or ± 30 minutes of the event. - Event-related Costs - There may be a cost associated with attending the event, which would dissuade people from attending. - There may be a cost associated with travelling to the event, relative to where the person intending on attending the event may be travelling from. - The further away from an event the person is, the higher the cost associated with travelling to the event would be, and as such would decrease the likelihood of that person attending. - There may be no problem with getting to the event, but if the person is travelling by car, the cost of finding parking for the duration of the event may be prohibitive. - Conversely, if the event is free, it may signal to people that there is a lack of commitment towards the event by the organisers themselves. - Outside events - Nuclear War - Would obviously cause cancellation of the event, as everyone would have bigger things to worry about. - Volcanic eruption - Supervolcanic eruption - Hurricane - Flooding - Topic of the event is unclear or not understood - Irregularity of events #### The risks of theories If the only questions asked are ones that test the theory, chances are you'll find evidence that you're right. This is confirmation bias. ## How to define a research question - Specific - It points to one thing, not everything - Open - It doesn't assume the answer - Actionable > [!example]- Examples > - Why didn't the brunch work? > - Did the morning timing put people off? > - Why did people who signed up not attend? --- # Quantitative v. Qualitative - Which is best ## Qualitative Research ### How many and how long? Qualitative is about depth, not volume. Only a small sample is needed, but you do need longer with them. Studies show that 5 one-hour depth interviews is enough to uncover 80% of common themes. ### Identifying groups Even if you have a list of names in front of you to work from, you first need to identify the different audience groups your sample needs to include. #### What groups of people would you want to speak to, and why? - Those who attended - Those who signed up but didn't attend - Partners - Followers on applicable socials - Applicable groups in the wider community (for example, through platforms like Swagbucks) ### Topic Guides A topic guide is not a script; it's a structure to keep the interview purposeful while allowing for natural conversation and flow. > [!note]- Ways questions lead without meaning to > >- Was the timing difficult for you? >- What put you off the event? >- Was it the timing or the location? > [!example]+ An Example Guide > > **Opening** >> Thanks for taking the time to talk with me, this won't take long. We're just trying to understand how people experienced the invite for the brunch. > > **Open** >> Do you remember seeing the invite for the brunch? >> What was your first reaction when you saw it? > > **Specific** >> What made you decide not to respond either way? >> Was there a point where you thought about going? >> What happened after that? >> What do you remember about the invite itself? > > **Probing (used live, not pre-written)** >> Tell me more about that. >> What do you mean by that? >> What happened next? > > **Close** >> Is there anything that would have made a difference to your decision? Anything else you think we should know, even if I haven't asked about it? > [!example]+ Example - Someone Who Signed Up, Didn't Attend > > **Open** >>Thanks for taking the time to talk. This shouldn't take long. I understand that you have signed up for the event, but never attended it. > >**Opening** >>Do you remember seeing the invite for the event, and how did you feel initially when you saw the invite? > >**Specific** >> What was it that made you decide not to attend the event? >> Was there a point where you thought about going to the event? >> What do you remember about the invite that had grabbed your attention? > >**Probing** >> What happened? >> Any pre-existing commitments? > >**Close** >>?? ### Interview techniques - Allow for silence - People need a few seconds to think about their answer, don't jump in and fill it. - Watch physical responses - Nodding more to some answers than others can signal which answer you were looking for - Don't complete sentences - Even if it's helpful, those are your words, not theirs. - Avoid suggestions - If they pause, don't offer a suggested answer for the questions ![[Research Training - Interview Notes]] ### Topic Write-Up > [!example] > **Headline** > For some, the brunch wasn't a priority. Not a no, just a not now. > > **Evidence** > "I just didn't get round to it." > > **Recommendation** > As we know multiple reminders are sent, this suggests there is a relevance, or framing issue. As the invite list is growing, it may be worth understanding what topics would be worth ![[Research Training Interview Write-Up]] --- ## Quantitative Research ### Closed v. Open Questions > [!note] Closed Questions > Fixed option, easy to count at scale and easy to compare. > [!note] Open Questions > Their own words, harder to analyse at scale but can catch things you hadn't thought of. > However, people are lazy in survey mode and don't always engage with this type of question meaningfully. #### Writing a good closed question - Exhaustive - Covers the full range of possible answers - Mutually exclusive - No overlap between options - Balanced - Not loaded towards one answer > [!example] An example closed question > > Which one of these best describes why you didn't attend? > > - Clash with work or personal commitments > - Didn't see the invite, or saw it too late > - Time or day didn't work for me > - It didn't feel relevant enough to prioritise > - Something else (please tell us) ### Choosing the right answer format - Single response - "What was the main reason you didn't attend?" - Use when there's usually only one dominant reason, and a single answer is realistic. - Multiple response - "Which of these applied to you?" - Use when several things can be true at once. - Ranking - "Rank these in order, most to least important to you" - Use when you need the whole hierarchy. > [!note] Closed Question > > Which of these times for the event would work best for you? > > a.) 9am to 11am > b.) 11am to 1pm > c.) 1pm to 3pm > d.) 3pm to 5pm > e.) Some other time (please tell us) > [!note] Diagnostic Question > > Do you remember seeing the invitation email? > > a.) Yes > b.) No > c.) Perhaps > d.) I can't remember > [!note] Decision Question >> [!example]+ Decision Question 1 >> Given what has happened in the event, has it affected your decision to attend any following events? >> >> a.) Yes. >> b.) No. >> c.) Maybe. >> d.) Unsure. >> e.) Something else (please tell us) > >> [!example]+ Decision Question 2 >> Would you want the event to be hosted in a better location the next time it is held? >> >> a.) Yes >> b.) No >> c.) Unsure. >> d.) Something else (please tell us) Note - Not sure if those two even count as decision questions. Had a fucking nightmare thinking of them in the first place. Correction - They were diagnostic... and the second was a little bit of a loaded question. Eh... shit happens. ## Principles for Survey Build - Order - Start easy and low-stakes to build momentum - Move to the specific, harder questions in the middle. - End with anything sensitive, or the open "anything else" - Length - Every extra question increases drop-off - Before including a question, ask: what will we actually do with this answer? - If you can't answer that, remove it. Who you need to hear from and who you actually reach aren't always the same group. How you reach people shapes who responds #### Different routes for different people - Email / CRM - Reaches people who have already engaged directly - Social media - Wider reach but unstructured and skewed towards existing followers - In person - Only covers those encountered by the day. - Third party - Reaches new people but can be costly and very depending on the criteria. ### Realistic response rate Sending to 50 people does not mean 50 answers. A good response rate for something like this is 20-30%, not 80-90% Plan the sample size you send to with that in mind, not the number you'd like back. ### Knowing who's who If you want to compare groups later (e.g. attendees v. no-shows), you need to know which response came from with group. Either ask directly in the survey - a simple "*Did you attend?*" ### Interpreting the results - Simple count - For each question: how many people picked each option? - If your platform doesn't do this for you, turn it into a percentage as it's easier to read and compare. - Work out the dominant response - Compare - Does the answer change on who's answering - Attendees v. no-shows - Previous attendees v. first-time invitees - This is where "knowing who's who" pays off > [!example] Example Finding >> [!note] Table of Results >>| Reason for not attending | Overall | Previous attendees | First-time invitees | >>| ----------------------------------------- | ------: | -----------------: | ------------------: | >>| Didn't feel relevant enough to proiritise | 47% | 18% | 61% | >>| Clashed with work or personal commitments | 26% | 27% | 26% | >>| Didn't see the invite, or saw it too late | 15% | 9% | 17% | >>| Time or day didn't work | 9% | 41% | 0% | >>| Something else | 3% | 5% | 4% | > >**Headline** >The largest share of responses were due to non-attendees not considering the event relevant enough to be a priority for them. However, previous attendees cited that the time or day did not work for them. > >**Evidence** > >In the table, it is shown that nearly half of overall non-attendees considered the event to not be relevant enough to their interests to be worth prioritising, while a quarter of overall non-attendees said that it had clashed either with work or with pre-existing personal commitments. However, amongst previous attendees, the majority of responses cited that the time or the day did not work for them this time round. > >**Recommendation** > >a # Final Thoughts - Start with a real, single research question - Pick the right tool - quant or qual - Trust the evidence, not the hypothesis $fin$