Imagine you’re playing a multiplayer FPS game like Destiny 2. You’ve linked up with your normal group on Discord for a raid. Through your headset, you can hear five voices. There’s a fair amount of strategic comms, but there’s also trash talk. “You know you’re supposed to go for headshots with a sniper, right? Haha, nice body shot! Get f*cked!” In a fast-paced FPS, you might expect some level of trash talk in chat – it’s all part of the fun, but context is important.
Now imagine the same type of trash talk, but this time you’re playing a game intended for kids and teens like Among Us VR. Does it still feel fun or appropriate based on the context of the game?
In gaming environments, there is a fine line between trash talk and toxicity. How players converse casually in one game might be considered inappropriate or full out toxic in another. The tone of a comment can change depending on demographics and familiarity of the players present in the conversation. Without the right tools like ToxMod, gaming companies will struggle to get ahead of toxicity and keep gaming environments fun. So how are we building ToxMod to be able to differentiate between trash talk in voice and likely toxicity?
What is the difference between trash talk and toxicity?
Trash talk is often one core element of building community and camaraderie – there’s nothing wrong with a little healthy competition in multiplayer gaming. But where is the line between trash talk and toxicity? Depending on the context, trash talk and toxicity can sound almost identical, making it difficult to proactively monitor more nefarious behaviors like hate speech or sexualized violence and to flag problematic players. To differentiate between the two, we first have to understand the intent and impact of the interaction itself.
With trash talk, the intent is to participate in a competitive gaming atmosphere, engage in friendly banter, and “psych out” competitors to gain an edge. The impact of trash talk can be intense at times, but it’s not intended to be harmful or demeaning. Trash talk is mutual among players and widely expected in certain gaming environments.
In contrast, there are some behaviors which harm people – things like continued harassment on mic, griefing in-game, name calling that has nothing to do with gameplay, or even doxxing – which we know as toxicity. It’s these actions that lead to a culture of exclusion, especially for marginalized groups. In the case of toxicity, the intent is to demean, dehumanize, or discriminate against other players.
Should you proactively monitor voice chat for toxicity?
In many instances trash talk and toxicity can sound very similar depending on the context. So if there’s overlap between trash talk and toxicity, should gaming companies even bother trying to tell the difference? The short answer is yes. While trash talk can be one component of building camaraderie in gaming communities, toxicity and hate speech can have a detrimental impact on the overall gaming experience, resulting in:
- Emotional distress and isolation for individual gamers.Increased stress and low self esteem for gamers.
- Hostile gaming atmosphere and deterioration of community.
- Exclusion of marginalized groups.Barriers for new players.
- Reputational damage for gaming companies and gaming as a whole.
- Legal consequences (violation of terms of service).
- Undermining the purpose of gaming: to entertain and build community.
Furthermore, 67% of multiplayer gamers say they would likely stop playing a multiplayer game if another player were exhibiting toxic behavior. That’s why it is important to not only monitor, but proactively monitor, hate speech in multiplayer games.
ToxMod is the only proactive voice chat moderation solution purpose-built for games. Using machine learning, ToxMod triages voice chat to flag bad behavior, analyzes the nuances of each conversation to determine toxicity, and enables moderators to quickly respond to each incident by supplying relevant and accurate context. Without machine learning, it would be impossible for gaming companies to effectively and efficiently moderate toxicity at the scale needed to make an impact — so how does it work?
How does ToxMod differentiate toxicity from similar-sounding trash talk?
ToxMod works by triaging, analyzing, and escalating toxicity in gaming communities. During the analysis phase, ToxMod takes an input audio stream and analyzes the following components to produce an overall toxicity score:
-Transcription: this is the most obvious factor - in order to detect toxicity, we must have some sense of what’s being said! The first layer of transcription analysis focuses on looking for phrases or slurs which are unequivocally bad, though as mentioned below we then augment this with deeper sentiment analysis
-Emotion and speech modes: recognizes emotions including anger, which can help differentiate between playful banter and genuine hurt.
- Perceived age and gender presentation: if there is a mix of perceived ages and perceived biological sexes present in a conversation, it can increase the overall risk of clip (for example, due to gender-based harassment).
- Text sentiment analysis: analyzes the full textual utterance (in combination with preceding/following comments from others in the chat) to better understand the intent and sentiment with which it was spoken.
What about keywords and phrases which are sometimes bad, but sometimes fine?
ToxMod does not simply look at keywords in a vacuum. For example, not every single instance of “f*ck” results in ToxMod raising an alarm - it depends whether the game intends to be child-friendly, whether the f-word was used in combination with more directly hateful terms, etc. This is crucial, as being too quick to raise an alarm can have serious negative impacts on players, including incorrect muting, bans, or accidental punishment of friendly trash talk.
Instead, ToxMod combines what’s being said with how it was spoken and who may have been exposed to the utterance, to truly determine whether an interaction was harmful. For instance, someone shouting the F-word might be doing so in a celebratory manner after a near escape (nearly always fine outside of extremely child-focused games); or might be taunting an adversary (sometimes ok depending on the nature of the game), or might be in the midst of a graphic tirade about how one ethnic group is superior to another (never okay). ToxMod also receives continuous feedback from the moderators who review its reports – so if ToxMod misunderstands context, it promptly learns about what it missed and is able to grow and adapt based on the needs of the studio, its code of conduct, and a game’s player base. To help moderators pinpoint the types of behavior that are most egregious in their games, ToxMod breaks its detection into several categories like sexual harassment, racial hate speech, gender hate speech, and more.
See ToxMod in action
Toxicity is one powerful reason why gamers stop playing multiplayer games. Proactively flagging toxicity is crucial for gaming companies but at such an enormous scale, it’s nearly impossible to quickly and accurately differentiate between trash talk and toxicity without a machine learning-based solution.
Learn how ToxMod can help you protect and empower your gamers and moderation teams. Book a demo.
