How to stop the cheating -- some thoughts for the Memrise staff

I wanted to submit a post to address the cheating that I believe is occurring on the app. Some explanation of where I’m coming from:

  • I am a language professor in the United States and use your courses in my classrooms (I give my students extra credit bonuses if they use your website). I believe you have the best product out there as far as finding a balance between gamifying learning and using self-generated visual mnemonics.
  • I have found, from my students, that it is fairly easy to create a bot to answer the 5000 word review courses automatically – for example, see the following Github page where a user coded a Memrise cheat bot: GitHub - Flying-Toast/memrise-cheat: I didn't want to do my French homework today, so I made a bot to do it for me.
  • As a more personal example – I teach Spanish and was going through the the “First 5000 words of Spanish” course. Once I finished the course, the only option to continue to review was on Speed Review mode through the desktop interface (either on my Mac or by using Safari to access the desktop interface on an iPhone; Speed Review is not available on the app after completely finishing the course). I don’t have an Android, so I am not sure what is possible with their interface, though it’s presumably the same as iOS.
  • While I was doing so, I noticed a user by the name of mariapo that I strongly suspect is cheating. The reason for my suspicion is as follows:
  1. I was completing the rounds with 100% accuracy (100 words out of 100) and as fast as humanly possible (I have been on Memrise for several years and can go extremely fast)
  2. While I was doing this, I had an opportunity to quiz side-by-side with mariapo and notice their pattern of point accumulation.
  3. It was essentially no breaks and a 100% score each time over the course of several, several hours. While this is possible for a human reader for short periods, ultimately operator fatigue, needing to go to the bathroom, eat food, sleep come into play. However, this user was doing so without any breaks. It looked exactly like a bot was turned on until this user was at the top of the leaderboard.
  4. This user has a 54 million point total for the course, a 15 million point differential between them and the 2nd place user for all time, and the 5000 word Spanish word course is essentially the only course they have ever done.
  5. Unless there is some serious OCD at play, this is a very bizarre pattern of usage for your website and looks like they simply created the account to test out their bot.
  • I do believe there is a relatively simple solution to this issue that doesn’t involve reprogramming the entire backend: there need to be audio tests built in to the desktop interface for Speed Review at periodic intervals, as this is one of the main vulnerabilities of Memrise to bot abuse.

  • Bots are unable to deal with audio prompts (yet), so the only way a user could “cheat” would be a hybridized approach where the bot answered the speed review questions and any time there was an audio break, a human user would need to be present to respond. I doubt most of the malcontents who have been abusing your site would have the patience to do this. Basically a learning version of the “capture” that security protocols have you answer to verify you’re a human user (ie. when you’re logging on to your bank account and it has you “click every chimney in this picture” etc).

  • Alternatively, you could disable speed-review mode or make it extremely point-depleted once the course is finished, similar to the app, though this would severely discourage long term users. I think it would be better to employ the audio checks and go the other way with speed review (make it accessible on the app after the 5000 word courses are finished, instead of disabling it).

  • The worst “solution” would be removing Speed Review altogether once the course is finished (as on your app) as this functionally hamstrings your learning – it is difficult to get my more advanced students to continue using the app (over competitors like Ankii) without Speed Review, as progress slows to a snail’s pace otherwise once the courses are completed and SR is not available.

  • It is very important to note that the pace of Speed Review mimics the speed of translation needed in real conversations far more closely and allows for a smoother transition to fluency. The absence of the magic wand also allows learning to be more sincere and robust.

  • What I mean to say is this: Speed Review is a remarkable tool that allows learning to remain effervescent over time; it just needs to be modified slightly to limit its abuse potential.

I really hesitated to post this as I am aware that on some level I am just showing the cheaters how to mimic humans better with their bots and avoid getting caught. But I suspect this information is already out there. As the app continues to grow, this will continue to be a significant issue that would be best to nip in the bud early.

Regardless of how you respond, I sincerely appreciate your team’s efforts. You are doing a good thing.

2 Likes