How Equip Protects its Questions
tl;dr: Equip has (i) lots of questions, (ii) makes them available to only verified accounts and (iii) refreshes the questions regularly
Questions "leaking" is a big challenge with online assessments. How do you ensure that candidates cannot find the questions before they attempt your assessment? Especially in a self-serve product like Equip where anyone with a business email can create an account, the challenge is even bigger. Here's how Equip addresses the challenges.
Restricting access to Questions: While Equip shows sample content for its skills, the actual questions that a candidate sees are not available without a recruiter account. No question that is asked on a real assessment can be seen unless (i) it is a recruiter account; or (ii) a candidate has been invited to take the specific assessment
Search Engines cannot index the questions: As the questions need a recruiter account for access, search engines like Google, Bing, etc cannot find these questions. So if a candidate searches, say, "Equip questions on Python", the only questions that they may find would be from sample content
Only business emails can create an account: As mentioned, only users with a business email (non gmail.com, outlook.com etc) can create a recruiter account. A typical candidate who has been invited to the assessment by a recruiter from a different team cannot create their own account to see the questions. Further, even if a recruiter account is created and we detect suspicious activity, we block the account
Large Number of Questions: As a skill assessment software, we obviously realise that our questions are the most important reason recruiters want to use us. So, we spend a lot of time and effort adding questions. We have several hundreds of quiz questions and a hundred or so coding questions. We will continue to add more questions
Refreshing Questions: We have built internal tools that track how long ago a question was added, how often it has already been seen by a candidate and so on. Once we are notified that a question is "old", we retire the question. So, newer candidates can no longer see the question.
Though Equip supports multiple test types, Quizzes remain the most popular amongst them. Because each Skill has very specific questions related to it, we are extra-careful about Quiz questions. In fact, even recruiters (our paying customers) cannot see Quiz questions while creating the assessment, or in the candidate report. We do this specifically to prevent questions from leaking.
Further, with Equip's content, the quiz questions are picked and shown randomly. So even if two candidates started the same assessment at the same time, they will likely not see any common questions, and almost certainly will not see them in the same order.
In the case of Coding Challenges like Programming, SQL and CSS, the recruiter can select the question while creating the assessment, and also see them in the candidate's assessment report. But, unlike Quiz questions, this isn't an issue because there are many, many more coding questions than Quiz questions for a given skill. This is because, once we create a question for, say, Python, the same question can also work for JavaScript, C++, etc. So, for the candidate to know which question they will be asked, they will have to know ALL the questions in our database.
Also, it is much easier to remember the answer to an MCQ than to a coding question!
General Features for all Test Types
Questions "leaking" is a big challenge with online assessments. How do you ensure that candidates cannot find the questions before they attempt your assessment? Especially in a self-serve product like Equip where anyone with a business email can create an account, the challenge is even bigger. Here's how Equip addresses the challenges.
Restricting access to Questions: While Equip shows sample content for its skills, the actual questions that a candidate sees are not available without a recruiter account. No question that is asked on a real assessment can be seen unless (i) it is a recruiter account; or (ii) a candidate has been invited to take the specific assessment
Search Engines cannot index the questions: As the questions need a recruiter account for access, search engines like Google, Bing, etc cannot find these questions. So if a candidate searches, say, "Equip questions on Python", the only questions that they may find would be from sample content
Only business emails can create an account: As mentioned, only users with a business email (non gmail.com, outlook.com etc) can create a recruiter account. A typical candidate who has been invited to the assessment by a recruiter from a different team cannot create their own account to see the questions. Further, even if a recruiter account is created and we detect suspicious activity, we block the account
Large Number of Questions: As a skill assessment software, we obviously realise that our questions are the most important reason recruiters want to use us. So, we spend a lot of time and effort adding questions. We have several hundreds of quiz questions and a hundred or so coding questions. We will continue to add more questions
Refreshing Questions: We have built internal tools that track how long ago a question was added, how often it has already been seen by a candidate and so on. Once we are notified that a question is "old", we retire the question. So, newer candidates can no longer see the question.
Specific To Quizzes
Though Equip supports multiple test types, Quizzes remain the most popular amongst them. Because each Skill has very specific questions related to it, we are extra-careful about Quiz questions. In fact, even recruiters (our paying customers) cannot see Quiz questions while creating the assessment, or in the candidate report. We do this specifically to prevent questions from leaking.
Further, with Equip's content, the quiz questions are picked and shown randomly. So even if two candidates started the same assessment at the same time, they will likely not see any common questions, and almost certainly will not see them in the same order.
Specific To Coding Challenges
In the case of Coding Challenges like Programming, SQL and CSS, the recruiter can select the question while creating the assessment, and also see them in the candidate's assessment report. But, unlike Quiz questions, this isn't an issue because there are many, many more coding questions than Quiz questions for a given skill. This is because, once we create a question for, say, Python, the same question can also work for JavaScript, C++, etc. So, for the candidate to know which question they will be asked, they will have to know ALL the questions in our database.
Also, it is much easier to remember the answer to an MCQ than to a coding question!
Updated on: 20/03/2024
Thank you!