Skip to main content
Equip has a large library of coding questions, categorized by programming language and difficulty level. You can also build your own coding questions as Custom Tests. Because coding challenges are graded automatically, a small setup mistake can silently break grading for every candidate. This article covers the most common issues and how to catch them before candidates do.

​
Common Problems

A coding challenge is far more complex to set up than a quiz. Even experienced developers need to get the boilerplate (driver and starter code), input and output handling, and edge cases exactly right. Most reported issues trace back to one of these setup mistakes:
Because grading is fully automatic, an input/output bug does not fail loudly. It quietly marks all submissions wrong, and you may only notice after candidates have completed the assessment.
This is why Equip recommends using Equip’s own coding content where possible. Each question in Equip’s library has been attempted by thousands of candidates, so it is proven to work. And unlike Equip quizzes, Equip’s coding content lets you select the specific question while building the assessment and see it alongside the candidate’s results.

​
Validating Test Cases

If you do build your own challenge, validate it thoroughly before inviting anyone. The goal is to confirm that a correct solution passes every test case in every enabled language.
1

Review the problem statement

State the input format, the output format, and any constraints explicitly. Assume the candidate has never used Equip before.
2

Check the boilerplate for every language

Each enabled language has its own driver and starter code. Verify each one, not just the language you wrote the question in.
3

Verify sample and real test cases

Work through your sample test cases and real test cases by hand. Confirm the expected output matches exactly, including spacing and line breaks.
4

Attempt the challenge in Preview

Add the test to an assessment and open it in Preview. Custom-test questions appear in the preview exactly as candidates will see them, so submit a known-correct solution in each enabled language and confirm all test cases pass.
Previews are free and unlimited for your whole team, so you can test as many times as you need without spending credits.
If you randomize coding questions, validate every variant separately. Each variant must keep at least one enabled language, and the builder will block you otherwise.

​
Getting Help

If a custom coding question misbehaves, in almost all cases the question was not set up correctly. Working through the validation steps above resolves the vast majority of issues. Please note that Equip’s support team cannot triage custom coding questions for you. Debugging one means bringing in developers, sometimes for languages our team is not familiar with, and as a small company running a self-serve platform we do not have the bandwidth for that.
There is an alternative: if the skill you want to test would be broadly useful to other companies, Equip can create the test for you, usually free of charge. Reach out via live chat; turnaround is around 5 business days for most skills. See Can’t Find Your Role or Skill? for details.