CodeZinger authoring tool provides two paths to create autograded coding problems/ assignments:

1. Standard Evaluation: With this approach, a coding problem can be setup for black box testing of student's solution by comparing ‘output generated by student’s code' vs. ‘expected output’. Student earns ‘All-or-Nothing’ score for each test case.
   
2. Advanced Evaluation: With this approach, at a time of grading, faculty’s custom evaluation code will be executed to evaluate the student’s solution. This is kind of white box testing of student’s solution by validating method’s output. This approach can be adapted to check intermediate output, multiple valid output, etc. e.g. Each method in a student’s multiple method solution can be evaluated separately and it is possible for student to earn partial credit for each test case.
   

Create Advanced Evaluation type problem

Step-1: Problem Details

1. Select evaluation method as "Advanced Evaluation"
   
2. Select Programming Language(s), Suggested Time to solve, and enter Problem Title and Problem Statement.
   
3. Select valid Tags for the problem. You can add custom tags too.
   

Step-2: Code Template

1. Select template from available starter code templates. Or, you can create custom template (and also save for later re-use too). In Advanced Evaluation, a starter template is mandatory with logic to evaluate student submission.
   
2. Some considerations while creating a starter template for this type of problem:
   
   • Main method required for languages such as C, C++, Java, etc. whereas in languages such as Python, you need to include Evaluation method. Main/Evaluation method of the template must print an integer (between 0 and 100) in a new line as last generated output. CodeZinger scoring engine will ignore output generated in any lines before the last line. This method should include logic to evaluate student's solution by comparing output of student's solution with expected output.
     
   • A student is assigned a score based on last generated output number. It is a percentage of score-weight for test case. e.g. Let's say, a method prints a final output 70 and that test case has weight 50, final score for that test case is = 50*70/100 = 35.
     
   • Your evaluation code should include logic to generate last output number and print it.
     
   • It is recommended you set the evaluation code as read-only file (or in a single file problem, have your evaluation code as read-only lines-in-a-file) to protect the logic of calculating last generated output number and printing it. (Also, in languages requiring entrypoint, please set entrypoint file as read-only to prevent students from being able to bypass your Main/Evaluation method).
     
   • You can include logic to evaluate multiple methods in student's solution and generate final score.
     

Step-3: Test Cases

1. In the single data field called Input, provide input values and expected output values. [Your Main/Evaluation method must include code to read input and expected output values entered in your test cases.]
   
2. Enter “Score Weight” for each test case.
   
3. Set "Execute in Quick Run" checkbox to have CodeZinger execute test in Quick Run. You should have at least one test case available for Quick Run.
   
4. Set "Hidden" checkbox (optional) to have CodeZinger hide test case from student but execute test case on student submission to check for brute-force solution.
   
5. Set "Add command line parameters" to pass command line parameter to Main method.
   

Step-4: Provide possible solution to validate problem

1. You are required to enter a possible solution (to simulate submission of a solution by student) to validate starter/evaluation code and test cases.
   
2. You may also specify “Language Compiler configurations” to be used at time of code execution.
   
3. “Run All Test Cases” is available to iteratively validate if your problem is functioning correctly.
   
4. If your sample solution is ‘passing’ all test cases, your coding problem may be ready to be activated.
   
5. Click “Verify & Activate” to activate your custom coding problem.
   

After activation, problem will be available to be added to any of your CodeZinger classes.