Documentation

Creating Simple Checks#^ TOP

In this article we will learn how to create a simple check. If you don't fully understand what checks are, head over to the Exercise Checks article.

We will build a fairly boring check which verifies that a student's solution passes the PSR2 coding standard. Lets get started!

Creating a check begins with creating a file and a class for our check . We need to implement the interface PhpSchool\PhpWorkshop\Check\SimpleCheckInterface which extends from PhpSchool\PhpWorkshop\Check\CheckInterface. Let's breakdown these methods before we start coding:

getName()

This method should just return a string which represents the name of the check. This will be printed on the terminal during the verification process. This will be PSR2 Code Check for our check.

getExerciseInterface()

This method should just return a string which is the FQCN (Fully Qualified Class Name) of the interface that the exercise needs to implement when requiring our check. Because we don't need any extra information for our check we can just use PhpSchool\PhpWorkshop\Exercise\ExerciseInterface.

canRun(ExerciseType $exerciseType)

This method receives an ExerciseType instance which represents the type of exercise, we use this to inform the workshop which exercise types our check works with: CLI, CGI or CUSTOM. We will support CLI & CGI.

check(ExerciseInterface $exercise, Input $input)

This is the method where we actually perform our check logic, executing PHP_CodeSniffer. This method receives an instance of the current exercise and the input arguments passed to our workshop, which will contain the file name of the student's solution.

This method needs to return an instance of PhpSchool\PhpWorkshop\Result\ResultInterface. Depending on whether you want a success or failure to be recorded it will be an instance of PhpSchool\PhpWorkshop\Result\SuccessInterface or PhpSchool\PhpWorkshop\Result\FailureInterface. Learn about results here.

getPosition()

This method should return one of two constants PhpSchool\PhpWorkshop\Check\SimpleCheckInterface::CHECK_BEFORE, PhpSchool\PhpWorkshop\Check\SimpleCheckInterface::CHECK_AFTER. This value indicates when the check should be run.

The process of verifying a student's solution looks something like the following (pseudo code):

//run before checks
foreach ($beforeChecks as $check) {
    $result = $check->check($exercise, $submissionFilePath);

    if (!$result->isSuccessful()) {
        return;
    }
}

//compare output of student solution and reference solution
$this->verifier->compareOutput($exercise);

foreach ($afterChecks as $check) {
    $result = $check->check($exercise, $submissionFilePath);
    //store result
}

Before Verifying

When a check uses CHECK_BEFORE mode it is run before the output verification. The process is also short circuited if a check returns a failure. No more checks will be run and the output will not be compared.

After Verifying

When a check use CHECK_AFTER mode it is run after the output verification. This means that the check is run after the student's solution has been run. After checks are useful for verifying that something was actually performed in the students submission, for example, inserting a row into the database.

Build the check#^ TOP

Now - let's build it! We will use the already built tutorial workshop as a base - the finished code is available on the custom-simple-check branch of the tutorial repository. We will start fresh from the master branch for this tutorial, so if you haven't already got it, git clone it and install the dependencies:

Our check will run the PHP_CodeSniffer tool against the student's submission and report a success or failure based on the result.

1. Require the PHP_CodeSniffer tool as a dependency

2. Create the folder and class

3. Write the class

<?php

namespace PhpSchool\SimpleMath\Check;

use PhpSchool\PhpWorkshop\Check\SimpleCheckInterface;
use PhpSchool\PhpWorkshop\Exercise\ExerciseInterface;
use PhpSchool\PhpWorkshop\Exercise\ExerciseType;
use PhpSchool\PhpWorkshop\Result\Failure;
use PhpSchool\PhpWorkshop\Result\Success;
use PhpSchool\PhpWorkshop\Input\Input;

class Psr2Check implements SimpleCheckInterface
{

    public function getName()
    {
        return 'PSR2 Code Check';
    }

    public function getExerciseInterface()
    {
        return ExerciseInterface::class;
    }

    public function canRun(ExerciseType $exerciseType)
    {
        return in_array($exerciseType->getValue(), [ExerciseType::CGI, ExerciseType::CLI]);
    }

    public function check(ExerciseInterface $exercise, Input $input)
    {
        $phpCsBinary = __DIR__ . '/../../vendor/bin/phpcs';
        $cmd = sprintf('%s %s --standard=PSR2', $phpCsBinary, $input->getArgument('program'));
        exec($cmd, $output, $exitCode);

        if ($exitCode === 0) {
            return new Success($this->getName());
        }

        return new Failure($this->getName(), 'Coding style did not conform to PSR2!');
    }

    public function getPosition()
    {
        return static::CHECK_BEFORE;
    }
}

If the phpcs binary returns a non-zero exit code - a failure occurred: probably the solution did not pass the coding standard check. So we return a failure with an error message. Otherwise a Success is returned.

As we brought in the tool via Composer, we can rest assured that the binary phpcs is available in our projects vendor directory.

4. Register the Check and add a factory

Now you need to tell the application about your new check. We need to register a factory. What's a factory? Open up app/config.php and add an entry for your check. The resulting file should look like:

<?php

use function DI\factory;
use function DI\object;
use Interop\Container\ContainerInterface;
use PhpSchool\SimpleMath\Check\Psr2Check;
use PhpSchool\SimpleMath\Exercise\Mean;
use Symfony\Component\Filesystem\Filesystem;

return [
    //Define your exercise factories here
    Mean::class => object(),

    //my checks
    Psr2Check::class => object(),
];

Note the new entry for Psr2Check::class => object(),. Finally, we need to tell the application about our check in app/bootstrap.php. After the application object is created you just call addCheck with the name of check class. Your final app/bootstrap.php file should look something like:

<?php

ini_set('display_errors', 1);
date_default_timezone_set('Europe/London');
switch (true) {
    case (file_exists(__DIR__ . '/../vendor/autoload.php')):
        // Installed standalone
        require __DIR__ . '/../vendor/autoload.php';
        break;
    case (file_exists(__DIR__ . '/../../../autoload.php')):
        // Installed as a Composer dependency
        require __DIR__ . '/../../../autoload.php';
        break;
    case (file_exists('vendor/autoload.php')):
        // As a Composer dependency, relative to CWD
        require 'vendor/autoload.php';
        break;
    default:
        throw new RuntimeException('Unable to locate Composer autoloader; please run "composer install".');
}

use PhpSchool\PhpWorkshop\Application;
use PhpSchool\SimpleMath\Check\Psr2Check;
use PhpSchool\SimpleMath\Exercise\Mean;

$app = new Application('Simple Math', __DIR__ . '/config.php');

$app->addExercise(Mean::class);
$app->addCheck(Psr2Check::class);

$art =<<<ART
  ∞ ÷ ∑ ×

 PHP SCHOOL
SIMPLE MATH
ART;

$app->setLogo($art);
$app->setFgColour('red');
$app->setBgColour('black');

return $app;

5. Require the check in an exercise

Open up the Mean Average exercise file: src/Exercise/Mean.php and add in the following method, take care to import the necessary classes (PhpSchool\PhpWorkshop\ExerciseDispatcher & PhpSchool\SimpleMath\Check\Psr2Check):

public function configure(ExerciseDispatcher $dispatcher)
{
    $dispatcher->requireCheck(Psr2Check::class);
}

Hopefully you will remember this from the previous section - we are just telling the exercise to use our custom check!

Try it out!#^ TOP

Run the workshop and select the Mean Average exercise. Verifying a solution which does not pass the PSR2 coding standard will yield the output:

And a solution which does pass will yield the output:

Custom Interface#^ TOP

When you build checks, sometimes you need extra information from the exercise to configure the check. For example, the FunctionRequirementsCheck check calls getRequiredFunctions() & getBannedFunctions() on the exercise, these methods are defined on the extra interface FunctionRequirementsExerciseCheck which the exercise must implement if it requires the FunctionRequirementsCheck check.

Maybe we want to make the standard for our check configurable - it could be PSR1, PSR2, PEAR or any of the other standards PHP_CodeSniffer supports. We will make this configuration available through the method getStandard().

1. Define our interface

We need an interface to define our required method. Let's do that first:

Now would probably be a good idea to change our check name to something a little less specific, but we'll leave that up to you, probably PhpCsCheck might be a little better. Okay, lets define our interface. We want the one method getStandard to return a string representing one of the available standards:

<?php

namespace PhpSchool\SimpleMath\ExerciseCheck;

interface Psr2ExerciseCheck
{
    /**
     * @return string
     */
    public function getStandard();
}

2. Update our check

We need to update the getExerciseInterface() method in our check to return the name of our new interface. Open up src/Check/Psr2Check.php and change the getExerciseInterface() method to match below:

public function getExerciseInterface()
{
    return Psr2ExerciseCheck::class;
}

We also need to modify our check() method to actually use this data:

public function check(ExerciseInterface $exercise, $fileName)
{
    if (!$exercise instanceof Psr2ExerciseCheck) {
        throw new \InvalidArgumentException;
    }

    $standard = $exercise->getStandard();

    if (!in_array($standard, ['PSR1', 'PSR2', 'PEAR'])) {
        throw new \InvalidArgumentException('Standard is not supported');
    }

    $phpCsBinary = __DIR__ . '/../../vendor/bin/phpcs';
    $cmd = sprintf('%s %s --standard=%s', $phpCsBinary, $input->getArgument('program'), $standard);
    exec($cmd, $output, $exitCode);

    if ($exitCode === 0) {
        return new Success($this->getName());
    }

    return new Failure($this->getName(), 'Coding style did not conform to PSR2!');
}

We've added a couple of things here - we make sure the exercise actually implements our required interface, if not we throw an exception. We check if the standard provided is in a small subset of supported standards, and finally, we pass the standard along to the phpcs command.

3. Update our exercise

Now we have to implement the new interface and methods in our exercise, for our Mean Average exercise, we will still require PSR2 as the coding standard. The final exercise should look similar to below:

<?php

namespace PhpSchool\SimpleMath\Exercise;

use PhpSchool\PhpWorkshop\Exercise\AbstractExercise;
use PhpSchool\PhpWorkshop\Exercise\CliExercise;
use PhpSchool\PhpWorkshop\Exercise\ExerciseInterface;
use PhpSchool\PhpWorkshop\Exercise\ExerciseType;
use PhpSchool\PhpWorkshop\ExerciseDispatcher;
use PhpSchool\SimpleMath\Check\Psr2Check;
use PhpSchool\SimpleMath\ExerciseCheck\Psr2ExerciseCheck;

class Mean extends AbstractExercise implements
    ExerciseInterface,
    CliExercise,
    Psr2ExerciseCheck
{

    /**
     * @return string
     */
    public function getName()
    {
        return 'Mean Average';
    }

    /**
     * @return string
     */
    public function getDescription()
    {
        return 'Simple Math';
    }

    /**
     * @return array
     */
    public function getArgs()
    {
        $numArgs = rand(0, 10);

        $args = [];
        for ($i = 0; $i < $numArgs; $i ++) {
            $args[] = rand(0, 100);
        }

        return $args;
    }

    /**
     * @return ExerciseType
     */
    public function getType()
    {
        return ExerciseType::CLI();
    }

    public function configure(ExerciseDispatcher $dispatcher)
    {
        $dispatcher->requireCheck(Psr2Check::class);
    }

    /**
     * @return string
     */
    public function getStandard()
    {
        return 'PSR2';
    }
}

You should be able to run it just the same as before we added the extra interface. You can now easily update your exercise to use a different coding standard without modifying the check.

Maybe you could try updating the check to take into account the standard when returning the result? It currently has PSR2 hardcoded in the message!

You can see the finished, working code on the custom-interface-check branch of the tutorial repository.