Documentation

Creating Listener Checks#^ TOP

In the previous section, we learned of all the events dispatched throughout the process of verifying and running a student's solution to an exercise. In this this section we will learn how these events can be used to build a Listener Check.

What is a Listener Check?#^ TOP

We learned about Simple Checks in Exercise Checks, they are simple pieces of code which can run before or after verifying a student's solution to an exercise. Listener Checks allow us to hook in to the verifying and running process with more granular precision. Listener Checks can run pieces of code at any point where an event is dispatched. Check the Events page for a list of available events which your Listener Check can listen to.

Listener Checks are one of the most complex components of the workshop application, so in order to demonstrate their use-case, we will build a Listener Check which allows us to interact with Couch DB. We will then build an exercise in our tutorial application which utilises this check.

The finished Couch DB Check Exercise utilising the check

Check Specification#^ TOP

Before we build anything we should design our check. What should it do?

Couch DB is a NoSQL database, which stores data as JSON documents and it's API is provided via regular HTTP.

So, we want to introduce the features of Couch DB via this Listener Check. What should it do?

• Be applicable to only CLI type exercises.
• Create 2 databases, one for the student solution and one for the reference solution.
• Pass the databases names to the programs.
• Remove the databases at the end of the verify/run process and in case of any failures.
• Allow for exercises to seed the two databases with data.
• Allow for exercises to verify the data in the database after the solutions have executed.

What events to use?#^ TOP

Reading this specification we can see that we will need to hook in to various events to provide this functionality, we will now break down each point and decide what events to listen to.

Creating the databases

We will need to create databases in both verify & run mode, we can do this immediately in our attach method, which is automatically called when we register our check within an exercise.

Seed the database

We will need to allow the exercise to seed the database, we should do this early on verify.start & run.start are the earliest events dispatched. These sound like good candidates to perform this task. We will pass a client object to the exercise seed method so they can create documents.

Pass database name to the programs

We will need to pass the database names to the programs (student's solution & the reference solution) so the programs can access it via the $argv array. We can do this with any events which trigger with an instance of CliExecuteEvent. We can use cli.verify.reference-execute.pre, cli.verify.student-execute.pre & cli.run.student-execute.pre.

Verify the database

We will need to allow the exercise to verify the database, we should do this after output verification has finished. We can pick one of the last events triggered, verify.finish will do! We will pass the database client object again to the exercise verify method so they can verify the state of the database.

Cleanup the database

We will need to remove the databases we created at the end of the process. We can use verify.finish & run.finish to do this. We will also listen to cli.verify.reference-execute.fail so in case something goes wrong, we still cleanup.

Now let's build the check!#^ TOP

The finished Couch DB check is available as a separate Composer package for you to use in your workshops right away, but, for the sake of this tutorial we will build it using the tutorial application as a base so we do not have to setup a new project with composer files, register it with Packagist and so on.

We will start fresh from the master branch for this tutorial, so if you haven't already got it, git clone it and install the dependencies:

1. Require doctrine/couchdb as a dependency

We will use this library to interact with Couch DB.

2. Create the folders and classes

3. Define our interface

We mentioned before that we needed a way for the exercise to seed and verify the database, so we will define an interface which describes these methods which the exercise must implement for the Couch DB check. These methods will automatically be invoked by the check. Open up src/ExerciseCheck/CouchDbExerciseCheck.php and add the following code to it:

<?php

namespace PhpSchool\SimpleMath\ExerciseCheck;

use Doctrine\CouchDB\CouchDBClient;

interface CouchDbExerciseCheck
{
    /**
     * @param CouchDBClient $couchDbClient
     * @return void
     */
    public function seed(CouchDBClient $couchDbClient);

    /**
     * @param CouchDBClient $couchDbClient
     * @return bool
     */
    public function verify(CouchDBClient $couchDbClient);
}

We define, two methods seed() & verify(), both receive an instance of CouchDBClient which will be connected to the database created for the student, seed() should be called before the student's solution is run and verify() should be called after the student's solution is run.

4. Write the check

For this check, we assume that Couch DB is always running at http://localhost:5984/as is default when Couch DB is installed.

Now we write the check - there is quite a lot of code here so we will do it in steps, open up src/Check/CouchDbCheck.php and start with the following:

<?php

namespace PhpSchool\SimpleMath;

use Doctrine\CouchDB\CouchDBClient;
use Doctrine\CouchDB\HTTP\HTTPException;
use PhpSchool\PhpWorkshop\Check\ListenableCheckInterface;
use PhpSchool\PhpWorkshop\Event\EventDispatcher;
use PhpSchool\SimpleMath\ExerciseCheck\CouchDbExerciseCheck;

class CouchDbCheck implements ListenableCheckInterface
{
    /**
     * @var string
     */
    private static $studentDb = 'phpschool-student';

    /**
     * @var string
     */
    private static $solutionDb = 'phpschool';

    /**
     * Return the check's name
     *
     * @return string
     */
    public function getName()
    {
        return 'Couch DB Verification Check';
    }

    /**
     * This returns the interface the exercise should implement
     * when requiring this check
     *
     * @return string
     */
    public function getExerciseInterface()
    {
        return CouchDbExerciseCheck::class;
    }

    /**
     * @param EventDispatcher $eventDispatcher
     */
    public function attach(EventDispatcher $eventDispatcher)
    {

    }
}

There is not much going on here - we define getName() which is the name of our check, and getExerciseInterface() which should return the FQCN (Fully Qualified Class Name) of the interface we just defined earlier. This is so the workshop framework can check the exercise implements it. We also define some properties which describe the names of the Couch DB databases we will setup: one for the student and one for the reference solution.

The most important thing to note in this check is that we implement PhpSchool\PhpWorkshop\Check\ListenableCheckInterface instead of PhpSchool\PhpWorkshop\Check\SimpleCheckInterface. They both inherit from PhpSchool\PhpWorkshop\Check\CheckInterface which introduces getName() & getExerciseInterface(). ListenableCheckInterface brings in one other additional method: attach(). This method is called immediately when an exercise requires any Listener Check and is passed an instance of PhpSchool\PhpWorkshop\Event\EventDispatcher allowing the check to listen to any events which might be dispatched throughout the verifying/running process.

Our check will listen to a number of those events so we will build this method up step by step.

Create the databases

The first thing we need to do is create the two databases, so we create two Couch DB clients and issue the createDatabase method:

$studentClient = CouchDBClient::create(['dbname' => static::$studentDb);
$solutionClient = CouchDBClient::create(['dbname' => static::$solutionDb]);

$studentClient->createDatabase($studentClient->getDatabase());
$solutionClient->createDatabase($solutionClient->getDatabase());

Seed the databases for verify mode

We need to allow the exercise to seed the database to create documents, for example. The database for the student and the reference solution should contain the same data, but they must be different databases.

The reason why both programs need their own database is fairly simple. Say the exercise's lesson was to teach how to remove a document in the database. It would first need to create a document in the database using the seed method. The student's solution should remove that document. If the student's solution and the reference solution shared one database, then the reference solution would run first and remove the row. Then the student's solution would run...it can't remove the document because it's not there anymore!

We can't just call seed() again because seed() can return dynamic data and then the student's solution and the reference solution would run with different data sets; which makes it impossible to compare their output.

$eventDispatcher->listen('verify.start', function (Event $e) use ($studentClient, $solutionClient) {
    $e->getParameter('exercise')->seed($studentClient);
    $this->replicateDbFromStudentToSolution($studentClient, $solutionClient);
});

We listen to the verify.start event which (as you can probably infer) triggers right at the start of the verify process. The listener is an anonymous function that grabs the exercise instance from the event and calls the seed() method passing in the CouchDBClient which references the database created for the student. We also need to seed the database for reference solution, we need it to be exactly the same as the student's so we basically select all documents from the student database and insert them in to the reference solution database. We do this in the method replicateDbFromStudentToSolution. This method looks like the following:

/**
 * @param CouchDBClient $studentClient
 * @param CouchDBClient $solutionClient
 * @throws \Doctrine\CouchDB\HTTP\HTTPException
 */
private function replicateDbFromStudentToSolution(CouchDBClient $studentClient, CouchDBClient $solutionClient)
{
    $response = $studentClient->allDocs();

    if ($response->status !== 200) {
        return;
    }

    foreach ($response->body['rows'] as $row) {
        $doc = $row['doc'];

        $data = array_filter($doc, function ($key) {
            return !in_array($key, ['_id', '_rev']);
        }, ARRAY_FILTER_USE_KEY);

        try {
            $solutionClient->putDocument(
                $data,
                $doc['_id']
            );
        } catch (HTTPException $e) {
        }
    }
}

Seed the database for run mode

When in run mode, no output is compared - we merely run the student's solution - so we only need to seed the student's database. There is a similar event to verify.start when in run mode, aptly named run.start, let's use that:

$eventDispatcher->listen('run.start', function (Event $e) use ($studentClient) {
    $e->getParameter('exercise')->seed($studentClient);
});

Adding the database name to the programs' arguments

We need the programs (student solution & the reference solution) to have access to their respective database names, the best way to do this is via command line arguments - we can add arguments to the list of arguments to be sent to the programs with any event which triggers with an instance of CliExecuteEvent. It exposes the prependArg() & appendArg() methods.

We use cli.verify.reference-execute.pre to prepend the reference database name to the reference solution program when in verify mode and we use cli.verify.student-execute.pre & cli.run.student-execute.pre to prepend the student database name to the student solution in verify & run mode, respectively.

$eventDispatcher->listen('cli.verify.reference-execute.pre', function (CliExecuteEvent $e) {
    $e->prependArg('phpschool');
});

$eventDispatcher->listen(
    ['cli.verify.student-execute.pre', 'cli.run.student-execute.pre'],
    function (CliExecuteEvent $e) {
        $e->prependArg('phpschool-student');
    }
);

Verify the database

After the programs have been executed, we need a way to let the exercise verify the contents of the database. We hook on to an event during the verify process named verify.finish (this is the last event in the verify process) and insert a verifier function. We don't need to verify the database in run mode because all we do in run mode is run the students submission in the correct environment (with args and database).

$eventDispatcher->insertVerifier('verify.finish', function (Event $e) use ($studentClient) {
    $verifyResult = $e->getParameter('exercise')->verify($studentClient);

    if (false === $verifyResult) {
        return Failure::fromNameAndReason($this->getName(), 'Database verification failed');
    }

    return Success::fromCheck($this);
});

Verify functions are used to inject results into the result set, which is then reported to the student. So you can see that if the verify method returns true we return a Success to the result set but if it returns false we return a Failure result, with a message, so the student knows what went wrong.

The Event Dispatcher takes care of running the verifier function at the correct event and injects the returned result in to the result set.

Cleanup the databases

The final stage is to remove the databases, we listen to verify.post.execute for the verify process & run.finish for the run process:

$eventDispatcher->listen(
    [
        'verify.post.execute',
        'run.finish'
    ],
    function (Event $e) use ($studentClient, $solutionClient) {
        $studentClient->deleteDatabase(static::$studentDb);
        $solutionClient->deleteDatabase(static::$solutionDb);
    }
);

Great - our check is finished! You can see the final result as a separate Composer package, available here.

Our final check should look like:

<?php

namespace PhpSchool\SimpleMath;

use Doctrine\CouchDB\CouchDBClient;
use Doctrine\CouchDB\HTTP\HTTPException;
use PhpSchool\PhpWorkshop\Check\ListenableCheckInterface;
use PhpSchool\PhpWorkshop\Event\EventDispatcher;
use PhpSchool\SimpleMath\ExerciseCheck\CouchDbExerciseCheck;

class CouchDbCheck implements ListenableCheckInterface
{
    /**
     * @var string
     */
    private static $studentDb = 'phpschool-student';

    /**
     * @var string
     */
    private static $solutionDb = 'phpschool';

    /**
     * Return the check's name
     *
     * @return string
     */
    public function getName()
    {
        return 'Couch DB Verification Check';
    }

    /**
     * This returns the interface the exercise should implement
     * when requiring this check
     *
     * @return string
     */
    public function getExerciseInterface()
    {
        return CouchDbExerciseCheck::class;
    }

    /**
     * @param EventDispatcher $eventDispatcher
     */
    public function attach(EventDispatcher $eventDispatcher)
    {
        $studentClient = CouchDBClient::create(['dbname' => static::$studentDb);
        $solutionClient = CouchDBClient::create(['dbname' => static::$solutionDb]);

        $studentClient->createDatabase($studentClient->getDatabase());
        $solutionClient->createDatabase($solutionClient->getDatabase());

        $eventDispatcher->listen('verify.start', function (Event $e) use ($studentClient, $solutionClient) {
            $e->getParameter('exercise')->seed($studentClient);
            $this->replicateDbFromStudentToSolution($studentClient, $solutionClient);
        });

        $eventDispatcher->listen('run.start', function (Event $e) use ($studentClient) {
            $e->getParameter('exercise')->seed($studentClient);
        });

        $eventDispatcher->listen('cli.verify.reference-execute.pre', function (CliExecuteEvent $e) {
            $e->prependArg('phpschool');
        });

        $eventDispatcher->listen(
            ['cli.verify.student-execute.pre', 'cli.run.student-execute.pre'],
            function (CliExecuteEvent $e) {
                $e->prependArg('phpschool-student');
            }
        );

        $eventDispatcher->listen(
            [
                'verify.post.execute',
                'run.finish'
            ],
            function (Event $e) use ($studentClient, $solutionClient) {
                $studentClient->deleteDatabase(static::$studentDb);
                $solutionClient->deleteDatabase(static::$solutionDb);
            }
        );
    }

    /**
     * @param CouchDBClient $studentClient
     * @param CouchDBClient $solutionClient
     * @throws \Doctrine\CouchDB\HTTP\HTTPException
     */
    private function replicateDbFromStudentToSolution(CouchDBClient $studentClient, CouchDBClient $solutionClient)
    {
        $response = $studentClient->allDocs();

        if ($response->status !== 200) {
            return;
        }

        foreach ($response->body['rows'] as $row) {
            $doc = $row['doc'];

            $data = array_filter($doc, function ($key) {
                return !in_array($key, ['_id', '_rev']);
            }, ARRAY_FILTER_USE_KEY);

            try {
                $solutionClient->putDocument(
                    $data,
                    $doc['_id']
                );
            } catch (HTTPException $e) {
            }
        }
    }
}

Build an exercise using the Couch DB check#^ TOP

So then, this Couch DB check is not much use if we don't utilise it! let's build an exercise which retrieves a document from a database, sums a bunch of numbers and adds the total to the document, finally we should output the total. The document with the numbers in it will be automatically created by our exercise in the seed() method and will be random.

As always we will start from a fresh copy of the tutorial application:

We will use the check that is available in the already built Composer package, so, pull it in to your project:

We have to manually require doctrine/couchdb even though it is a dependency of php-school/couch-db-check because there is no stable release available. Indirect dependencies cannot install non-stable versions.

Problem file#^ TOP

Create a problem file in exercises/couch-db-exercise/problem/problem.md. Here we describe the problem we mentioned earlier when we decided what we wanted our exercise to do:

Write a program that accepts the name of database and a Couch DB document ID. You should load this document using the
provided ID from the provided database. In the document will be a key named `numbers`. You should add them all up
and add the total to the document under the key `total`. You should save the document and finally output the total to
the console.

You must have Couch DB installed before you run this exercise, you can get it here:
  [http://couchdb.apache.org/#download]()

----------------------------------------------------------------------
## HINTS

You could use a third party library to communicate with the Couch DB instance, see this doctrine library:
  [https://github.com/doctrine/couchdb-client]()

Or you could interact with it using a HTTP client such as Guzzle:
  [https://github.com/guzzle/guzzle]()

Or you could simply use `curl`.

Check out how to interact with Couch DB documents here:
  [http://docs.couchdb.org/en/1.6.1/intro/api.html#documents]()

You will need to do this via PHP.

You specifically need the `GET` and `PUT` methods, or if you are using a library abstraction, you will need to
`find` and `update` the document.


You can use the doctrine library like so:

```php
<?php
require_once __DIR__ . '/vendor/autoload.php';

use Doctrine\CouchDB\CouchDBClient;
$client = CouchDBClient::create(['dbname' => $dbName]);

//get doc
$doc = $client->findDocument($docId);

//update doc
$client->putDocument($updatedDoc, $docId, $docRevision);
```

`{appname}` will be supplying arguments to your program when you run `{appname} verify program.php` so you don't need to supply them yourself. To test your program without verifying it, you can invoke it with `{appname} run program.php`. When you use `run`, you are invoking the test environment that `{appname}` sets up for each exercise.

----------------------------------------------------------------------

We note that the student must have Couch DB installed, we give a few links, an example of how to use the Doctrine Couch DB client and we describe the actual task.

Write the exercise#^ TOP

Create the exercise in src/Exercise/CouchDbExercise.php:

<?php

namespace PhpSchool\SimpleMath\Exercise;

use Doctrine\CouchDB\CouchDBClient;
use PhpSchool\CouchDb\CouchDbCheck;
use PhpSchool\CouchDb\CouchDbExerciseCheck;
use PhpSchool\PhpWorkshop\Exercise\AbstractExercise;
use PhpSchool\PhpWorkshop\Exercise\CliExercise;
use PhpSchool\PhpWorkshop\Exercise\ExerciseInterface;
use PhpSchool\PhpWorkshop\Exercise\ExerciseType;
use PhpSchool\PhpWorkshop\ExerciseDispatcher;

class CouchDbExercise extends AbstractExercise implements
    ExerciseInterface,
    CliExercise,
    CouchDbExerciseCheck
{
    /**
     * @var string
     */
    private $docId;

    /**
     * @var int
     */
    private $total;

    /**
     * @return string
     */
    public function getName()
    {
        return 'Couch DB Exercise';
    }

     /**
     * @return string
     */
    public function getDescription()
    {
        return 'Intro to Couch DB';
    }

    /**
     * @return string[]
     */
    public function getArgs()
    {
        return [$this->docId];
    }

    /**
     * @return ExerciseType
     */
    public function getType()
    {
        return ExerciseType::CLI();
    }

    /**
     * @param ExerciseDispatcher $dispatcher
     */
    public function configure(ExerciseDispatcher $dispatcher)
    {
        $dispatcher->requireCheck(CouchDbCheck::class);
    }

    /**
     * @param CouchDBClient $couchDbClient
     * @return void
     */
    public function seed(CouchDBClient $couchDbClient)
    {
        $numArgs = rand(4, 20);
        $args = [];
        for ($i = 0; $i < $numArgs; $i ++) {
            $args[] = rand(1, 100);
        }

        list($id) = $couchDbClient->postDocument(['numbers' => $args]);

        $this->docId = $id;
        $this->total = array_sum($args);
    }

    /**
     * @param CouchDBClient $couchDbClient
     * @return bool
     */
    public function verify(CouchDBClient $couchDbClient)
    {
        $total = $couchDbClient->findDocument($this->docId);

        return isset($total->body['total']) && $total->body['total'] == $this->total;
    }
}

So - in seed we create a random number of random numbers and insert a document containing these numbers under a key named numbers. We store the total (for verification purposes) and also the document ID (this is auto generated by Couch DB) so we can pass it to the solutions as an argument.

In the verify method, we try load the document with the stored ID, check for the presence of the total property and check that it is equal to the stored total we set during seed.

In configure() we require our Couch DB check and in getType() we inform the the workshop framework that this is a CLI type exercise.

In getArgs() we return the Document ID we set during seed.

Because seed is invoked from an event which is dispatched before getArgs, we can rely on anything set there.

The students solution would therefore be invoked like: php my-solution.php phpschool-student 18. The argument phpschool-student being the database name created for the student by the check (remember the check prepends this argument to the argument list) and 18 being the ID of the document we created!

Write the reference solution#^ TOP

Our reference solution will also use the Doctrine Couch DB library - let's go ahead and create the solution in exercises/couch-db-exercise/solution. We will need three files composer.json, composer.lock and solution.php:

solution.php

<?php
require_once __DIR__ . '/vendor/autoload.php';

use Doctrine\CouchDB\CouchDBClient;

$client = CouchDBClient::create(['dbname' => $argv[1]]);
$doc = $client->findDocument($argv[2])->body;

$total = array_sum($doc['numbers']);
$doc['total'] = $total;
$client->putDocument(['total' => $total, 'numbers' => $doc['numbers']], $argv[2], $doc['_rev']);
echo $total;

composer.json

{
    "name": "php-school/couch-db-exercise-ref-solution",
    "description": "Intro to Couch DB",
    "require": {
        "doctrine/couchdb": "^1.0@beta"
    }
}

composer.lock

composer.lock is auto generated by Composer, by running composer install in exercises/couch-db-exercise/solution

Wire it all together#^ TOP

Now we have to add the factories for our check and exercise and register it with the application, add the following to app/config.php and don't forget to import the necessary classes.

CouchDbExercise::class => object(),
CouchDbCheck::class => object(),

The result should look like:

<?php

use function DI\factory;
use function DI\object;
use Interop\Container\ContainerInterface;
use PhpSchool\SimpleMath\Exercise\GetExercise;
use PhpSchool\CouchDb\CouchDbCheck;
use PhpSchool\SimpleMath\Exercise\CouchDbExercise;
use PhpSchool\SimpleMath\Exercise\Mean;
use PhpSchool\SimpleMath\Exercise\PostExercise;
use PhpSchool\SimpleMath\MyFileSystem;

return [
    //Define your exercise factories here
    Mean::class => factory(function (ContainerInterface $c) {
        return new Mean($c->get(\Symfony\Component\Filesystem\Filesystem::class));
    }),

    CouchDbExercise::class => object(),
    CouchDbCheck::class => object(),
];

Finally we need to tell the application about our new check and exercise in app/bootstrap.php. After the application object is created you just call addCheck & addExercise with the name of check class and exercise class respectively. Your final app/bootstrap.php file should look something like:

<?php

ini_set('display_errors', 1);
date_default_timezone_set('Europe/London');
switch (true) {
    case (file_exists(__DIR__ . '/../vendor/autoload.php')):
        // Installed standalone
        require __DIR__ . '/../vendor/autoload.php';
        break;
    case (file_exists(__DIR__ . '/../../../autoload.php')):
        // Installed as a Composer dependency
        require __DIR__ . '/../../../autoload.php';
        break;
    case (file_exists('vendor/autoload.php')):
        // As a Composer dependency, relative to CWD
        require 'vendor/autoload.php';
        break;
    default:
        throw new RuntimeException('Unable to locate Composer autoloader; please run "composer install".');
}

use PhpSchool\CouchDb\CouchDbCheck;
use PhpSchool\PhpWorkshop\Application;
use PhpSchool\SimpleMath\Exercise\CouchDbExercise;
use PhpSchool\SimpleMath\Exercise\Mean;

$app = new Application('Simple Math', __DIR__ . '/config.php');

$app->addExercise(Mean::class);
$app->addExercise(CouchDbExercise::class);
$app->addCheck(CouchDbCheck::class);

$art = <<<ART
  ∞ ÷ ∑ ×

 PHP SCHOOL
SIMPLE MATH
ART;

$app->setLogo($art);
$app->setFgColour('red');
$app->setBgColour('black');

return $app;

Our exercise is complete - let's try it out!

Try it out!#^ TOP

Make sure you have Couch DB installed, run the workshop and select the Couch DB Exercise exercise.

Try verifying with the solution below which incorrectly sets the total to 30, hopefully you will see a failure.

<?php
require_once __DIR__ . '/vendor/autoload.php';

use Doctrine\CouchDB\CouchDBClient;

$client = CouchDBClient::create(['dbname' => $argv[1]]);
$doc = $client->findDocument($argv[2])->body;

$total = 30; //we guess total is 30
$doc['total'] = $total;
$client->putDocument(['total' => $total, 'numbers' => $doc['numbers']], $argv[2], $doc['_rev']);
echo $total;

And a solution which does pass will yield the output: