Testing Setup

Edmonds Commerce Testing Module

To make a start on the testing you should clone the testing module from bitbucket and run through the README.

Redacted

Basic Setup

On a new Magento 2 container the tests for Magento are found in the dev folder and are structured as follows

ec@clientname-magento2-desktop tree -L 1 dev/tests/
dev/tests/
|-- api-functional
|-- functional
|-- integration
|-- js
|-- static
`-- unit

To begin with you will want to focus on Unit tests, Functional Tests, and Integration tests.

Tests should live within your module, in a Test folder split up into Intergration and Unit folders.

Configuration

In order to get the tests to run, you need to configure the different frameworks. To do this go into the dev/tests/{type} folder and you should see a phpunit.xml.dist file.

Move this to phpunit.xml and make the folling changes.

<testsuites>
    <testsuite name="EdmondsCommerce Tests">
        <directory suffix="Test.php">../../../app/code/EdmondsCommerce/*/Test/Unit</directory>
    </testsuite>
</testsuites>
<filter>
    <whitelist addUncoveredFilesFromWhiteList="true">
        <directory suffix=".php">../../../app/code/EdmondsCommerce</directory>
    </whitelist>
</filter>

Obviously merge this into the existing nodes and update the directory paths to match the project you are working on.

For tests that are going to run in isolation, you need to configure the test database that will be used. The file to configure this lives in different places, but is always called install-config-mysql.php.dist

Find this file, it should be in the etc or config directory, and move it to install-config-mysql.php.

The file returns an array with various configuration options. You need to change the following

<?php
/**
 * Copyright © 2013-2017 Magento, Inc. All rights reserved.
 * See COPYING.txt for license details.
 */

return [
    'db-host' => '',                                                             /* Change this */
    'db-user' => '',                                                             /* And this */
    'db-password' => '',                                                         /* And this */
    'db-name' => '',                                                             /* And this */  
    'db-prefix' => '',                                                           /* Probably not this */
    'backend-frontname' => 'backend',                                            /* Certainly not this */
    'admin-user' => \Magento\TestFramework\Bootstrap::ADMIN_NAME,                /* And definitly none of these */
    'admin-password' => \Magento\TestFramework\Bootstrap::ADMIN_PASSWORD,
    'admin-email' => \Magento\TestFramework\Bootstrap::ADMIN_EMAIL,
    'admin-firstname' => \Magento\TestFramework\Bootstrap::ADMIN_FIRSTNAME,
    'admin-lastname' => \Magento\TestFramework\Bootstrap::ADMIN_LASTNAME,
];

Helpfully Magneto the tests will fail if the database listed does not exist, so you will need to create it.

Do not use the live database. Each time the tests are run the database listed will be dropped and recreated using install scripts and fixtures. It is not backed up, and it is not restored, so you will lose all data in it.

Fixtures

For the integration tests, you will need to populate the database with known data. Thankfully there is a simple way to do this on a per test basis, unfortunately Magento has completely buggered it up. This will explain how to fix these issues.

To load a fixture for a certain test, you need to annotate it with either @magentoDataFixture or @magentoApiDataFixture in the doc block above the test. You then follow this with the fixture file to load, i.e.

/**
* @magentoDataFixture Magento/Bundle/_files/product.php
*/

The problem with this is that it only checks the dev/test/integration folder for the file, and as our tests are living in the module this can not be used.

You can also specify a static method as the fixture path, and this will be called instead. This allows you to specify a path within the module which will then be loaded, so you test should look like this.

/**
* @magentoApiDataFixture getCatalogFixture
*/
public function testGetCatalogCall()
{
    /** Do some testing stuff */
}

public static functon getCatalogFixture()
{
    require __DIR__ . '/../_files/catalog_fixture.php';
}

public static function getCatalogFixtureRollback()
{
    require __DIR__ . '/../_files/catalog_fixture_rollback.php';
}

The Rollback method is called after the test and should be used to return the database to a know state.

This will work absolutely fine, as long as you are not going to write any API functional tests. If you are going to then you need to take a couple of extra steps.

If you are going to test the API then it is likely that your tests will extend from Magento\TestFramework\TestCase\WebapiAbstract, which only exists in the api-functional framework.

Unfortunately when the main integration framework is run, it does things with classes that have static variables. It finds these files by searching for the word static within any file that it finds that is not explicitly excluded.

For reasons that are hard to fathom, the excluded directories are hard coded to only include the Test/Unit directory and not the entire Test directory. This means that your classes with static fixture functions will be included, and as the WebapiAbstract class does not exist in the integration framework, a Fatal Error will be triggered. Take a look at dev/tests/integration/framework/Magento/TestFramework/Workaround/Cleanup/StaticProperties.php for more details

However, abstract classes will not be included in this, so you will need to create a base FixtureLoader class, and then have all of your API tests extend from that.

Cleaning up

A different problem with the API tests, are that they create a series of Test modules in the app/code directory each time they are run.

Sadly they don't then remove them after the tests are run, and the test modules will trigger fatal errors when you try to continue running Magento.

To fix this, you need to create a test listener that will run after the API tests have finished. This will delete every folder in the app/code/Magento folder that starts with the word Test and then delete the Magento folder if it is empty. You shouldn't have any modules like this in your Project, but please confirm before setting this up.

Once you have confirmed this, you need to edit the dev/tests/api-functional/phpunit.xml and add in this line

<!-- Test listeners -->
<listeners>
    <listener class="Magento\TestFramework\Event\PhpUnit"/>
    <!-- Add in this line -->
    <listener class="EdmondsCommerce\API\Test\Integration\Listener" />
</listeners>

And then create this class here app/code/EdmondsCommerce/API/Test/Integration/Listener.php

<?php

namespace EdmondsCommerce\API\Test\Integration;

/**
 * The api-functional tests, copy a load of unwanted Test Modules into the app/code/Magento folder, and then doesn't
 * remove them. These then break the other tests, setup scripts, and other bit of functionality. It's also fucking
 * retarded and should never have made it into production code
 *
 * To fix this we are going to hook into the api-functional listeners and delete all of these.
 *
 * Class Listener
 *
 * @package EdmondsCommerce\API\Test\Integration
 */
class Listener extends \PHPUnit_Framework_BaseTestListener
{
    /**
     * This is called at the "end of the test suite", which is actually after each test. However only the final call
     * has an empty name, so we will check that.
     *
     * @param \PHPUnit_Framework_TestSuite $suite
     */
    public function endTestSuite(\PHPUnit_Framework_TestSuite $suite)
    {
        $suiteName = $suite->getName();
        if ($suiteName == '') {
            $this->removeTestModules();
        }
    }

    /**
     * This lists all of the modules in the Magento folder that start with Test. As far as I can tell, this should only
     * be test modules that have been copied across, as there are no Magento core modules that start with test, at the
     * moment. Anyway all of the Magneto code should be in the vendor directory if you have set up the project
     * correctly.
     *
     * On the other hand, who knows Magento will do in the future, or how the project will be configured going
     * forwards, so please confirm the two assumptions above before blindly running this, as it will delete everything
     * it finds
     */
    private function removeTestModules()
    {
        $pathToMagentoModules = __DIR__ . '/../../../../Magento/';
        if (!is_dir($pathToMagentoModules)) {
            return;
        }
        $testModules = glob($pathToMagentoModules . 'Test*');
        if (empty($testModules)) {
            return;
        }
        foreach ($testModules as $dir) {
            $this->deleteDirectory($dir);
        }
        if (empty(glob($pathToMagentoModules . '*'))) {
            rmdir($pathToMagentoModules);
        }
    }

    /**
     * Recursively loops over a directory, deleting every file it finds until it is empty. It then deletes the directory
     *
     * @param $path
     */
    private function deleteDirectory($path)
    {
        $files = glob($path . '/*');
        foreach ($files as $file) {
            (is_dir($file)) ? $this->deleteDirectory($file) : unlink($file);
        }
        rmdir($path);

        return;
    }
}