QUnit Parameterize plugin

I got my hands on QUnit tool for automatic unit testing and found that some of the plugins for it were obsolete. One example is QUnit Parameterize plugin, it was made for QUnit below 2.x versions. I reworked it using Aptana Studio with JSlint enabled and created my own solution: Parameterize Pull request
I also created SO discussion page which I also answered once I had solution here: http://stackoverflow.com/questions/39017452/cannot-use-parametrize-plugin-for-qunit/39103346#39103346

This is how a QUnit Parameterize test should look like, utilizing QUnit.module(moduleName, hooks) function where hooks object contains before(), beforeEach(), afterEach() and after() functions to cover test atomicity:

qunit.module("addColumns(innerPartsSlotMatrix, col, colShift, ceilOn, floorOn)", { before: function () { }, beforeEach: function () { var emptySlotObject = [], HBF_Slot = [], emptyMatrix_rows2_cols2 = [], emptyMatrix_rows2_cols3 = [], emptyMatrix_rows2_cols4 = [], emptyMatrix_rows3_cols2 = [], emptyMatrix_rows2_cols9 = [], emptyMatrix_FloorOn_rows2_cols9 = [], emptyMatrix_CeilOn_FloorOn_rows2_cols9 = [], emptyMatrix_CeilOn_rows2_cols9 = [], emptyMatrix_rows3_cols3 = [], emptyMatrix_FloorOn_rows3_cols3 = [], emptyMatrix_CeilOn_FloorOn_rows3_cols3 = [], emptyMatrix_CeilOn_rows3_cols3 = [], emptyMatrix_rows3_cols6 = [], emptyMatrix_FloorOn_rows3_cols6 = [], emptyMatrix_CeilOn_rows3_cols6 = [], emptyMatrix_CeilOn_FloorOn_rows3_cols6 = []; this.emptySlotObject = [ frnConstAppKeys.EMPTY_SLOT_KEY, frnConstAppKeys.EMPTY_SLOT_KEY, frnConstAppKeys.EMPTY_SLOT_KEY, frnConstAppKeys.EMPTY_SLOT_KEY]; emptySlotObject = this.emptySlotObject; this.HBF_Slot = [ frnConstAppKeys.HORZ_BAFFLE_KEY, frnConstAppKeys.EMPTY_SLOT_KEY, frnConstAppKeys.EMPTY_SLOT_KEY, frnConstAppKeys.EMPTY_SLOT_KEY]; HBF_Slot = this.HBF_Slot; this.emptyMatrix_rows2_cols2 = [ [emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject] ]; emptyMatrix_rows2_cols2 = this.emptyMatrix_rows2_cols2; this.emptyMatrix_rows3_cols2 = [ [emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject] ]; emptyMatrix_rows3_cols2 = this.emptyMatrix_rows3_cols2; this.emptyMatrix_rows3_cols3 = [ [emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject] ]; emptyMatrix_rows3_cols3 = this.emptyMatrix_rows3_cols3; this.emptyMatrix_FloorOn_rows3_cols3 = [ [emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, HBF_Slot] ]; emptyMatrix_FloorOn_rows3_cols3 = this.emptyMatrix_FloorOn_rows3_cols3; this.emptyMatrix_CeilOn_FloorOn_rows3_cols3 = [ [emptySlotObject, emptySlotObject, HBF_Slot], [emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, HBF_Slot] ]; emptyMatrix_CeilOn_FloorOn_rows3_cols3 = this.emptyMatrix_CeilOn_FloorOn_rows3_cols3; this.emptyMatrix_CeilOn_rows3_cols3 = [ [emptySlotObject, emptySlotObject, HBF_Slot], [emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject] ]; emptyMatrix_CeilOn_rows3_cols3 = this.emptyMatrix_CeilOn_rows3_cols3; this.emptyMatrix_rows3_cols6 = [ [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject] ]; emptyMatrix_rows3_cols6 = this.emptyMatrix_rows3_cols6; this.emptyMatrix_FloorOn_rows3_cols6 = [ [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot] ]; emptyMatrix_FloorOn_rows3_cols6 = this.emptyMatrix_FloorOn_rows3_cols6; this.emptyMatrix_CeilOn_rows3_cols6 = [ [emptySlotObject, emptySlotObject, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot], [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject] ]; emptyMatrix_CeilOn_rows3_cols6 = this.emptyMatrix_CeilOn_rows3_cols6; this.emptyMatrix_CeilOn_FloorOn_rows3_cols6 = [ [emptySlotObject, emptySlotObject, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot], [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot] ]; emptyMatrix_CeilOn_FloorOn_rows3_cols6 = this.emptyMatrix_CeilOn_FloorOn_rows3_cols6; this.emptyMatrix_rows2_cols3 = [ [emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject] ]; emptyMatrix_rows2_cols3 = this.emptyMatrix_rows2_cols3; this.emptyMatrix_rows2_cols4 = [ [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject] ]; emptyMatrix_rows2_cols4 = this.emptyMatrix_rows2_cols4; this.emptyMatrix_rows2_cols9 = [ [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject] ]; emptyMatrix_rows2_cols9 = this.emptyMatrix_rows2_cols9; this.emptyMatrix_FloorOn_rows2_cols9 = [ [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject], [emptySlotObject, emptySlotObject, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot] ]; emptyMatrix_FloorOn_rows2_cols9 = this.emptyMatrix_FloorOn_rows2_cols9; this.emptyMatrix_CeilOn_rows2_cols9 = [ [emptySlotObject, emptySlotObject, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot], [emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject, emptySlotObject] ]; emptyMatrix_CeilOn_rows2_cols9 = this.emptyMatrix_CeilOn_rows2_cols9; this.emptyMatrix_CeilOn_FloorOn_rows2_cols9 = [ [emptySlotObject, emptySlotObject, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot], [emptySlotObject, emptySlotObject, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot, HBF_Slot] ]; emptyMatrix_CeilOn_FloorOn_rows2_cols9 = this.emptyMatrix_CeilOn_FloorOn_rows2_cols9; }, afterEach: function () { this.emptySlotObject = null; this.HBF_Slot = null; this.emptyMatrix_rows2_cols2 = null; this.emptyMatrix_rows2_cols3 = null; this.emptyMatrix_rows2_cols4 = null; this.emptyMatrix_rows3_cols2 = null; this.emptyMatrix_rows2_cols9 = null; this.emptyMatrix_FloorOn_rows2_cols9 = null; this.emptyMatrix_CeilOn_rows2_cols9 = null; this.emptyMatrix_CeilOn_FloorOn_rows2_cols9 = null; this.emptyMatrix_rows3_cols3 = null; this.emptyMatrix_FloorOn_rows3_cols3 = null; this.emptyMatrix_CeilOn_rows3_cols3 = null; this.emptyMatrix_CeilOn_FloorOn_rows3_cols3 = null; this.emptyMatrix_rows3_cols6 = null; this.emptyMatrix_FloorOn_rows3_cols6 = null; this.emptyMatrix_CeilOn_rows3_cols6 = null; this.emptyMatrix_CeilOn_FloorOn_rows3_cols6 = null; } }); qunit .cases .init([ {t: 1, numCols: 1, ceilOn: false, floorOn: false, matrixStubName: 'emptyMatrix_rows2_cols2', expected: 'emptyMatrix_rows2_cols3'}, {t: 2, numCols: 2, ceilOn: false, floorOn: false, matrixStubName: 'emptyMatrix_rows2_cols2', expected: 'emptyMatrix_rows2_cols4'}, {t: 3, numCols: 7, ceilOn: false, floorOn: false, matrixStubName: 'emptyMatrix_rows2_cols2', expected: 'emptyMatrix_rows2_cols9'}, {t: 4, numCols: 7, ceilOn: false, floorOn: true, matrixStubName: 'emptyMatrix_rows2_cols2', expected: 'emptyMatrix_FloorOn_rows2_cols9'}, {t: 5, numCols: 7, ceilOn: true, floorOn: true, matrixStubName: 'emptyMatrix_rows2_cols2', expected: 'emptyMatrix_CeilOn_FloorOn_rows2_cols9'}, {t: 6, numCols: 7, ceilOn: true, floorOn: false, matrixStubName: 'emptyMatrix_rows2_cols2', expected: 'emptyMatrix_CeilOn_rows2_cols9'}, {t: 7, numCols: 1, ceilOn: false, floorOn: false, matrixStubName: 'emptyMatrix_rows3_cols2', expected: 'emptyMatrix_rows3_cols3'}, {t: 8, numCols: 1, ceilOn: false, floorOn: true, matrixStubName: 'emptyMatrix_rows3_cols2', expected: 'emptyMatrix_FloorOn_rows3_cols3'}, {t: 9, numCols: 1, ceilOn: true, floorOn: true, matrixStubName: 'emptyMatrix_rows3_cols2', expected: 'emptyMatrix_CeilOn_FloorOn_rows3_cols3'}, {t: 10, numCols: 1, ceilOn: true, floorOn: false, matrixStubName: 'emptyMatrix_rows3_cols2', expected: 'emptyMatrix_CeilOn_rows3_cols3'}, {t: 11, numCols: 4, ceilOn: false, floorOn: false, matrixStubName: 'emptyMatrix_rows3_cols2', expected: 'emptyMatrix_rows3_cols6'}, {t: 12, numCols: 4, ceilOn: false, floorOn: true, matrixStubName: 'emptyMatrix_rows3_cols2', expected: 'emptyMatrix_FloorOn_rows3_cols6'}, {t: 13, numCols: 4, ceilOn: true, floorOn: true, matrixStubName: 'emptyMatrix_rows3_cols2', expected: 'emptyMatrix_CeilOn_FloorOn_rows3_cols6'}, {t: 14, numCols: 4, ceilOn: true, floorOn: false, matrixStubName: 'emptyMatrix_rows3_cols2', expected: 'emptyMatrix_CeilOn_rows3_cols6'} ]) .test("", function (params, assert) { var matrixStub = this[params.matrixStubName], expected = this[params.expected], calculated = ismUtils.addColumns(matrixStub, params.numCols, params.ceilOn, params.floorOn); assert.deepEqual(calculated, expected, "Test: " + params.t + ", expected: " + JSON.stringify(expected) + ", calculated: " + JSON.stringify(calculated)); }); 

If you scroll at the middle part after the module() function you will see a statement qunit.cases.init([/* array of tests parameters */]).test("test name", function (params, assert) {/* your test code goes here */});

"cases" subobject of QUnit is created by Parameterize plugin and I made it to use init() function to set the array of parametric tests.

Params variable sent to the test function contains the parameters sent with each object of the array inside init([/* your test parameters here */]).

You can also add expected number of assertions in your test, and QUnit will check if the asked number of assertions are run. The test will be failure if not.

Also another useful feature, if you see, the matrices defined inside beforeEach() are added to the "this" object. This means the matrices are available inside the test callback function this "this" keyword, as I use it: this[params.matrixStubName]

Another useful thing I do is, I added the "t" parameter for each test containing the order number of the test, so when a test fails you can easiliy search it in your code.

Parameterize contains also sequential and combinatorial parametric tests. I however didn't have a need for those. Combinatorial can create combinations of parametric tests from two sets, and as for sequential, I think my test here in the example is already sequential test.

You may be asking how can I automate build process using  QUnit with Jenkins CI?

Well I used Karma JS test runner and it has plugins for QUnit and for Jenkins CI.
I was working on a project and created two configuration files for Karma JS, one for Jenkins and one for local test execution.

karma.conf.local.js
karma.conf.js

If you have a problem with a Jenkins build in hanging state but everything else seems good then you need to set "singleRun" to true in karma.conf.js. This will execute the tests once and it will stop the karma test runner. If it is set to false the build will hang forever :)))

You should configure Jenkins with Karma as in this guide.

Jenkins Build

Jenkins source code management

Build environment

No need to tell Jenkins to upload the node_modules folder, just install Node JS plugin for Jenkins and then in your build add the command "npm install". This will automaticall install all Node JS dependencies.

Also you may want the "karma start" shell command to trigger running of your unit tests.

Jenkins shell commands

Karma will export the tests using the JUnit reporter plugin and a post build action is required to export the tests in xml format. I had folder called test-reports and set post build action to "test-reports/**/*.xml"

QUnit Test reports in Jenkins

"Jenkins: The Definitive Guide" - Book Review

In this post I will express my thoughts on a book written for a specific but very useful tool called Jenkins CI. The book is called "Jenkins: The Definitive Guide"  written by John Ferguson Smart.
So what is Jenkins CI (Continuous Integration)? Jenkins is a tool created by Kohsuke Kawaguchi
and its main purpose is as stated to create environment for continuous integration of your software.

Jenkins is a Java based tool, a web aplication to be more specific. Runs on various operating systems and can be installed in various ways, from a system service to running it into Winstone , a servlet container tool. You can use Jenkins to automate building your software, generate documentation for it, run unit/integration/acceptance tests, publish test reports in HTML/PDF format and even upload files to remote server.
While building, errors can happen so good notification support in Jenkins is here to help you get notified when that happens. There is support for email/SMS/RSS/IRC/Instant Messaging notifications. Even special devices that blink red when build fails and green when succedes.
Jenkins has good support for build automation tools like Maven and Ant.
It has embeded support for SVN and support for Git/Mercurial and others trough plugins.
On the other hand, you can test network applications using distributed builds with JMeter and try various test environments using Multi-configuration builds.
You can run parametrized builds as well run code coverage and code quality tests.
There are ton of plugins built for specific cases and usage which you can easily download and use in your builds. You can also write your own Jenkins plugin.

Now something about the book.

 From the book preface:
"This book is aimed at relatively technical readers, though no prior experience with Continuous Integration is assumed. You may be new to Continuous Integration, and would like to learn about the benefits it can bring to your development team. Or, you might be using Jenkins or Hudson already, and want to discover how you can take your Continuous Integration infrastructure further.
Much of this book discusses Jenkins in the context of Java or JVM-related projects. Nevertheless, even if you are using another technology stack, this book should give you a good grounding in Continuous Integration with Jenkins. We discuss how to build projects using several non-Java technologies, including as Grails, Ruby on Rails and .NET. In addition, many topics, such as general configuration, notification, distributed builds and security are applicable no matter what language you are using."

Written in july/2011, in around 400 pages you will get hands on various parts of this powerful tool. My advice is to try and install Jenkins on your system and install or try things mentioned in the book. Various scenarious require various hardware and software so you will not have all the possibilites to try all of them. But at least you can download the plugins and check them shortly what they do. Some plugins are obsolete but I made a list of plugins I installed as test on my computer.
All in all the book gives a good grasp of all that might need you in your future project. Apart from some typos the book is a great read, and you should definitely spend some time with it. I give 4.5/5.It will be useful for year to come.

More about the chapters.

1. Chapter 1: Introducing Jenkins. A short introduction what is and what it is used for. A short history how Jenkins came from Hudson. How to install and what is CI(Continuous Integration).
2. Chapter 2:  Your first steps with Jenkins. Covers how to make working environment for Jenkins, installing JDK, Git, Github and Maven. Your first build job and running premade tests on it. Specificaly this project https://github.com/wakaleo/game-of-life. You will use some Jenkins plugins like Cobertura. I tried it and it is buggy. I recommend using Jacoco instead. Here are the plugins used in this chapter: Git plugin, GitHub plugin, Coberturra plugin,Jacoco plugin.
3. Chapter 3: Installing Jenkins. Covers how to install, possible ways how to install o different OS systems. Jenkins home directory, memory considerations, java/maven/ant options etc... Upgrading, backing up jenkins etc...
4. Chapter 4: Configurin your Jenkins server. Adding JDK/Maven/Ant installations. Configuring global properties, mail server, reverse proxy etc...
5. Chapter 5: Setting up your build jobs. Covers types of builds, like free style job or mave build job. How to configure source code management tools like SVN and Git trough Git/GitHub plugins. Build triggers, polling the SCM, or calling a hook script to start the build process. Build steps, post build steps, and other useful plugins. Plugins mentioned:
   Gerrit Trigger plugin, Groovy, Gradle, Grails, Jython, MSBuild, Phing, Phyton, Rake, Ruby, Artifactory plugin, Nexus plugin, NUnit plugin, NAnt plugin
6. Chapter 6: Automated Testing. Automatin Unit  and Acceptance tests. Configuring test reports, code coverage, automated preformance testing with JMeter. Plugins mentioned: Clover, NCover, Emma, HTML Publisher plugin, JMeter plugin, Performance plugin
7. Chapter 7: Securing Jenkins. Activating security. Using simple security or security with LDAP server. Then using Active Directory or Unix users and groups. Atlassian Crowd commercial tool. Authorization, who can do what with matrix based security or with project based matrix security. There is even a tool for enabling Role based security with a plugin. Auditing, keeping track of user actions. Plugins mentioned: LDAP Plugin, Active Directory plugin, Crowd plugin, Role Strategy plugin, Audit Trail plugin, Job Configuration History plugin.
8. Capter 8: Notification. Email notification, claiming builds, instant messaging or IRC messaging. Desktop notifiers like Notifo, mobile/SMS notification, making noise with extreme feedback devices. Plugins mentioned: Email Extension plugin, Instant Messaging plugin, Jabber notifier plugin.
9. Chapter 9: Code Quality. Code quality in your build process. Code quality tools like  PMD/CPD, FindBugs, Checkstyle, CodeNarc. Reporting code quality problems with Violations plugin. Reporting with SonarQube. Plugins mentioned: Violations plugin, FindBugs, PMD/CPD, Checkstyle, JDepend, NCover, CodeNarc, Sonar Gerrit Plugin, SonarQube plugin
   Coverage Complexity Scatter plugin, Static Analysis Utilities plugin, Task Scanner plugin.
10. Chapter 10: Advanced builds. Parametrized builds and parametrized triggers. Multi-configuration build jobs. Parallel builds and build pipelines. Plugins mentioned: Build Promotion plugin, Maven Release Plugin, Copy Artifact plugin, promoted builds plugin, ArtifactPromotion plugin, Build Pipeline plugin, Parametrized Trigger plugin, Maven Jenkins plugin, Dependency Graph Viewer plugin, Locks and Latches plugin.
11. Chapter 11: Distributed builds. Master/Slave strategies. Associating builds to slaves. Node monitoring and cloud computing. Plugins mentioned: Amazon EC2 plugin, CloudBees Docker Build and Publish plugin.
12. Chapter 12: Automated Deployment and Continous delivery. Deployment script, database updates, smoke tests. Deploying Java or PHP/Ruby application. Plugins mentioned: Capitomcat plugin.
13. Chapter 13: Maintaining Jenkins. Monitoring disk space and server load. Backing up configuration and Jenkins builds. Archiving and migrating build jobs. Plugins mentioned: Disk Usage plugin, Backup plugin, ThinBackup plugin, Monitoring plugin, Deploy plugin, Deploy WebSphere
14. Appendix: Automating your Unit and Integration Tests. Automating your tests with Maven and Ant. 

I hope this post is useful. Best regards. Vlado