Code Defenders - Study 2016

Contents

• Introduction
• Survey Results
• Selected Classes
• Generated Tests and Mutants
• Multiplayer Scoring System

Introduction

This page provides additional information on Code Defenders experiments ran in 2016.

Survey Results

Below are the results for both surveys in CSV format.

• Suitability Survey Results
• Crowdsourcing Survey Results

Class Selection

There were 20 classes selected

Game ID	Class Name	Source Project	NCSS*	Major Mutants
1001	ByteArrayHashMap	sf-vuze	179	17
1002	ByteVector	sf-jiprof	128	31
1003	ChunkedLongArray	sf-summa	102	23
1004	FTPFile	ac-net	158	11
1005	FontInfo	sf-squirrel-sql	104	6
1006	HierarchyPropertyParser	sf-weka	261	28
1007	HSLColor	sf-vuze	160	65
1008	ImprovedStreamTokenizer	sf-caloriecount	128	11
1009	ImprovedTokenizer	sf-caloriecount	163	7
1010	Inflection	sf-schemaspy	112	10
1011	IntHashMap	sf-vuze	113	14
1012	ParameterParser	ac-fileupload	108	17
1013	Range	ac-lang3	128	15
1014	RationalNumber	ac-imaging	108	28
1015	SubjectParser	sf-newzgrabber	117	13
1016	TimeStamp	ac-net	103	20
1017	VCardBean	sf-heal	188	18
1018	WeakHashtable	ac-logging	168	9
1019	XmlElement	sf-inspirento	196	16
1020	XMLParser	sf-fim1	162	76

* NCSS - None Comment Source Statements

To see the refactored versions of these classes, download the zip

Generated Tests and Mutants

To download the tests suites generated from these classes, click here

To download the mutants generated from these classes, click here

Multiplayer Scoring System Example

Alice (attacker)		Andy (attacker)		Dan (defender)		David (defender)		Explanation
Action	Points	Action	Points	Action	Points	Action	Points
m1								Alice submits m1.
	1							No tests exist, m1 gains a point.
		m2						Andy submits m2.
			1					No tests exist, m2 gains a point.
				t1				Dan submits t1
	2	m2	1		2			m1 survives t1, gaining a point. m2 killed by t1
						t2		David submits t2
m1	2						3	m1 killed by t2
m3								Alice submits m3
m3	2						4	m3 survives t1, but is killed by t2
		m4						Andy submits m4
			4					m4 survives all current tests
m5								Alice submits m5
	5							m5 survives all current tests
				eq(m5)				Dan marks m5 as equivalent
m5	2				3			Alice accepts m5 as equivalent, losing its points. Dan gains a point.
		m6						Andy submits m6
			7					m6 survives all current tests
						eq(m6)		David marks m6 as equivalent
		m6	7					Andy submits a killing test, proving m6 is not equivalent
	2		7		3		4	Game ends, Attackers win 9 : 7

All tests execute all mutated lines in this example. Points are cumulative for respective player