Introduction to Software Metrics

Measurement

Measurement is a process via which numbers or symbols are assigned to attributes of entities in the real world in order to describe them according to some set of rules.

  • An entity is an object or event in the real world
  • Attributes are features or properties of entities

Example of entities / attributes in software engineering

Entity Attribute Measure
Program code Length # of lines of code (LOC)
Program code Quality # of faults found per KLOC
Program code Reliability Mean time to failure (MTF) in CPU hours
Completed project Duration Months from start to finish

Representational theory of Measurement

The representational theory of measurement seeks to formalise our intuition about the way the world works. It states that:

  • Data we obtain as measures should represent attributes of the entities we observe
  • Manipulation of the data should preserve relationships that we observe among the entities
  • Our intuition is the starting point for all measurement

Empirical relations

An empirical relation is one for which there is a reasonable consensus about which entities are in the relation. For example, "taller than" is a relation defined on the set of pairs of people - we say that "taller than" is an empirical relation for height.

Formally we define measurement as a mapping from empirical world → formal relational world. A measurement is the number/symbol assigned to an entity by this mapping in order to characterise an attribute.

The representational condition of measurement

The measurement mapping M must map attributes of entities into numbers and empirical relations to numerical relations in such a way that the empirical relations preserve and are preserved by the numerical relations.

eg.
Joe is taller than (empirical relation) Fred if and only if M(Joe) > M(Fred) (numerical relation)


Measurement and models

A model is an abstraction of reality. This allows us to strip away detail and view an entity or concept from a particular perspective. Models come in many different forms (equations, diagrams, mappings, etc..).
The representational condition requires every measure to be associated with a model of how the measure maps the entities and attributes in the real world to the elements of a numerical system.

Example: to measure the length of programs using lines of code we need to create a model of a program:…

  • Do we treat separate statements on the same line as distinct lines of code?
  • Should comments be counted?
  • Do we count data declarations?

It should also handle:

  • Programs written in a combination of languages
  • Situations where different versions run on different platforms

Direct Measurement

Direct measurement of an attribute of an entity involves no other attribute or entity (eg: the length of a physical object can be measured without reference to any other objects or attributes).

Examples (Software engineering):

  • Length of source code (LOC)
  • Duration of testing process (hours)
  • Number of defects found during testing

Indirect Measurement

Indirect measurement of an attribute of an entity involves other attributes or entities (eg: density of a physical object can only be measured in terms of mass and volume).
Examples (Software engineering):

  • Module defect density (# of defects / module size)
  • Requirement stability (# of initial requirements / total # of requirements)

Other Examples of Common Indirect Measures
$\mbox{Programmer productivity} = \frac{\mbox{LOC produced}}{\mbox{Person months of effort}}$
$\mbox{Defect detection efficiency} = \frac{\mbox{Number of defects detected}}{\mbox{Total number of defects}}$
$\mbox{Test effectiveness ratio} = \frac{\mbox{Number of items covered}}{\mbox{Total number of items}}$
$\mbox{System spoilage} = \frac{\mbox{Effect spent fixing faults}}{\mbox{Total project effort}}$

Measurement scales

The purpose of measurement (ie: performing the mapping between an empirical relation and a numerical system) is to:

  • Be able to manipulate the data
  • Draw conclusions about the attribute in the empirical system

The differences among the mappings restrict the kind of analysis we can do. Measurement scales help us to understand these differences. There are five major types of measurement scales:

  • Nominal
  • Ordinal
  • Interval
  • Ratio
  • Absolute

Transformation

A relational system R is said to be richer that another S if all relations in S are contained in R. A measurement scale allows certain transformations. eg:

  • Length can be measured in a number of different units (cm, m, inch…)
  • A measurement in metres can be transformed to a measurement in cm by the transformation: M' = aM (where a = 100).
  • However, transformations such as M' = b + aM or M' = aMb are not allowed in the scale used for length measurement

"A measurement from one acceptable measure to another acceptable one is called an admissible transformation".

Nominal scale

A nominal scale is a primitive form of measurement. In it entities are grouped into different classes / categories based on the value of some attribute.

  • The empirical relation system consists of different classes (no notion of ordering among the classes)
  • Any distinct numbering of (or symbols used for) the classes is an acceptable measure (but there is no notion of magnitude associated with the numbers or symbols).

Example: say we wish to measure the source of faults in software. We can have a measure for a fault x as:

  • M(x) = 1 … if x is a specification fault
  • M(x) = 2 … if x is a design fault
  • M(x) = 3 … if x is a code fault

There is, however, no magnitude relation between the numbers 1, 2 and 3.
The class of admissible transformations for a nominal scale measure is the set of all one-to-one mappings:

  • Any two mappings M and M', where M' is attained from M by one-to-one mapping
  • No arithmetic operations can be meaningfully applied to the classes.

Ordinal scale

The empirical relation system consists of classes that are ordered with respect to the attribute.

  • Any mapping preserving this order (any monotonic function) is acceptable.
  • The numbers represent ranking only so arithmetic operations have no meaning.
  • The classes can be combined as long as the combination makes sense with respect to the ordering.

Example: say we want to measure defects in a software product. We define four different classes based on the complexity of the defect (trivial, simple, moderate, serious).

  • There is a "more complex than" ordering between classes (a numerical representation of this should also preserve this relation)
    • M(x) = 1 … if x trivial
    • M(x) = 2 … if x simple
    • M(x) = 3 … if x moderate
    • M(x) = 4 … if x serious

Interval scale

The interval scale carries even more information than the nominal and ordinal scales.

  • Preserves order (ordinal scale)
  • Preserves differences but not ratios
  • Addition and subtraction are applicable to the classes in the range of the mapping (but not multiplication and division)
  • For some numbers a and b, the mapping M = aM' + b is allowable.

Example: temperature measurement is a interval scale.
Measurement in Celsius can be transformed to a measurement in Fahrenheit using:

(1)
\begin{align} F = \frac{9}{5} C + 32 \end{align}

Ratio Scale

Ratio scales;

  • Preserve ordering, interval size and ratios
  • Have a zero element representing the absence of the attribute
  • Measurement mappings must start at zero and increase at equal intervals (units)
  • All arithmetic operations can be meaningfully applied to the classes in the range of the mapping
  • For some positive scalar a, M = aM' is allowable.

Example: can measure length in different units.

  • If length (a) = 10 and length(b) = 20, we can say that b is twice as long as a.

Absolute scale

  • Measurement is made simply by counting the number of elements in an entity set
  • The attribute always takes the form "the number of occurances of x in the entity".
  • There is only one possible measurement mapping (that is, actually count)
  • Transformation: M(x) = M'(x)
  • All arithmetic operations are applicable.

Measurement in software

Measurement in software is often a luxury - most engineers fail to set measurable targets for software products. Eg:

  • User friendly, reliable, maintainable…
  • How do we determine whether or not we have achieved these goals?

Gilb's principle

Gilb's principle states:

"Projects without clear goals will not achieve their goals clearly".

Why measure software?

We should measure software because we cannot control what we cannot measure. Measurement is a fundamental principle of any engineering discipline. As well as this, managers need to be able to determine things like cost, code-quality and user satisfaction.
Finally engineers need to know whether the requirements are testable, have all faults been determined, has a product met its goal(s) and what will/should happen in future.

Objectives of Software Measurement

  • Understanding: can make aspects of the process and product more visible
  • Control: can make changes to processes and products to help us meet our goals
  • Improvement: helps us to improve our future processes / products
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License