gcov

gcovA coverage analysis tool that reveals how much of the code was actually exercised.

gcov_compiler

gcov_compiler is a wrapper for gcc. It will invoke the GNU compiler collection (gcc) to compile the code and to instrument the binary code for coverage analysis. It has the same interface as gcc which is also called by this script. This results in .bb and .bbg files for every compilation unit (source code file). When the program, that has been compiled with this script, is executed, coverage logging is automatically enabled. After execution, a .da file will exist for every compilation unit that has been compiled with gcov_compiler. The results of the coverage analysis can be evaluated with gcov_scan(1).

Analyzing the results

The results of the coverage analysis can be evaluated after executing the code. See qac_scan(1).

gcov_scan

gcov_scan is a script which evaluated the coverage results from executing the instrumented code. It will check the amount of coverage with the expected coverage level as specified in the suppression tag. By default, a coverage level of 80 percent of statement coverage is required. Otherwise a warning is issued.

Output

Normally, the output format of gcov_scan is he same as the gcc compiler collection. This format is $<$file$>$,$<$line$>$,$<$message$>$. This allows the programming to parse the output by an editor. It is possible to generate output in HTML output. Add -html as an optional argument for this purpose.

It is also important to note that gcov_scan also results in an annotated source file in which it is specified which line or block has not been executed. This is needful in order to design effective test cases that achieves more coverage.

Suppression tags

It is important that warnings with respect to the coverage constraints, can be suppressed by a special tag that must be added to the compilation unit by comment. Below, an example is given in which a minimal constraint of 60 percent statement coverage is enabled. This means, a minimal of 60 percent of statement coverage must be achieved by unit testing. By default, a minimum of 80 percent of statement coverage is applied.

/*
 *
 * This tag must be somewhere in the code
 *
 *
 * GCOV EXPECT 60
 *
 */

The regular expression of the GCOV expect tag is as follows :

".*\/*GCOV\ *EXPECT\ *[0-9]+\ *(BRANCH)?.*"

When BRANCH is specified, branch coverage will be in effect for the scan.

Dealing with infeasible coverage

It can be very time consuming to achieve a 100 % statement- or branch coverage. If certain parts of the code are difficult to test because, for instance, a specific exception is hard to enforce during testing, this part of the code must be labelled as infeasible coverage as follows:

.....
catch( ... )
{
        // <INFEASIBLE_COVERAGE>
        // <REASON> We don't know the type of exception to expect here </REASON>

         ...... (code that cannot be tested)

        // </INFEASIBLE_COVERAGE>
}

When the code of a class contains infeasible coverage parts, these must documenten as shown by the previous example and discussed with another designer/implementer.

Example

The next example shows a piece of code which contains a switch

erwinn>> more test.cpp
        #include <iostream>
        using std::cout;
        using std::endl;

        // QACPP EXPECT 4327; argv will not be used
        int main ( int argc, const char* argv )
        {
                cout << "hello nerd" << endl;
                switch( argc )
                {
                        case 1:
                        {
                                cout << "nice" << endl;
                                break;
                        }
                        case 2:
                        {
                                cout << "also nice" << endl;
                                break;
                        }
                        default:
                        {
                                cout << "something is wrong" << endl;
                                break;
                        }

                }

                return 0;
        }

Compiling this code using the gcov_compiler will prepare the application for coverage analysis:

erwinn>> gcov_compiler test.cpp
erwinn>> ls
a.out     test.bb   test.bbg  test.cpp  test.o
erwinn>>

Typically, files with extension .bb and .bbg are generated. These file are needed by the coverage analysis. Executing the program will result in a .da file which contains coverage information collected during the execution:

erwinn>> gcov_compiler test.cpp
erwinn>> ls
a.out     test.bb   test.bbg  test.cpp  test.o
erwinn>> ./a.out
hello nerd
nice
erwinn>> ls
a.out     test.bb   test.bbg  test.cpp  test.da   test.o
erwinn>>

The results can be analysed by the gcov_scan tool:

uns67c86-di502> gcov_scan
In file : test.cpp
                72% (statement) coverage

Verdict : fail
erwinn>>

gcov_scan will automatically generate a file named test.cpp.gcov containing annotated source code. This file show which part of the source has not been covered yet. This helps to come up with new ideas for test input.

erwinn>> more test.cpp.gcov
                        #include <iostream>
                        using std::cout;
                        using std::endl;


           1            int main ( int argc, const char* argv )
           1            {
           1                    cout << "hello nerd" << endl;
                                switch( argc )
                                {
                                        case 1:
           1                            {
           1                                    cout << "nice" << endl;
                                                break;
                                        }
                                        case 2:
      ######                            {
      ######                                    cout << "also nice" << endl;
                                                break;
                                        }
                                        default:
      ######                            {
           1                                    cout << "something is wrong" << endl;
                                                break;
                                        }

                                }
           1
           1                    return 0;
                        }
erwinn>>

By default 80% statment coverage must be achieved, therefore gcov_scan issues this warning. If we would add an extra argmument on the command line, we will achieve more coverage:

erwinn>> ./a.out try2
hello nerd
also nice
erwinn>> gcov_scan
Verdict : success


Now, more than 80\% of statement coverage is achieved. The test.cpp.gcov file shows which part has still not been excersized:
\begin{verbatim}
erwinn>> more test.cpp.gcov
                        #include <iostream>
                        using std::cout;
                        using std::endl;


                        // QACPP EXPECT 4327; argv will not be used
           2            int main ( int argc, const char* argv )
           2            {
           2                    cout << "hello nerd" << endl;
                                switch( argc )
                                {
                                        case 1:
           1                            {
           1                                    cout << "nice" << endl;
                                                break;
                                        }
                                        case 2:
           1                            {
           1                                    cout << "also nice" << endl;
                                                break;
                                        }
                                        default:
      ######                            {
           2                                    cout << "something is wrong" << endl;
                                                break;
                                        }

                                }
           2
           2                    return 0;
                        }
erwinn>>

When this would not be a piece of safety critical software, this coverage would be sufficient. From this we conclude that two test cases are needed to achieved the required percentage of coverage.