1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Test</title><meta name="generator" content="DocBook XSL-NS Stylesheets V1.76.1"/><meta name="keywords" content=" ISO C++ , test , testsuite , performance , conformance , ABI , exception safety "/><meta name="keywords" content=" ISO C++ , library "/><link rel="home" href="../spine.html" title="The GNU C++ Library"/><link rel="up" href="appendix_porting.html" title="Appendix B. Porting and Maintenance"/><link rel="prev" href="internals.html" title="Porting to New Hardware or Operating Systems"/><link rel="next" href="abi.html" title="ABI Policy and Guidelines"/></head><body><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Test</th></tr><tr><td align="left"><a accesskey="p" href="internals.html">Prev</a> </td><th width="60%" align="center">Appendix B.
Porting and Maintenance
</th><td align="right"> <a accesskey="n" href="abi.html">Next</a></td></tr></table><hr/></div><div class="section" title="Test"><div class="titlepage"><div><div><h2 class="title"><a id="manual.intro.setup.test"/>Test</h2></div></div></div><p>
The libstdc++ testsuite includes testing for standard conformance,
regressions, ABI, and performance.
</p><div class="section" title="Organization"><div class="titlepage"><div><div><h3 class="title"><a id="test.organization"/>Organization</h3></div></div></div><div class="section" title="Directory Layout"><div class="titlepage"><div><div><h4 class="title"><a id="test.organization.layout"/>Directory Layout</h4></div></div></div><p>
The directory <span class="emphasis"><em>libsrcdir/testsuite</em></span> contains the
individual test cases organized in sub-directories corresponding to
chapters of the C++ standard (detailed below), the dejagnu test
harness support files, and sources to various testsuite utilities
that are packaged in a separate testing library.
</p><p>
All test cases for functionality required by the runtime components
of the C++ standard (ISO 14882) are files within the following
directories.
</p><pre class="programlisting">
17_intro
18_support
19_diagnostics
20_util
21_strings
22_locale
23_containers
25_algorithms
26_numerics
27_io
28_regex
29_atomics
30_threads
</pre><p>
In addition, the following directories include test files:
</p><pre class="programlisting">
tr1 Tests for components as described by the Technical Report on Standard Library Extensions (TR1).
backward Tests for backwards compatibility and deprecated features.
demangle Tests for __cxa_demangle, the IA 64 C++ ABI demangler
ext Tests for extensions.
performance Tests for performance analysis, and performance regressions.
</pre><p>
Some directories don't have test files, but instead contain
auxiliary information:
</p><pre class="programlisting">
config Files for the dejagnu test harness.
lib Files for the dejagnu test harness.
libstdc++* Files for the dejagnu test harness.
data Sample text files for testing input and output.
util Files for libtestc++, utilities and testing routines.
</pre><p>
Within a directory that includes test files, there may be
additional subdirectories, or files. Originally, test cases
were appended to one file that represented a particular section
of the chapter under test, and was named accordingly. For
instance, to test items related to <code class="code"> 21.3.6.1 -
basic_string::find [lib.string::find]</code> in the standard,
the following was used:
</p><pre class="programlisting">
21_strings/find.cc
</pre><p>
However, that practice soon became a liability as the test cases
became huge and unwieldy, and testing new or extended
functionality (like wide characters or named locales) became
frustrating, leading to aggressive pruning of test cases on some
platforms that covered up implementation errors. Now, the test
suite has a policy of one file, one test case, which solves the
above issues and gives finer grained results and more manageable
error debugging. As an example, the test case quoted above
becomes:
</p><pre class="programlisting">
21_strings/basic_string/find/char/1.cc
21_strings/basic_string/find/char/2.cc
21_strings/basic_string/find/char/3.cc
21_strings/basic_string/find/wchar_t/1.cc
21_strings/basic_string/find/wchar_t/2.cc
21_strings/basic_string/find/wchar_t/3.cc
</pre><p>
All new tests should be written with the policy of one test
case, one file in mind.
</p></div><div class="section" title="Naming Conventions"><div class="titlepage"><div><div><h4 class="title"><a id="test.organization.naming"/>Naming Conventions</h4></div></div></div><p>
In addition, there are some special names and suffixes that are
used within the testsuite to designate particular kinds of
tests.
</p><div class="itemizedlist"><ul class="itemizedlist"><li class="listitem"><p>
<span class="emphasis"><em>_xin.cc</em></span>
</p><p>
This test case expects some kind of interactive input in order
to finish or pass. At the moment, the interactive tests are not
run by default. Instead, they are run by hand, like:
</p><pre class="programlisting">
g++ 27_io/objects/char/3_xin.cc
cat 27_io/objects/char/3_xin.in | a.out
</pre></li><li class="listitem"><p>
<span class="emphasis"><em>.in</em></span>
</p><p>
This file contains the expected input for the corresponding <span class="emphasis"><em>
_xin.cc</em></span> test case.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>_neg.cc</em></span>
</p><p>
This test case is expected to fail: it's a negative test. At the
moment, these are almost always compile time errors.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>char</em></span>
</p><p>
This can either be a directory name or part of a longer file
name, and indicates that this file, or the files within this
directory are testing the <code class="code">char</code> instantiation of a
template.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>wchar_t</em></span>
</p><p>
This can either be a directory name or part of a longer file
name, and indicates that this file, or the files within this
directory are testing the <code class="code">wchar_t</code> instantiation of
a template. Some hosts do not support <code class="code">wchar_t</code>
functionality, so for these targets, all of these tests will not
be run.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>thread</em></span>
</p><p>
This can either be a directory name or part of a longer file
name, and indicates that this file, or the files within this
directory are testing situations where multiple threads are
being used.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>performance</em></span>
</p><p>
This can either be an enclosing directory name or part of a
specific file name. This indicates a test that is used to
analyze runtime performance, for performance regression testing,
or for other optimization related analysis. At the moment, these
test cases are not run by default.
</p></li></ul></div></div></div><div class="section" title="Running the Testsuite"><div class="titlepage"><div><div><h3 class="title"><a id="test.run"/>Running the Testsuite</h3></div></div></div><div class="section" title="Basic"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.basic"/>Basic</h4></div></div></div><p>
You can check the status of the build without installing it
using the dejagnu harness, much like the rest of the gcc
tools.</p><pre class="programlisting"> make check</pre><p>in the <span class="emphasis"><em>libbuilddir</em></span> directory.</p><p>or</p><pre class="programlisting"> make check-target-libstdc++-v3</pre><p>in the <span class="emphasis"><em>gccbuilddir</em></span> directory.
</p><p>
These commands are functionally equivalent and will create a
'testsuite' directory underneath
<span class="emphasis"><em>libbuilddir</em></span> containing the results of the
tests. Two results files will be generated: <span class="emphasis"><em>
libstdc++.sum</em></span>, which is a PASS/FAIL summary for each
test, and <span class="emphasis"><em>libstdc++.log</em></span> which is a log of
the exact command line passed to the compiler, the compiler
output, and the executable output (if any).
</p><p>
Archives of test results for various versions and platforms are
available on the GCC website in the <a class="link" href="http://gcc.gnu.org/gcc-4.3/buildstat.html">build
status</a> section of each individual release, and are also
archived on a daily basis on the <a class="link" href="http://gcc.gnu.org/ml/gcc-testresults/current">gcc-testresults</a>
mailing list. Please check either of these places for a similar
combination of source version, operating system, and host CPU.
</p></div><div class="section" title="Variations"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.variations"/>Variations</h4></div></div></div><p>
There are several options for running tests, including testing
the regression tests, testing a subset of the regression tests,
testing the performance tests, testing just compilation, testing
installed tools, etc. In addition, there is a special rule for
checking the exported symbols of the shared library.
</p><p>
To debug the dejagnu test harness during runs, try invoking with a
specific argument to the variable RUNTESTFLAGS, as below.
</p><pre class="programlisting">
make check-target-libstdc++-v3 RUNTESTFLAGS="-v"
</pre><p>
or
</p><pre class="programlisting">
make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v"
</pre><p>
To run a subset of the library tests, you will need to generate
the <span class="emphasis"><em>testsuite_files</em></span> file by running
<span class="command"><strong>make testsuite_files</strong></span> in the
<span class="emphasis"><em>libbuilddir/testsuite</em></span> directory, described
below. Edit the file to remove the tests you don't want and
then run the testsuite as normal.
</p><p>
There are two ways to run on a simulator: set up DEJAGNU to point to a
specially crafted site.exp, or pass down --target_board flags.
</p><p>
Example flags to pass down for various embedded builds are as follows:
</p><pre class="programlisting">
--target=powerpc-eabism (libgloss/sim)
make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim"
--target=calmrisc32 (libgloss/sid)
make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid"
--target=xscale-elf (newlib/sim)
make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim"
</pre><p>
Also, here is an example of how to run the libstdc++ testsuite
for a multilibed build directory with different ABI settings:
</p><pre class="programlisting">
make check-target-libstdc++-v3 RUNTESTFLAGS='--target_board \"unix{-mabi=32,,-mabi=64}\"'
</pre><p>
You can run the tests with a compiler and library that have
already been installed. Make sure that the compiler (e.g.,
<code class="code">g++</code>) is in your <code class="code">PATH</code>. If you are
using shared libraries, then you must also ensure that the
directory containing the shared version of libstdc++ is in your
<code class="code">LD_LIBRARY_PATH</code>, or equivalent. If your GCC source
tree is at <code class="code">/path/to/gcc</code>, then you can run the tests
as follows:
</p><pre class="programlisting">
runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite
</pre><p>
The testsuite will create a number of files in the directory in
which you run this command,. Some of those files might use the
same name as files created by other testsuites (like the ones
for GCC and G++), so you should not try to run all the
testsuites in parallel from the same directory.
</p><p>
In addition, there are some testing options that are mostly of
interest to library maintainers and system integrators. As such,
these tests may not work on all cpu and host combinations, and
may need to be executed in the
<span class="emphasis"><em>libbuilddir/testsuite</em></span> directory. These
options include, but are not necessarily limited to, the
following:
</p><pre class="programlisting">
make testsuite_files
</pre><p>
Five files are generated that determine what test files
are run. These files are:
</p><div class="itemizedlist"><ul class="itemizedlist"><li class="listitem"><p>
<span class="emphasis"><em>testsuite_files</em></span>
</p><p>
This is a list of all the test cases that will be run. Each
test case is on a separate line, given with an absolute path
from the <span class="emphasis"><em>libsrcdir/testsuite</em></span> directory.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsuite_files_interactive</em></span>
</p><p>
This is a list of all the interactive test cases, using the
same format as the file list above. These tests are not run
by default.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsuite_files_performance</em></span>
</p><p>
This is a list of all the performance test cases, using the
same format as the file list above. These tests are not run
by default.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsuite_thread</em></span>
</p><p>
This file indicates that the host system can run tests which
involved multiple threads.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsuite_wchar_t</em></span>
</p><p>
This file indicates that the host system can run the wchar_t
tests, and corresponds to the macro definition <code class="code">
_GLIBCXX_USE_WCHAR_T</code> in the file c++config.h.
</p></li></ul></div><pre class="programlisting">
make check-abi
</pre><p>
The library ABI can be tested. This involves testing the shared
library against an ABI-defining previous version of symbol
exports.
</p><pre class="programlisting">
make check-compile
</pre><p>
This rule compiles, but does not link or execute, the
<span class="emphasis"><em>testsuite_files</em></span> test cases and displays the
output on stdout.
</p><pre class="programlisting">
make check-performance
</pre><p>
This rule runs through the
<span class="emphasis"><em>testsuite_files_performance</em></span> test cases and
collects information for performance analysis and can be used to
spot performance regressions. Various timing information is
collected, as well as number of hard page faults, and memory
used. This is not run by default, and the implementation is in
flux.
</p><p>
We are interested in any strange failures of the testsuite;
please email the main libstdc++ mailing list if you see
something odd or have questions.
</p></div><div class="section" title="Permutations"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.permutations"/>Permutations</h4></div></div></div><p>
To run the libstdc++ test suite under the <a class="link" href="debug_mode.html" title="Chapter 17. Debug Mode">debug mode</a>, edit
<code class="filename">libstdc++-v3/scripts/testsuite_flags</code> to add the
compile-time flag <code class="constant">-D_GLIBCXX_DEBUG</code> to the
result printed by the <code class="literal">--build-cxx</code>
option. Additionally, add the
<code class="constant">-D_GLIBCXX_DEBUG_PEDANTIC</code> flag to turn on
pedantic checking. The libstdc++ test suite should produce
precisely the same results under debug mode that it does under
release mode: any deviation indicates an error in either the
library or the test suite.
</p><p>
The <a class="link" href="parallel_mode.html" title="Chapter 18. Parallel Mode">parallel
mode</a> can be tested in much the same manner, substituting
<code class="constant">-D_GLIBCXX_PARALLEL</code> for
<code class="constant">-D_GLIBCXX_DEBUG</code> in the previous paragraph.
</p><p>
Or, just run the testsuites with <code class="constant">CXXFLAGS</code>
set to <code class="constant">-D_GLIBCXX_DEBUG</code> or
<code class="constant">-D_GLIBCXX_PARALLEL</code>.
</p></div></div><div class="section" title="Writing a new test case"><div class="titlepage"><div><div><h3 class="title"><a id="test.new_tests"/>Writing a new test case</h3></div></div></div><p>
The first step in making a new test case is to choose the correct
directory and file name, given the organization as previously
described.
</p><p>
All files are copyright the FSF, and GPL'd: this is very
important. The first copyright year should correspond to the date
the file was checked in to SVN.
</p><p>
As per the dejagnu instructions, always return 0 from main to
indicate success.
</p><p>
A bunch of utility functions and classes have already been
abstracted out into the testsuite utility library, <code class="code">
libtestc++</code>. To use this functionality, just include the
appropriate header file: the library or specific object files will
automatically be linked in as part of the testsuite run.
</p><p>
For a test that needs to take advantage of the dejagnu test
harness, what follows below is a list of special keyword that
harness uses. Basically, a test case contains dg-keywords (see
dg.exp) indicating what to do and what kinds of behavior are to be
expected. New test cases should be written with the new style
DejaGnu framework in mind.
</p><p>
To ease transition, here is the list of dg-keyword documentation
lifted from dg.exp.
</p><pre class="programlisting">
# The currently supported options are:
#
# dg-prms-id N
# set prms_id to N
#
# dg-options "options ..." [{ target selector }]
# specify special options to pass to the tool (eg: compiler)
#
# dg-do do-what-keyword [{ target/xfail selector }]
# `do-what-keyword' is tool specific and is passed unchanged to
# ${tool}-dg-test. An example is gcc where `keyword' can be any of:
# preprocess|compile|assemble|link|run
# and will do one of: produce a .i, produce a .s, produce a .o,
# produce an a.out, or produce an a.out and run it (the default is
# compile).
#
# dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]]
# indicate an error message <regexp> is expected on this line
# (the test fails if it doesn't occur)
# Linenum=0 for general tool messages (eg: -V arg missing).
# "." means the current line.
#
# dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]]
# indicate a warning message <regexp> is expected on this line
# (the test fails if it doesn't occur)
#
# dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]]
# indicate a bogus error message <regexp> use to occur here
# (the test fails if it does occur)
#
# dg-build regexp comment [{ target/xfail selector }]
# indicate the build use to fail for some reason
# (errors covered here include bad assembler generated, tool crashes,
# and link failures)
# (the test fails if it does occur)
#
# dg-excess-errors comment [{ target/xfail selector }]
# indicate excess errors are expected (any line)
# (this should only be used sparingly and temporarily)
#
# dg-output regexp [{ target selector }]
# indicate the expected output of the program is <regexp>
# (there may be multiple occurrences of this, they are concatenated)
#
# dg-final { tcl code }
# add some tcl code to be run at the end
# (there may be multiple occurrences of this, they are concatenated)
# (unbalanced braces must be \-escaped)
#
# "{ target selector }" is a list of expressions that determine whether the
# test succeeds or fails for a particular target, or in some cases whether the
# option applies for a particular target. If the case of `dg-do' it specifies
# whether the test case is even attempted on the specified target.
#
# The target selector is always optional. The format is one of:
#
# { xfail *-*-* ... } - the test is expected to fail for the given targets
# { target *-*-* ... } - the option only applies to the given targets
#
# At least one target must be specified, use *-*-* for "all targets".
# At present it is not possible to specify both `xfail' and `target'.
# "native" may be used in place of "*-*-*".
Example 1: Testing compilation only
// { dg-do compile }
Example 2: Testing for expected warnings on line 36, which all targets fail
// { dg-warning "string literals" "" { xfail *-*-* } 36
Example 3: Testing for expected warnings on line 36
// { dg-warning "string literals" "" { target *-*-* } 36
Example 4: Testing for compilation errors on line 41
// { dg-do compile }
// { dg-error "no match for" "" { target *-*-* } 41 }
Example 5: Testing with special command line settings, or without the
use of pre-compiled headers, in particular the stdc++.h.gch file. Any
options here will override the DEFAULT_CXXFLAGS and PCH_CXXFLAGS set
up in the normal.exp file.
// { dg-options "-O0" { target *-*-* } }
</pre><p>
More examples can be found in the libstdc++-v3/testsuite/*/*.cc files.
</p></div><div class="section" title="Test Harness and Utilities"><div class="titlepage"><div><div><h3 class="title"><a id="test.harness"/>Test Harness and Utilities</h3></div></div></div><div class="section" title="Dejagnu Harness Details"><div class="titlepage"><div><div><h4 class="title"><a id="test.harness.dejagnu"/>Dejagnu Harness Details</h4></div></div></div><p>
Underlying details of testing for conformance and regressions are
abstracted via the GNU Dejagnu package. This is similar to the
rest of GCC.
</p><p>This is information for those looking at making changes to the testsuite
structure, and/or needing to trace dejagnu's actions with --verbose. This
will not be useful to people who are "merely" adding new tests to the existing
structure.
</p><p>The first key point when working with dejagnu is the idea of a "tool".
Files, directories, and functions are all implicitly used when they are
named after the tool in use. Here, the tool will always be "libstdc++".
</p><p>The <code class="code">lib</code> subdir contains support routines. The
<code class="code">lib/libstdc++.exp</code> file ("support library") is loaded
automagically, and must explicitly load the others. For example, files can
be copied from the core compiler's support directory into <code class="code">lib</code>.
</p><p>Some routines in <code class="code">lib/libstdc++.exp</code> are callbacks, some are
our own. Callbacks must be prefixed with the name of the tool. To easily
distinguish the others, by convention our own routines are named "v3-*".
</p><p>The next key point when working with dejagnu is "test files". Any
directory whose name starts with the tool name will be searched for test files.
(We have only one.) In those directories, any <code class="code">.exp</code> file is
considered a test file, and will be run in turn. Our main test file is called
<code class="code">normal.exp</code>; it runs all the tests in testsuite_files using the
callbacks loaded from the support library.
</p><p>The <code class="code">config</code> directory is searched for any particular "target
board" information unique to this library. This is currently unused and sets
only default variables.
</p></div><div class="section" title="Utilities"><div class="titlepage"><div><div><h4 class="title"><a id="test.harness.utils"/>Utilities</h4></div></div></div><p>
</p><p>
The testsuite directory also contains some files that implement
functionality that is intended to make writing test cases easier,
or to avoid duplication, or to provide error checking in a way that
is consistent across platforms and test harnesses. A stand-alone
executable, called <span class="emphasis"><em>abi_check</em></span>, and a static
library called <span class="emphasis"><em>libtestc++</em></span> are
constructed. Both of these items are not installed, and only used
during testing.
</p><p>
These files include the following functionality:
</p><div class="itemizedlist"><ul class="itemizedlist"><li class="listitem"><p>
<span class="emphasis"><em>testsuite_abi.h</em></span>,
<span class="emphasis"><em>testsuite_abi.cc</em></span>,
<span class="emphasis"><em>testsuite_abi_check.cc</em></span>
</p><p>
Creates the executable <span class="emphasis"><em>abi_check</em></span>.
Used to check correctness of symbol versioning, visibility of
exported symbols, and compatibility on symbols in the shared
library, for hosts that support this feature. More information
can be found in the ABI documentation <a class="link" href="abi.html" title="ABI Policy and Guidelines">here</a>
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsuite_allocator.h</em></span>,
<span class="emphasis"><em>testsuite_allocator.cc</em></span>
</p><p>
Contains specialized allocators that keep track of construction
and destruction. Also, support for overriding global new and
delete operators, including verification that new and delete
are called during execution, and that allocation over max_size
fails.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsuite_character.h</em></span>
</p><p>
Contains <code class="code">std::char_traits</code> and
<code class="code">std::codecvt</code> specializations for a user-defined
POD.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsuite_hooks.h</em></span>,
<span class="emphasis"><em>testsuite_hooks.cc</em></span>
</p><p>
A large number of utilities, including:
</p><div class="itemizedlist"><ul class="itemizedlist"><li class="listitem"><p>VERIFY</p></li><li class="listitem"><p>set_memory_limits</p></li><li class="listitem"><p>verify_demangle</p></li><li class="listitem"><p>run_tests_wrapped_locale</p></li><li class="listitem"><p>run_tests_wrapped_env</p></li><li class="listitem"><p>try_named_locale</p></li><li class="listitem"><p>try_mkfifo</p></li><li class="listitem"><p>func_callback</p></li><li class="listitem"><p>counter</p></li><li class="listitem"><p>copy_tracker</p></li><li class="listitem"><p>copy_constructor</p></li><li class="listitem"><p>assignment_operator</p></li><li class="listitem"><p>destructor</p></li><li class="listitem"><p>pod_char, pod_int and associated char_traits specializations</p></li></ul></div></li><li class="listitem"><p>
<span class="emphasis"><em>testsuite_io.h</em></span>
</p><p>
Error, exception, and constraint checking for
<code class="code">std::streambuf, std::basic_stringbuf, std::basic_filebuf</code>.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsuite_iterators.h</em></span>
</p><p>
Wrappers for various iterators.
</p></li><li class="listitem"><p>
<span class="emphasis"><em>testsuite_performance.h</em></span>
</p><p>
A number of class abstractions for performance counters, and
reporting functions including:
</p><div class="itemizedlist"><ul class="itemizedlist"><li class="listitem"><p>time_counter</p></li><li class="listitem"><p>resource_counter</p></li><li class="listitem"><p>report_performance</p></li></ul></div></li></ul></div></div></div><div class="section" title="Special Topics"><div class="titlepage"><div><div><h3 class="title"><a id="test.special"/>Special Topics</h3></div></div></div><div class="section" title="Qualifying Exception Safety Guarantees"><div class="titlepage"><div><div><h4 class="title"><a id="test.exception.safety"/>
Qualifying Exception Safety Guarantees
<a id="id498690" class="indexterm"/>
</h4></div></div></div><div class="section" title="Overview"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.overview"/>Overview</h5></div></div></div><p>
Testing is composed of running a particular test sequence,
and looking at what happens to the surrounding code when
exceptions are thrown. Each test is composed of measuring
initial state, executing a particular sequence of code under
some instrumented conditions, measuring a final state, and
then examining the differences between the two states.
</p><p>
Test sequences are composed of constructed code sequences
that exercise a particular function or member function, and
either confirm no exceptions were generated, or confirm the
consistency/coherency of the test subject in the event of a
thrown exception.
</p><p>
Random code paths can be constructed using the basic test
sequences and instrumentation as above, only combined in a
random or pseudo-random way.
</p><p> To compute the code paths that throw, test instruments
are used that throw on allocation events
(<code class="classname">__gnu_cxx::throw_allocator_random</code>
and <code class="classname">__gnu_cxx::throw_allocator_limit</code>)
and copy, assignment, comparison, increment, swap, and
various operators
(<code class="classname">__gnu_cxx::throw_type_random</code>
and <code class="classname">__gnu_cxx::throw_type_limit</code>). Looping
through a given test sequence and conditionally throwing in
all instrumented places. Then, when the test sequence
completes without an exception being thrown, assume all
potential error paths have been exercised in a sequential
manner.
</p></div><div class="section" title="Existing tests"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.status"/>
Existing tests
</h5></div></div></div><div class="itemizedlist"><ul class="itemizedlist"><li class="listitem"><p>
Ad Hoc
</p><p>
For example,
<code class="filename">testsuite/23_containers/list/modifiers/3.cc</code>.
</p></li><li class="listitem"><p>
Policy Based Data Structures
</p><p>
For example, take the test
functor <code class="classname">rand_reg_test</code> in
in <code class="filename">testsuite/ext/pb_ds/regression/tree_no_data_map_rand.cc</code>. This uses <code class="classname">container_rand_regression_test</code> in
<code class="filename">testsuite/util/regression/rand/assoc/container_rand_regression_test.h</code>.
</p><p>
Which has several tests for container member functions,
Includes control and test container objects. Configuration includes
random seed, iterations, number of distinct values, and the
probability that an exception will be thrown. Assumes instantiating
container uses an extension
allocator, <code class="classname">__gnu_cxx::throw_allocator_random</code>,
as the allocator type.
</p></li><li class="listitem"><p>
C++0x Container Requirements.
</p><p>
Coverage is currently limited to testing container
requirements for exception safety,
although <code class="classname">__gnu_cxx::throw_type</code> meets
the additional type requirements for testing numeric data
structures and instantiating algorithms.
</p><p>
Of particular interest is extending testing to algorithms and
then to parallel algorithms. Also io and locales.
</p><p>
The test instrumentation should also be extended to add
instrumentation to <code class="classname">iterator</code>
and <code class="classname">const_iterator</code> types that throw
conditionally on iterator operations.
</p></li></ul></div></div><div class="section" title="C++0x Requirements Test Sequence Descriptions"><div class="titlepage"><div><div><h5 class="title"><a id="test.exception.safety.containers"/>
C++0x Requirements Test Sequence Descriptions
</h5></div></div></div><div class="itemizedlist"><ul class="itemizedlist"><li class="listitem"><p>
Basic
</p><p>
Basic consistency on exception propagation tests. For
each container, an object of that container is constructed,
a specific member function is exercised in
a <code class="literal">try</code> block, and then any thrown
exceptions lead to error checking in the appropriate
<code class="literal">catch</code> block. The container's use of
resources is compared to the container's use prior to the
test block. Resource monitoring is limited to allocations
made through the container's <span class="type">allocator_type</span>,
which should be sufficient for container data
structures. Included in these tests are member functions
are <span class="type">iterator</span> and <span class="type">const_iterator</span>
operations, <code class="function">pop_front</code>, <code class="function">pop_back</code>, <code class="function">push_front</code>, <code class="function">push_back</code>, <code class="function">insert</code>, <code class="function">erase</code>, <code class="function">swap</code>, <code class="function">clear</code>,
and <code class="function">rehash</code>. The container in question is
instantiated with two instrumented template arguments,
with <code class="classname">__gnu_cxx::throw_allocator_limit</code>
as the allocator type, and
with <code class="classname">__gnu_cxx::throw_type_limit</code> as
the value type. This allows the test to loop through
conditional throw points.
</p><p>
The general form is demonstrated in
<code class="filename">testsuite/23_containers/list/requirements/exception/basic.cc
</code>. The instantiating test object is <code class="classname">__gnu_test::basic_safety</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>.
</p></li><li class="listitem"><p>
Generation Prohibited
</p><p>
Exception generation tests. For each container, an object of
that container is constructed and all member functions
required to not throw exceptions are exercised. Included in
these tests are member functions
are <span class="type">iterator</span> and <span class="type">const_iterator</span> operations, <code class="function">erase</code>, <code class="function">pop_front</code>, <code class="function">pop_back</code>, <code class="function">swap</code>,
and <code class="function">clear</code>. The container in question is
instantiated with two instrumented template arguments,
with <code class="classname">__gnu_cxx::throw_allocator_random</code>
as the allocator type, and
with <code class="classname">__gnu_cxx::throw_type_random</code> as
the value type. This test does not loop, an instead is sudden
death: first error fails.
</p><p>
The general form is demonstrated in
<code class="filename">testsuite/23_containers/list/requirements/exception/generation_prohibited.cc
</code>. The instantiating test object is <code class="classname">__gnu_test::generation_prohibited</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>.
</p></li><li class="listitem"><p>
Propagation Consistent
</p><p>
Container rollback on exception propagation tests. For
each container, an object of that container is constructed,
a specific member function that requires rollback to a previous
known good state is exercised in
a <code class="literal">try</code> block, and then any thrown
exceptions lead to error checking in the appropriate
<code class="literal">catch</code> block. The container is compared to
the container's last known good state using such parameters
as size, contents, and iterator references. Included in these
tests are member functions
are <code class="function">push_front</code>, <code class="function">push_back</code>, <code class="function">insert</code>,
and <code class="function">rehash</code>. The container in question is
instantiated with two instrumented template arguments,
with <code class="classname">__gnu_cxx::throw_allocator_limit</code>
as the allocator type, and
with <code class="classname">__gnu_cxx::throw_type_limit</code> as
the value type. This allows the test to loop through
conditional throw points.
</p><p>
The general form demonstrated in
<code class="filename">testsuite/23_containers/list/requirements/exception/propagation_coherent.cc
</code>. The instantiating test object is <code class="classname">__gnu_test::propagation_coherent</code> and is detailed in <code class="filename">testsuite/util/exception/safety.h</code>.
</p></li></ul></div></div></div></div></div><div class="navfooter"><hr/><table width="100%" summary="Navigation footer"><tr><td align="left"><a accesskey="p" href="internals.html">Prev</a> </td><td align="center"><a accesskey="u" href="appendix_porting.html">Up</a></td><td align="right"> <a accesskey="n" href="abi.html">Next</a></td></tr><tr><td align="left" valign="top">Porting to New Hardware or Operating Systems </td><td align="center"><a accesskey="h" href="../spine.html">Home</a></td><td align="right" valign="top"> ABI Policy and Guidelines</td></tr></table></div></body></html>
|
