File Coverage

=head1 NAME

Test::Most - Most commonly needed test functions and features.

=head1 VERSION

Version 0.33

=cut

=head1 SYNOPSIS

Instead of this:

    use strict;
    use warnings;
    use Test::Exception 0.88;
    use Test::Differences 0.500;
    use Test::Deep 0.106;
    use Test::Warn 0.11;
    use Test::More tests => 42;

You type this:

    use Test::Most tests => 42;

=head1 DESCRIPTION

L<Test::Most> exists to reduce boilerplate and to make your testing life
easier.  We provide "one stop shopping" for most commonly used testing
modules.  In fact, we often require the latest versions so that you get bug
fixes through L<Test::Most> and don't have to keep upgrading these modules
separately.

This module provides you with the most commonly used testing functions, along
with automatically turning on strict and warning and gives you a bit more
fine-grained control over your test suite.

    use Test::Most tests => 4, 'die';

    ok 1, 'Normal calls to ok() should succeed';
    is 2, 2, '... as should all passing tests';
    eq_or_diff [3], [4], '... but failing tests should die';
    ok 4, '... will never get to here';

As you can see, the C<eq_or_diff> test will fail.  Because 'die' is in the
import list, the test program will halt at that point.

If you do not want strict and warnings enabled, you must explicitly disable
them.  Thus, you must be explicit about what you want and no longer need to
worry about accidentally forgetting them.

    use Test::Most tests => 4;
    no strict;
    no warnings;

=head1 EXPORT

All functions from the following modules will automatically be exported into
your namespace:

=over 4

=item * L<Test::More>

=item * L<Test::Exception>

=item * L<Test::Differences>

=item * L<Test::Deep> 

=item * L<Test::Warn>

=back

Functions which are I<optionally> exported from any of those modules must be
referred to by their fully-qualified name:

  Test::Deep::render_stack( $var, $stack );

=head1 FUNCTIONS

Several other functions are also automatically exported:

=head2 C<die_on_fail>

 die_on_fail;
 is_deeply $foo, bar, '... we throw an exception if this fails';

This function, if called, will cause the test program to throw a
L<Test::Most::Exception>, effectively halting the test.

=head2 C<bail_on_fail>

 bail_on_fail;
 is_deeply $foo, bar, '... we bail out if this fails';

This function, if called, will cause the test suite to BAIL_OUT() if any
tests fail after it.

=head2 C<restore_fail>

 die_on_fail;
 is_deeply $foo, bar, '... we throw an exception if this fails';

 restore_fail;
 cmp_bag(\@got, \@bag, '... we will not throw an exception if this fails';

This restores the original test failure behavior, so subsequent tests will no
longer throw an exception or BAIL_OUT().

=head2 C<set_failure_handler>

If you prefer other behavior to 'die_on_fail' or 'bail_on_fail', you can
set your own failure handler:

 set_failure_handler( sub {
     my $builder = shift;
     if ( $builder && $builder->{Test_Results}[-1] =~ /critical/ ) {
        send_admin_email("critical failure in tests");
     }
 } );

It receives the C<< Test::Builder >> instance as its only argument.  

B<Important>:  Note that if the failing test is the very last test run, then
the C<$builder> will likely be undefined.  This is an unfortunate side effect
of how C<Test::Builder> has been designed.

=head2 C<explain>

Similar to C<note()>, the output will only be seen by the user by
using the C<-v> switch with C<prove> or reading the raw TAP.

Unlike C<note()>, any reference in the argument list is automatically expanded
using C<Data::Dumper>.  Thus, instead of this:

 my $self = Some::Object->new($id);
 use Data::Dumper;
 explain 'I was just created', Dumper($self);

You can now just do this:

 my $self = Some::Object->new($id);
 explain 'I was just created:  ', $self;

That output will look similar to:

 I was just created: bless( {
   'id' => 2,
   'stack' => []
 }, 'Some::Object' )

Note that the "dumpered" output has the C<Data::Dumper> variables
C<$Indent>, C<Sortkeys> and C<Terse> all set to the value of C<1> (one).  This
allows for a much cleaner diagnostic output and at the present time cannot be
overridden.

Note that Test::More's C<explain> acts differently.  This C<explain>
is equivalent to C<note explain> in Test::More.

=head2 C<show>

Experimental.  Just like C<explain>, but also tries to show you the lexical
variable names:

 my $var   = 3;
 my @array = qw/ foo bar /;
 show $var, \@array;
 __END__
 $var = 3;
 @array = [
     'foo',
     'bar'
 ];

It will show C<$VAR1>, C<$VAR2> ... C<$VAR_N> for every variable it cannot
figure out the variable name to:

 my @array = qw/ foo bar /;
 show @array;
 __END__
 $VAR1 = 'foo';
 $VAR2 = 'bar';

Note that this relies on L<Data::Dumper::Names> version 0.03 or greater.  If
this is not present, it will warn and call L<explain> instead.  Also, it can
only show the names for lexical variables.  Globals such as C<%ENV> or C<%@>
are not accessed via PadWalker and thus cannot be shown.  It would be nice to
find a workaround for this.

=head2 C<always_explain> and C<always_show>

These are identical to C<explain> and C<show>, but like L<Test::More>'s
C<diag> function, these will always emit output, regardless of whether or not
you're in verbose mode.

=head2 C<all_done>

B<DEPRECATED>.  Use the new C<done_testing()> (added in
L<Test::More|Test::More> since 0.87_01).  Instead. We're leaving this in here
for a long deprecation cycle.  After a while, we might even start warning.

If the plan is specified as C<defer_plan>, you may call C<&all_done> at the
end of the test with an optional test number.  This lets you set the plan
without knowing the plan before you run the tests.

If you call it without a test number, the tests will still fail if you don't
get to the end of the test.  This is useful if you don't want to specify a
plan but the tests exit unexpectedly.  For example, the following would
I<pass> with C<no_plan> but fails with C<all_done>.

 use Test::More 'defer_plan';
 ok 1;
 exit;
 ok 2;
 all_done;

See L<Deferred plans> for more information.

=head1 EXPORT ON DEMAND

The following will be exported only if requested:

=head2 C<timeit>

Prototype: C<timeit(&;$)>

This function will warn if C<Time::HiRes> is not installed. The test will
still be run, but no timing information will be displayed.

 use Test::Most 'timeit';
 timeit { is expensive_function(), $some_value, $message }
    "expensive_function()";
 timeit { is expensive_function(), $some_value, $message };

C<timeit> accepts a code reference and an optional message. After the test is
run, will C<explain> the time of the function using C<Time::HiRes>. If a
message is supplied, it will be formatted as:

  sprintf "$message: took %s seconds" => $time;

Otherwise, it will be formatted as:

  sprintf "$filename line $line: took %s seconds" => $time;

=head1 DIE OR BAIL ON FAIL

Sometimes you want your test suite to throw an exception or BAIL_OUT() if a
test fails.  In order to provide maximum flexibility, there are three ways to
accomplish each of these.

=head2 Import list

 use Test::Most 'die', tests => 7;
 use Test::Most qw< no_plan bail >;

If C<die> or C<bail> is anywhere in the import list, the test program/suite
will throw a C<Test::Most::Exception> or C<BAIL_OUT()> as appropriate the
first time a test fails.  Calling C<restore_fail> anywhere in the test program
will restore the original behavior (not throwing an exception or bailing out).

=head2 Functions

 use Test::Most 'no_plan';
 ok $bar, 'The test suite will continue if this passes';

 die_on_fail;
 is_deeply $foo, bar, '... we throw an exception if this fails';

 restore_fail;
 ok $baz, 'The test suite will continue if this passes';

The C<die_on_fail> and C<bail_on_fail> functions will automatically set the
desired behavior at runtime.

=head2 Environment variables

 DIE_ON_FAIL=1 prove t/
 BAIL_ON_FAIL=1 prove t/

If the C<DIE_ON_FAIL> or C<BAIL_ON_FAIL> environment variables are true, any
tests which use C<Test::Most> will throw an exception or call BAIL_OUT on test
failure.

=head1 MISCELLANEOUS

=head2 Moose

It used to be that this module would produce a warning when used with Moose:

    Prototype mismatch: sub main::blessed ($) vs none

This was because L<Test::Deep> exported a C<blessed()> function by default,
but its prototype did not match the L<Moose> version's prototype. We now
exclude the L<Test::Deep> version by default. If you need it, you can call the
fully-qualified version or request it on the command line:

    use Test::Most 'blessed';

=head2 Excluding Test Modules

Sometimes you want a exclude a particular test module.  For example,
L<Test::Deep>, when used with L<Moose>, produces the following warning:

    Prototype mismatch: sub main::blessed ($) vs none

You can exclude this with by adding the module to the import list with a '-'
symbol in front:

    use Test::Most tests => 42, '-Test::Deep';

See
L<https://rt.cpan.org/Ticket/Display.html?id=54362&results=e73ff63c5bf9ba0f796efdba5773cf3f>
for more information.

=head2 Excluding Test Symbols

Sometimes you don't want to exclude an entire test module, but just a
particular symbol that is causing issues (e.g. see the 'blessed' example
above). You can exclude the symbol(s) in the standard way, by specifying the
symbol in the import list with a '!' in front:

    use Test::Most tests => 42, '!blessed';

=head2 Deferred plans

B<DEPRECATED>.  Use C<done_testing()> from L<Test::More> instead.

 use Test::Most qw<defer_plan>;
 use My::Tests;
 my $test_count = My::Tests->run;
 all_done($test_count);

Sometimes it's difficult to know the plan up front, but you can calculate the
plan as your tests run.  As a result, you want to defer the plan until the end
of the test.  Typically, the best you can do is this:

 use Test::More 'no_plan';
 use My::Tests;
 My::Tests->run;

But when you do that, C<Test::Builder> merely asserts that the number of tests
you I<ran> is the number of tests.  Until now, there was no way of asserting
that the number of tests you I<expected> is the number of tests unless you do
so before any tests have run.  This fixes that problem.

=head2 One-stop shopping

We generally require the latest stable versions of various test modules.  Why?
Because they have bug fixes and new features.  You don't want to have to keep
remembering them, so periodically we'll release new versions of L<Test::Most>
just for bug fixes.

=head2 C<use ok>

We do not bundle L<Test::use::ok>, though it's been requested.  That's because
C<use_ok> is broken, but L<Test::use::ok> is also subtly broken (and a touch
harder to fix).  See L<http://use.perl.org/~Ovid/journal/39859> for more
information.

If you want to test if you can use a module, just use it.  If it fails, the
test will still fail and that's the desired result.

=head1 RATIONALE

People want more control over their test suites.  Sometimes when you see
hundreds of tests failing and whizzing by, you want the test suite to simply
halt on the first failure.  This module gives you that control.

As for the reasons for the four test modules chosen, I ran code over a local
copy of the CPAN to find the most commonly used testing modules.  Here's the
top twenty as of January 2010 (the numbers are different because we're now
counting distributions which use a given module rather than simply the number
of times a module is used).

    1   Test::More                          14111
    2   Test                                 1736
    3   Test::Exception                       744
    4   Test::Simple                          331
    5   Test::Pod                             328
    6   Test::Pod::Coverage                   274
    7   Test::Perl::Critic                    248
    8   Test::Base                            228
    9   Test::NoWarnings                      155
    10  Test::Distribution                    142
    11  Test::Kwalitee                        138
    12  Test::Deep                            128
    13  Test::Warn                            127
    14  Test::Differences                     102
    15  Test::Spelling                        101
    16  Test::MockObject                       87
    17  Test::Builder::Tester                  84
    18  Test::WWW::Mechanize::Catalyst         79
    19  Test::UseAllModules                    63
    20  Test::YAML::Meta                       61

L<Test::Most> is number 24 on that list, if you're curious.  See
L<http://blogs.perl.org/users/ovid/2010/01/most-popular-testing-modules---january-2010.html>.

The modules chosen seemed the best fit for what C<Test::Most> is trying to do.
As of 0.02, we've added L<Test::Warn> by request.  It's not in the top ten, but
it's a great and useful module.

=cut

=head1 AUTHOR

Curtis Poe, C<< <ovid at cpan.org> >>

=head1 BUGS

Please report any bugs or feature requests to C<bug-test-extended at
rt.cpan.org>, or through the web interface at
L<http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Test-Most>.  I will be
notified, and then you'll automatically be notified of progress on your bug as
I make changes.

=head1 SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc Test::Most

You can also look for information at:

=over 4

=item * RT: CPAN's request tracker

L<http://rt.cpan.org/NoAuth/Bugs.html?Dist=Test-Most>

=item * AnnoCPAN: Annotated CPAN documentation

L<http://annocpan.org/dist/Test-Most>

=item * CPAN Ratings

L<http://cpanratings.perl.org/d/Test-Most>

=item * Search CPAN

L<http://search.cpan.org/dist/Test-Most>

=back

=head1 TODO

=head2 Deferred plans

Sometimes you don't know the number of tests you will run when you use
C<Test::More>.  The C<plan()> function allows you to delay specifying the
plan, but you must still call it before the tests are run.  This is an error:

 use Test::More;

 my $tests = 0;
 foreach my $test (
     my $count = run($test); # assumes tests are being run
     $tests += $count;
 }
 plan($tests);

The way around this is typically to use 'no_plan' and when the tests are done,
C<Test::Builder> merely sets the plan to the number of tests run.  We'd like
for the programmer to specify this number instead of letting C<Test::Builder>
do it.  However, C<Test::Builder> internals are a bit difficult to work with,
so we're delaying this feature.

=head2 Cleaner skip()

 if ( $some_condition ) {
     skip $message, $num_tests;
 }
 else {
     # run those tests
 }

That would be cleaner and I might add it if enough people want it.

=head1 CAVEATS

Because of how Perl handles arguments, and because diagnostics are not really
part of the Test Anything Protocol, what actually happens internally is that
we note that a test has failed and we throw an exception or bail out as soon
as the I<next> test is called (but before it runs).  This means that its
arguments are automatically evaluated before we can take action:

 use Test::Most qw<no_plan die>;

 ok $foo, 'Die if this fails';
 ok factorial(123456),
   '... but wait a loooong time before you throw an exception';

=head1 ACKNOWLEDGEMENTS

Many thanks to C<perl-qa> for arguing about this so much that I just went
ahead and did it :)

Thanks to Aristotle for suggesting a better way to die or bailout.

Thanks to 'swillert' (L<http://use.perl.org/~swillert/>) for suggesting a
better implementation of my "dumper explain" idea
(L<http://use.perl.org/~Ovid/journal/37004>).

=head1 COPYRIGHT & LICENSE

Copyright 2008 Curtis Poe, all rights reserved.

This program is free software; you can redistribute it and/or modify it
under the same terms as Perl itself.


=cut