The Firebird Project will soon be releasing the first public "alpha"
release of Firebird 2.0. Version 2.0 is a long-awaited important major
release of Firebird with many new features, enhancements and
bugfixes (see alpha Release Notes
for details). In number of changes, the jump in this release is equivalent
if not greater than the transition from version 1.0 to version 1.5.
You know that we care about quality, and that we will not release the
final version 2.0 until it meets our strict QA requirements. For version
1.5, it took about a year before we were satisfied. But this time we are
in a slightly different situation.
The Vulcan project reached the *general usability milestone*, with only
a few loose ends left, and we would like to merge both codebases as
soon as possible. This merge should result in Firebird 3.0 with full
SMP support, unified architecture (no more separate
classic/superserver/embedded builds) and other enhancements (see
http://www.ibphoenix.com/downloads/VulcanOverview.pdf for details).
Beside clear benefits for Firebird users, this merge will result in
cleaner and concise architecture and codebase, and will complete our
transition from old procedural C code to fully OOP C++ code. This will
open gates for developers to design and implement more complex features
like namespaces, temporary tables and other much requested features.
But we can't fully focus on the Firebird/Vulcan merge before the final
Firebird 2.0 is released, hence we would like to shorten the QA phase
as much as possible, but without compromising our strict quality
requirements for final release. We *can* do that by making our QA
process more effective. The effectiveness of the Firebird QA process
heavily depends on feedback from end users, so it's natural for our
quest for more effective QA to start with it.
So far, user feedback was random and fully in the hands of end users.
Basically, we would release a build, wait for feedback for some time
and solve the reported issues (along with other issues we did find
internally over that period). If no important issues were
found/reported since the last Release Candidate build, that build was
repackaged as final. Of course, there are also alpha and beta stages
that follow this pattern too, but differ in what developers are
allowed to change in the ncode.
While this routine has worked nicely for us in the past, it has an
important drawback: We don't know how much the build is tested in
field, in both scale and functionality coverage. We can only guess
approximate figures based on download count, direct feedback, hearsay,
development stage etc. to estimate the "quality index". We also have
only one gauge to work with: time, hence the long release cycle.
To improve on that, we would like to initiate a managed field-tests
program, starting from Firebird 2.0. This managed field-test *will
not* replace the *usual* field-test scenario (or internal testing),
but should work as a complement to other QA routines we use. The
objective of managed field-test is to collect precise information
about field-test (i.e. how the released build was tested and with what
outcome), so we can better estimate the outcome of the QA process,
focus on open gaps in QA and thus allocate our QA resources more
precisely, so eventually we would build our trust in quality of code
more quickly.
The participation in managed field-test is very simple. You need to
sign-in by e-mail to pcisar , where you'll describe
how you would like to test our releases. We prefer any testing method
that is close to real use. This means that if you have an application
that runs with Firebird, you can simply take it on a "test drive" with
the new Firebird release in some testing environment, preferably with
real-world data. Of course, you can pick up any testing method you
like, and you can even focus only on a particular area you're most
interested in (for example performance, backup/restore, new features,
optimizer etc.). You must also describe what Firebird flavour(s)
(CS/SS/Embedded) and platform(s) you want to test. We will send you a
notification whenever a new filed-test build is available, and we will
expect a report from you about the outcome (either good or bad) of
your tests.
We know that such commitment may not be easy to fulfil, so it's
possible that you may skip testing of the field-test release or leave
the program altogether at any point, so we will reward those who help
us! We have created a "prize pool" that right now contains a Firebird
T/polo-shirt in color and shape of your choice from IBPhoenix, but we
believe that we'll get more prizes into this pool before Firebird 2.0
final release. At the end of the release cycle, we will reward the
most "active" tester(s) and one randomly selected tester.
The managed field-test program is open to anyone, at any time point in
the release cycle (starting from fist alpha till last RC), but those who
sign-in early will have better chance to get a reward for their help.
Pavel Cisar
Firebird Project QA Division
No comments:
Post a Comment