Project Title:

 A First Look at the NetBeans Requirements and Release Process

Team Members:

© Maulik Oza	moza@ics.uci.edu	http://www.ics.uci.edu/~moza
© Eugen Nistor	enistor@ics.uci.edu	http://www.ics.uci.edu/~enistor
© Susan Hu	xhu@uci.edu	http://www.ics.uci.edu/~xhu
© Chris Jensen	cjensen@ics.uci.edu	http://www.ics.uci.edu/~cjensen
© Walt Scacchi (instructor)	wscacchi@ics.uci.edu	http://www.ics.uci.edu/~wscacchi

Abstract:

The way in which open source communities develop software is fundamentally different than their corporate counterparts. Though Open Source Software Development (OSSD) projects still require some semblance of system requirements acquisition and the release process, the extent of explicit definition of these needs varies widely from community to community. Our investigation focuses on the analyzing the maturity of this process in the NetBeans Open Source Community. We explore how new functionality (and non-functionality) proposals are formulated, negotiated, and adopted into NetBeans IDE and Platform development, and how these ideas are transformed into a release. In the course of this investigation, we determine modes of communication and collaboration in devising and revising module proposals, release proposals, the factors affecting its acceptance into the community development scheme, and how developers join in contributing to a module's development. The goal of this project is to provide a better understanding of the aspects involved in the open source community development processes that have allowed them to persist competitively with closed source development and construct a language with which to describe this process.  

Introduction and Overview:

We began with a simple question: how do open source software communities determine the feature set of their software. To answer this, we turned to the NetBeans open source community. There have been some studies that have been previously done on open source projects but have focused on high level overview of the open source development process, this study tries to provide an in-depth analysis of the open source community with respect to the requirements and the release process.

NetBeans is an open source community that is backed by Sun Microsystems [29], for the development of the NetBeans IDE as well as the NetBeans Platform. With three of being open source the NetBeans community has been able to meet the requirements of the individuals that use the IDE and better satisfy the customers with their involvement in the project. NetBeans is available for use under the Sun Public License, which complies with the open source definition [28].

Research in the field of software processes for open source projects could lead to a better understanding of the process involved in managing large not collocated teams working to meet a particular deadline. This could be useful in coming up with a model of software development process that could be useful for other projects in the future in any organization. On the other hand this would also serve as a model for other open source communities by being a successful model for software development. Work has been done already in the field of understanding how software development can be supported in virtual enterprises [30], but it does not highlight how people not caring about organizations boundaries develop software. With a paradigm shift in the software engineering community, where there is a focus on component-oriented software engineering, it has become essential to study such a model that could be able to assist in such a scenario.

One other purpose of the study was to give a comparative idea with respect to the current best practices in the so-called cathedral style [31] of software development. We wanted to look at the similarities and differences between the tools and resources that are being used for development. A typical example would be the tools used for nightly builds, or the builds used by developers for tasks such as bug fixes etc. that is generally not found in a conventional software development environment. Some other issues included the conventional artifacts of software engineering such as design documents, requirement specifications that typically do not exist in an open source scenario.

So for the purpose of the study, we looked at what roles are being played in the community, its organizational makeup, artifacts of development, such as documentation, code, and feedback, the steps in the process, and the tools they use to complete them. In this paper, we will present our findings. We will set the stage with a detailed description of the community as it relates to the requirements and release process. We will then present the process model in terms of the aforementioned roles, tools, and artifacts associated with them, as well as visualization of this process. Having modeled the process, we offer a semi-formal, structured view of the process, and a sequence of use-cases to interrelate the tools, resources, and actions of the process. We conclude with a prototype and by casting our findings into a formal description of the process.

Problem Domain Characterization:

Although many believe that open source software development is a largely unstructured and unconventional activity conducted by self described "hackers" working at obscene hours of the night, such is not the case with the NetBeans community. On the contrary, with well-defined -and adhered to- community regulations, the maturity of the NetBeans community is enough to rival many of its corporate counterparts. The challenge in analyzing NetBeans is not in the lack of documentation of the process, but more in locating and organizing the plethora of information available, much of which is catalogued within the dense email archives, as opposed to the main web pages.

The NetBeans open source project was started by SUN Microsystems in June 2000, which still pays the salaries of most of the developers of the project. The NetBeans project started with SUN contributing the code base from Forte for Java Community Edition to the community. The current NetBeans code base is used by SUN to develop Forte for Java, which is a commercial product. The NetBeans project is basically sub-divided into two parts - the NetBeans IDE and the NetBeans platform. The platform along with the IDE modules makes up the IDE. Various feature of the IDE make NetBeans a very powerful tool for developing projects. NetBeans as described above is very different from the notion of people about open source software. It is a very well documented project with processes defined for virtually every process being followed. It more resembles the cathedral style [31] of software development but without the small communities and the closeness. It follows a specified timeline for each and every release; it has coordinators (not managers) for various tasks. It has a QA team that is part of SUN Microsystems, which is the main sponsor of the project. The NetBeans IDE has support for various languages including Java, HTML, XML, C, C++, as well as support for JSP, CORBA, RMI, JINI, JDBC and Servlet technologies. It has also support for Ant, CVS and other version control tools. The Platform on the other hand has services for User Interface Management, Configuration management, Storage Management along with many others and thus the project as a whole really benefits developers and encourages them to contribute (source or testing by downloading the IDE). The view of the project can be shown as in Figure 1.

Figure 1 - Big Picture

 

The development process at Netbeans is closely watched by SUN Microsystems. Forte for Java represents an important part in the business plans at Sun, the strategic decisions of the Netbeans development process are decided by SunÆs employees. Such decisions include which features the new releases of Netbeans must include, as well as which features it is not supposed to have, such that it will not offer exactly the same functionality as its commercial distribution. One of the three members of the board is appointed by Sun Microsystems. There is a QA team from Sun Microsystems that checks once per week the latest development build in order to maintain the number of bugs at a reasonable level. Also, Netbeans is licensed under the Sun Public License.

In this paper we discuss the NetBeans requirements and the release process, which described in a few words means the process by which the community decides what features have to be included in a particular release and the actual process of releasing the product. Hence, this paper is concerned with the process of turning ideas into reality for an open source project. In this section we provide the classical software engineering view of the software process that we will be presenting for an open source project. This would include a view of the requirements acquisition process as it is normally referred to as in the classical software engineering discipline, as well as the Deployment or the release process for a cathedral kind of development style.

A typical requirements engineering process for a high quality, reliable product consists of the following phases as mentioned by experts [40,41,42],

Eliciting requirements, whereby engineers identify all the stakeholders, their goals, needs, expectations and limitations if any. This process generally consists of conducting interviews, gathering data through questionnaires, documentation review, and focus groups.

The next step in the process of requirements engineering includes, requirements specification. In this phase, there is an attempt to specify the functional as well as non-functional requirements for the system. The functional requirements are captured in the form of state transition diagrams by identifying states, events - input/output and data. The data specification may include data sources, data objects, user displays etc. Functional requirements also merit the depiction of the flow of data in the system. The non-functional requirements on the other hand specify the constraints on the system. It might also include specifying certain goals or capabilities for the system.

After the requirements have been specified, there is a need for analysis and validation of the requirements. This involves the use of techniques for sanity checking critics, whereby the requirements are tested for reach ability, deadlocks etc. This might also be done by running a simulation of the requirements specified using a formal model. Validating the requirements is a similar process to analyzing them, but involves checking for the feasibility of the specified system. It could involve views from technical feasibility to usability of the system as analyzed by experts.

Requirements freeze is the next logical step after validation. In this step the requirements document is frozen and called the requirements specification document and is used in the future for traceability of the system to the requirements.

The release process on the other hand has incremental steps such as alpha, beta and final releases. The alpha release is generally meant for testing by the users in the domain of the developers. This generally is not a "stable" release. After identifying bugs and fixing the same, a beta release is released that is meant to be tested by the customer at his/her site. After the beta release has been stabilized a final release version of the system is shipped to the customer. The system may be packaged based on the requirements specified initially in the requirements engineering phase. Thus the release process is generally a small phase. If the product is some kind of incremental, then the releases are scheduled to be not-so-frequent, i.e. a schedule with different new features, bug fixes is decided and is generally over a period of a few years.

With a view of the classical software engineering view of the requirements and the release process, we can now have a look at the open source processes as related to this particular project (NetBeans) and look at the similarities as well as the differences. This will just stand as groundwork for comparison of the models of classical software engineering with the practice followed in the open source universe.

The willingness to contribute to the development of a robust application is the driving force behind any OSSD process. Unlike in conventional software development scenarios, where programmers sometimes do things that they don't like, because they need to obey their boss's decision or they don't want to express their disproval, in open-source projects developers only contribute parts that they really enjoy doing. The contributors are interested in finding bugs and in writing good bug fixes. They are interested in proposing new enhancements that they want to implement. There are times where there is no one assigned for a certain task, but always at least one contributor volunteers for the job. An important part of the contributions to the open-source community is done by volunteer unpaid work. These contributors have various reasons for working in such projects for free: they are either not interested in the material part of the project and contribute code as a free-time hobby, they want to show this work into their resumes as proof of being capable to work to real projects, or they have interest in promoting a certain subcomponent as a part of the project.

The backbone of the OSSD process is the communication support for the community members. Mailing lists and newsgroups are the most utilized tools. These communication tools make possible collaboration between community members located in various global geographic sites. All the important decisions are taken in a democratic process, where everybody gives their comments. There is a special concern in the development process that every decision is taken by a general agreement. While this is good because it promotes developing of good features through criticism, long discussions can make the development process hard to measure and control. Therefore, there are guidelines about the release dates, about how long a release should be tested, about how many bugs a release must have, but in the end the overall process does not pay much attention to them. Almost every open-source process has not met the proposed deadlines, and Netbeans makes no exceptions.

An important part of the contributions to the open-source community is done by volunteer unpaid work. These contributors have various reasons for working in such projects for free: they are either not interested in the material part of the project and contribute code as a free-time hobby, they want to show this work into their resumes as proof of being capable to work to real projects, or they have interest in promoting a certain subcomponent as a part of the project.

Process Modeling and Visualization:

Software production architecture for NetBeans

Agent Roles

The Netbeans community has a flat structure of 9 roles. Their responsibilities and rights define these roles, and one person can carry one or multiple roles. The basic roles are defined in the NetBeans Governance [2] document, which states five of them. The other four where either considered not relevant from the open-source developer point of view, or were implied. They were identified while closely examining the email archives or while trying to detail the development process. The list of the roles and their responsibilities is given here.

User. The users are the people that use the Netbeans IDE. They are encouraged to download only the release builds from the web server, with a minimal number of bugs. The users use the Netbeans IDE to write Java code or build a new application on the Netbeans platform. The users are encouraged to report any bug and to propose new features or suggestions to the Netbeans community through the use of Issuezilla.

Developer. The developers are individuals that submit patches, bug fixes and module code, but do not have CVS code check-in access. The way that they can contribute to the project is by writing emails to the module mailing list with their proposals and with the source code attached, or to attach the source code to the bug report in Issuezilla. Somebody else will revise the work and decide including the source code in the project.

Contributor. These are special developers that have CVS code check-in access. The decision to upgrade someone from the developer to the contributor status is taken by the module manager, which controls the access rights to everything related to his module. The developer can ask the module manager for new access rights to CVS.

Module manager. All problems related to a module are taken care by the module manager. In order to become a module manager, a developer can volunteer to begin development of a certain new module or to take over the control of an old module that is left by the old manager. At the moment when it is decided that a new module should be added to Netbeans, a new website extension and a new CVS branch are created and given in care to the new manager. The module manager must coordinate all the activities related to his module and serve as a moderator for any issues arising during the module development. He must negotiate with the release managers which features must be included into the module for the next release. After the module plan is done, he must ensure that the module is actually going to have all the required features. He must coordinate the work of all the developers for the module; merge their patches and bug fixes into the project. He can give permissions to certain developers to commit code into the CVS by changing their status to contributors. The integrity of the source code is checked by examining if the bug fixes actually correct the flows. The module managers are also responsible for building the module into the development builds and integrating it within the whole Netbeans builds. Each module has a web site at the http://<module>.netbeans.org/ address. The content of the site is entirely the responsibility of the module manager, and it must include descriptions of the module, links to the builds and important announcements. The developers may send proposals for new or updated content of the web site, but the manager must decide which and how these contributions will be added.

CVS manager. One or more CVS managers control the CVS repository and the global access rights. They check the integrity of the CVS repository and grant or deny rights or create accounts for authorized users.

Release manager. A release manager coordinates each release phase. The release manager can be assigned before the start of the release process, or can be chosen from a list of module managers that volunteer themselves for the job [18]. The release process follows the guidelines from the Netbeans roadmap, but all the decisions are taken by general agreement and overviewed by the release manager. The release manager must submit an initial release proposed plan to the newsgroup and email thread, and after that acknowledge any feedback from the users and module managers. This plan contains only general features, and the module managers with the agreement of the release manager decide the actual module feature plan. The development phase is in control of the module managers, but the release managers constantly coordinate their work to ensure the release stays on schedule. After all the requested features have been implemented, the release manager decides the start of the feature freeze stage of the development process, in which only bug fixes can be contributed to the existing implementation. Based on heuristics about the duration of the feature freeze phase or the number of bugs remaining to be fixed, the release manager decides when the current development build becomes the release candidate 1, the release candidate 2 or the final release. For each of these releases, a new branch in CVS must be made, and the builds must be moved and named accordingly. Also, the release manager must keep everybody informed about the state of the release, by posting announcements to the web site or the mailing list, such as the number of bugs to be fixed or the priority of the bugs. He must ensure that there is someone in charge for taking care of every bug in the list. However, the release manager cannot take decisions by his own, without waiting for feedback. After each release, a post-mortem discussion on what and how has been done must be initiated by the release manager.

Board member. The Board is composed of three individuals and has the role of ensuring the fairness throughout the community by solving any dispute. Two of the members are voted by the community, and one is appointed by Sun Microsystems. They are appointed for a period of six months [12].

QA team member. The members of the QA team are Sun Microsystems employees.  They check periodically the development builds (usually every Friday of the week) and decide if a Q-build can be released. The Q-build is an intermediary build with a reasonable number of bugs, which can be used by the developers and users. The QA team performs sanity checks, and both manual and automated tests. The results of the tests are summarized into specific reports and have three types of bugs: critical bugs, highly visible bugs and minor issues. If during the test a critical bug appears, the development build will not be promoted to a Q-build. The whole list of bugs is added in Issuezilla and taken care by the developers and module managers.

Site administrator. This is the person that takes care about the Netbeans site, decides the content of the site and creates new branches for the module managers. The site administrators participate in the last part of the release process, by posting the links for the releases to the web site.

Tools and Network Infrastructure (http://www.netbeans.org/docs/DomainFAQ.html)

Each individual module in NetBeans project is hosted on SourceCast web portal. SourceCast by CollabNet is dedicated web space for registered developers to contribute to the module/project of their interest. They can participate with other like-minded developers who share the idea of open IDE. They can also invite other developers to join. As a strong integration, SourceCast provides all the web-based tools one needs to manage his/her module and collaborate with other developers. The website is also indexed and searchable.

SourceCast provides simple, powerful web interfaces for version control, issue tracking, discussions and decision-making, automated build, automated testing, module management and knowledge management. Module owners can configure these tools within each module and grant roles to module participants to fit the needs of the project. We've identified the tools being used in NetBeans Requirement/Release Process in three categories û communications, configuration management and code development.

Communications tools

Mailing list management and archiving

Two open-source mail tools -- anzu and eyebrowse -- provide powerful, scalable functionality to administer, moderate, customize, and archive module mailing lists. The mailing lists are a primary medium of communication for developers and other interested parties to communicate with one another about aspects of a project, and to receive module announcements. Mail archive is maintained of all the mail sent to the lists and is indexed. Users may also post feedback such as bugs found and feature request. Others like release manager/module maintainers/developers can coordinate with each other and settle on important dates, issues and inform each other of such. Project/component mailing list archiving is handled by EyeBrowse. This is an open source tool that stores and indexes list messages, providing multiple options for browsing, reading, and searching mailing list archives via a web browser. Beyond storing messages, EyeBrowse adds a database and text-indexing engine to handle rapid message search and retrieval.

Tunneling with SSH

SSH is a flexible and more secure replacement for telnet and rlogin. It is widely used in development modules to provide access control and data-transport security. SSH can be used to create an unobtrusive, transparent "port tunnel" to the CVS (concurrent versions system) server. Data sent through the tunnel is encrypted, but the process is invisible to you or to the client software you are using to access the CVS repository. Thus it is recommend for developers accessing the CVS repository. Accessing the CVS repository without SSH runs the very real risk of having a third-party thief snoop your CVS password.

CVSWeb source code browsing

CVSWeb is a browser-based interface for CVS repositories. One can browse the file hierarchy by selecting directories. When clicking on a file, a view the revision history for that file will be provided. Selecting a revision number downloads that revision of the file. Each revision includes a link to display color-coded differences between that revision and the previous one.

IssueZilla issue tracking

IssueZilla is a customizable open-source tool for reporting, tracking, and managing all module tasks, defects (a.k.a. "bugs"), features, enhancements, and patches. It is a variant of BugZilla -- a bug tracking system you may already be familiar with. Each hosted module on this site has it's own, unique issues database. All registered users can query, view, report issues using the tool. Module members have additional permissions within IssueZilla to modify and reassign issues, as well as generate reports. Module members are also notified automatically by email whenever issues are assigned to them or there is activity on issues affecting their work. Developers may wish to enter the tasks they plan to work on as issues. This helps them devise and follow a work plan. Because everyone is aware of each module member's area of work, you can avoid duplicating efforts and even help out or offer feedback to other module members.

Configuration management tools

CVS source code version control

The Concurrent Versions System (CVS) is a widely used version control tool for tracking all modifications to module source code files. Developers can work on the same files and coordinate their contributions through a common repository. Multiple developers can edit their own working copies of files simultaneously, then deals with combining all the changes and notifying developers when there are conflicts. The CVS repository is maintained on a web server; clients run on users' machines and connect to the server via the Internet. Clients are available for nearly all platforms including: Unix, Windows, Macintosh, and any Java-based platform. Developers who are module members can obtain and modify module files from virtually anywhere. In the case of a Windows-based client, in order to add a new project to the CVS repository, one may choose to use CygWin under Windows.

Code development tool

Ant

The NetBeans IDE developers are encouraged to make a build with Ant, a build framework written in Java and driven by XML build scripts. In the case a new build is not necessary, just add patches to an existing IDE installation.

The above are hosted tools featured on NetBeans website. Client-side tools one needs, at minimum, are a web browser, email client, CVS, and optionally SSH to secure CVS connections. There may be additional domain specific tools developers need depending upon the types of development module they wish to contribute to.  

Artifacts

NetBeans developers aim to make frequent releases û the current cycle is approximately every 9 months (as of 2002-4-17). This means a major, stabilized and thoroughly QA'ed release. Using this approximate release schedule, and given that version 3.3 is released December 2001, NetBeans 3.4 is proposed to release around August 2002. Here is an overview of NetBeans Build Process. It's stated in NetBeans release process that the goal is two releases each year. In one release there may contain significant rewrite and/or addition of functionality while in the next a lot of stabilization work or incremental improvement may be done. According to the roadmap of NetBeans project, exact dates must be set for each release.

 

  Figure 2 - The NetBeans Roadmap

The release schedule/preliminary plan consists of five phases - feature freeze, stabilization phase, release candidate1, release candidate 2 and final release.

Each module web page is hosted constantly on the NetBeans site. The source code is in the CVS repository. After discussion among the community at large the modules/features to be included in the new release, the bugs to be fixed are agreed upon. A detailed project plan within the scope of a module will be created. After feature freeze, no more features will be added, only bug fixes are allowed to be checked in the CVS.

Night build from CVS trunk is run on every business day at 01:00AM GMT. Night build from release branches are run at 02:00AM GMT. Continuous build is run continuously, but is stopped during night build run. Main difference between the builds is that night build is published on website and is publicly available to all users. Continuous build serves developers to assure the source base is not broken. Also night build is available to all users as compiled build or source package and extra third party binaries, whereas continuous build is not publicly available, yet logs from build are available through mailing list.

A strong QA team exists in the high resistance process (bug justification/approval & code review) before a stable release can be produced. They aim at assuring the level of quality of the development builds of NetBeans. This process is carried out by the part of the Sun Microsystems' QA team responsible for the open-source modules in Forte for Java. Q-build is a build with an assured level of quality, which was a subject to QA testing. Developers and users are also expected to use it for their production work and expected to report issues not discovered by the QA tests. All bugs/issues are filed using IssueZilla. A sample query page (data form) on IssueZilla is shown here. Graphical representations are also created showing the quality of the development builds that lead to a stable distribution.

Here is a weighted defects/enhancements chart (bugs are weighted according to their priorities)

Figure 3 - Defect Rates

Here is a weekly delta chart

Figure 4 - Delta Graphs

The purpose of having a QA test weekly is to get the defects discovered and fixed early in the development cycle. Every Friday a dedicated group of QA engineers spends some time testing the latest nightly development build of NetBeans. The build is called a Q-build candidate. The QA group performs some sanity checks on the build and executes a selected set of manual tests on multiple platforms. In addition, automated tests are run on the build.

The test results are categorized into three groups:

1. Critical bugs, showstoppers, data losses (typically P1 bugs)

2. Highly visible bugs, malfunctions, major annoyances (typically P2s affecting the user)

3. Minor issues, minor annoyances and usability problems (P2s with low impact on the user, P3s and lower)

Issues from the first category are called Q-stoppers. The candidate build cannot be promoted until all Q-stoppers are fixed. Issues from the second category do not prevent the build from being promoted, but need to be addressed soon. If those bugs are not fixed within a single week, they automatically become Q-stoppers for the next candidate. QA produces a report with lists of bugs in both categories.

If there are no bugs in the first category, the candidate is promoted as a Q-build. In case there are any Q-stopper bugs in the QA report, the release coordinator asks the responsible developers to fix the bugs on a separate CVS branch. No other changes except the Q-stopper fixes are allowed on the branch. When all fixes are available (which make take several days), a new build is created. The release coordinator makes sure that the fixes are adequately tested and if everything is fine, announces a fixed Q-build on the nbdev and nbusers mailing lists.

Release candidate 1 is the first release candidate. If no serious issues are found the last release candidate automatically becomes the final release. There must be at least one week between the last release candidate and the final release. Release candidate 2 is the second release candidate if critical bugs are found in RC1. Final release is the final, public release of a new version of NetBeans.

Processes

The process diagram shows a more intuitive graphical representation of the release process in terms of the sequence of the tasks needed to be executed. The blue rectangles show the tasks and the tools used while performing the task. The artifacts are shown in the red ovals. Each task is performed by an agent, may have some artifacts as inputs, and creates one or more artifacts as outputs. The process starts with the board determining the roadmap for the project and the vision. This serves as a reference for the start of a new release phase when the release manager for that phase establishes a schedule. The schedule has to be approved by the developers upon which the release manager formulates a formal release proposal [6] with the module to be included in the release, based on which the module maintainers develop a detailed module plan for the particular release. It can be seen from this diagram that there is one important cycle, performed during the ôfeature freezeö phase [25]. After the detailed plan of the module is done and the development process has reached a point where all the functionalities have been implemented, there follows a phase where only bug fixes are accepted. Some of the bugs are discovered by the users, some by the QA team. The release manager decides which of the bugs need to be fixed. The actual bug fixes are contributed by the community and included in the nightly builds by the module managers. The release manager based on the schedule releases the release candidate 1 by following the exact actions listed in the release checklist. After the release candidate has been released, if there are no issues found with it, then it becomes the final release, else after bug fixes a new release candidate is released, called release candidate 2. The same process is repeated again for the final release [10,11]. The site administrator is the one who actually deploys the releases on the website for the community to access.

Figure 5. The process diagram

In the next sub-section we describe the process model of NetBeans in a hierarchical manner. We start with the highest level of the process and then break it down into sub-processes, tasks, and actions.

Figure 6 - The Process Model

Figure 6 shows the process model of the NetBeans Requirements and release process. It has two sub-processes - the requirements process and the release process. The diagram also shows all the agents and the roles associated with the process. The tools used in each of the sub-processes are also shown in the figure. The sub-processes shown above are again broken down into tasks; the figure 7 shows the Requirements sub-process as broken down into tasks. The sub-tasks associated with it are, setting of the project timeline and establishing the requirements for the project. Beginning each new release cycle, the previous release manager issues a call to the developer mailing list calling for a successor, promising that the tedious job will be a learning experience, if nothing more. Inevitably someone among the veteran community members will come forth and accept the responsibility. His/her first priority will be setting a timeline for development depending on the size of the release (a major X.Y or minor X.Y.Z), s/he looks at the roadmap and project vision to choose the appropriate date upon which all new development must be completed ("Feature Freeze") and a goal date for the final release to become stable. Announcing these dates on the developer message boards s/he solicits a request for modules to be included in the upcoming release [1]. As module maintainers get wind of the timetable, they must re-assess their own development timetable. Those maintainers who believe they can complete their modules by the decided date will proclaim their participation in the release. With these responses in mind the release manager sets about drafting the release proposal, in conjunction with the roadmaps and features to be included in the release.

Figure 7 - The requirements sub-process

The release process on the other hand consists of the sub-processes of stabilization and deployment. Figure 8 depicts the release process. In the stabilization process, the major task is detecting bugs/identifying issues. After the feature freeze date has been reached, no more features or enhancements are allowed on the release. The code for the release is now stored in a new branch of CVS that is labeled release_XYbase where X.Y represents the release. The branch also has a sub-branch for the nightly builds that are used by the developers for detecting bugs as well as fixing them. Whenever a bug is detected by the developers or the user community as a whole a bug report is filed with IssueZilla, with the appropriate classification of bugs. Depending on the severity of the bugs the release manager proposes the bugs that need to be fixed. The developers based on the recommendations select the bugs and the issues are transferred to them. After the bug fixes have been done the new code has to be committed into the current release branch of the CVS. Thus based on the bug fixes, a new build script is run every night to build NetBeans. These nightly builds are made available in the nightly build branch of the current release. The developers again test these and repeat the cycle.

Figure 8 - The Release sub-process

 

Please see the Appendix for more detailed diagrams of the Process model.

Rich Picture and Visualization

The Rich Picture [38] generally depicts the primary agents involved in the process, their interrelationships, their interactions with various tools and their concerns. The rich picture shown below shows all the agents, tools and their interactions for the NetBeans Requirements and Release process. It is another way of visualizing the process; it provides a better visual representation of all the relations that exist between agents and tools in the system. The rich picture shown below shows all the roles that have been described previously and their interactions with the various tools. The tools are shown in the center with all agents surrounding it. There are two layers to the picture, the first one shows the sponsor of the project, namely SUN Microsystems and other major groups (NetBeans.org). The callouts represent the concern of the particular agents and the other arrows represent the relations among the agents and the tools. This picture is taken from the rich hypermedia that provides hyperlinks from the relations, concerns and agents to their appropriate descriptions.

 

Figure 6 - The Rich Picture

Use Cases

The use cases for the relations shown in the rich picture in figure 6 above are classified below. The use cases basically illustrate the relations between the tools, agents and the tasks performed by the agents. From the above picture the use cases are the arrows from the agents to other agents and tools.

Propose module/feature

Community members who are interested in the project come out and propose new features that they think need to be incorporated into the project to improve the functionality of the system. The people who generally propose the project are considered the leaders of that particular feature often termed as "maintainers" or "module leaders". The suggested proposals maybe just in form of an idea or a prototype showing a dummy implementation of the idea. The feature has to be first sent to the developersÆ mailing list that approves it. The concept should also be valid with respect to the current NetBeans architecture.

Create Module Web page

After proposing a particular feature, the so-called "module leaders" or "maintainers" are assigned web space on the NetBeans website for hosting their idea so that people can have a look at it and join the particular module. This website needs to be updated by the "maintainer" with any additional ideas or issues in the project

Send Release Proposal
The release manager sends the release proposal for approval on the developersÆ mailing list. The proposal is prepared in consultation with all the "maintainers" whose modules are to be included in a particular release. The proposal might need to be edited based on the feedback from the developersÆ mailing list. Once the proposal has been approved the deadlines mentioned stand.

Select feature to develop

The developers in the community can browse through the different project pages of the NetBeans project and accordingly decide on what features they wish to develop. These features could either be bug fixes that have been reported or it could be a new feature of the project. Once a developer has decided on a particular project and a feature s/he is associated with the project and has to meet the deadlines for that particular feature. For the purpose of selecting a feature it should not have been selected by any other developer. The developer if not a "contributor" does not have commit access to the CVS and has to go through a "contributor" for putting up the source code in CVS.

Download NetBeans

This particular task involves two user groups, one is the regular user of the community that is not involved in the development of the project but is just interested in using the project. They are the ones that report bugs in the project. The end users generally develop the latest "stable" release of the project. They might also be involved in using the release candidates or beta releases. These releases are generally in binary format and can be just installed, or they can be updates from previous releases. Some users might also download nightly builds for curiosity purposes.

Another group that is involved in this process are the developers of the project, which include the "developers", "contributors", "maintainers" and the "QA Team". These people also download NetBeans for further development or just for testing purposes. These people generally download the latest nightly builds when involved with testing or might download the latest source code from CVS.

Create Branch for Current Release

The release manager is associated with the task of creating a new CVS branch for the current release. The branch is created at the feature freeze date - the date by which all features of the current released should be fixed and only bug fixes are taken in. The exact steps involved in the process are,

÷      The release manager before doing this task should warn the developers' mailing list about the particular date when the branch will be created so that the developes get enough time before the actual feature freeze to plan ahead.

¸         The release manager should also check if the build is usable, i.e. it has no exceptions or bad messages when running. If the build cannot be fixed trivially then the branching should be delayed.

¸         The base tag for the release should look like releaseXY_base where X.Y is the version of the release

¸         The next step is mapping the Java modules into the branch; a SUN employee should do this task. This involves mapping of the not-open-source modules (javac etc).

¸         Remove web pages from the branch

¸         Check out the newly created branch

¸         A binary list of the tag should be made and it should be checked out by a SUN employee.

¸         Make a complete build and run sanity checks on the same. This step is carried out to ensure that the IDE is usable.

¸         Make a branch tag from the base tag; this is also carried out by a SUN employee.

¸         Set up a continuous build process by assistance from SUN release engineering.

¸         Fix any version-related problems with the files in the branch.

¸         Version numbers of specifications have to be increased by one. For a trunk it has to be increased by 1 e.g. 1.5 to 1.6 and for a branch it has to be 1.5.1. The version numbers of the dependencies do not have to be increased

¸         The existing branch has to be mentioned in the documents.

¸         IssueZilla should contain the new release number as the version in all affected modules. This might be done by QA Team.

¸         All open issues have to be migrated to this particular release number for the affected modules

¸         Announce the creation of this branch to the developers' mailing list

¸         Start creating nightly builds for this particular branch

Release RC1

Before making a final release the release manager has to go through a series of steps. Releasing the first release candidate is one of them. The release candidate is chosen after the beta has been released and all the module leaders identify their module as stable according to the stability criteria. This phase of releasing the first release candidate is taken only after stabilization phase is completed, i.e. all the known p1, p2, p3 types of bugs have been fixed. A checklist needs to be followed by the release manager for performing a release; the steps to be followed are,

¸         File a list of all known issues in coordination with the QA team into a README.html file.

¸         Synchronize all the changes from the previous release into a CHANGES.html file

¸         Based on all the contributors (groups, companies, individuals) for the particular release include a file CREDITS.html

¸         SUN Release engineering is responsible for making the final build of the release

¸         Release Engineering or Webmaster makes these releases available on the website for download.

¸         All issues related to this current release in IssueZilla have to be migrated to reflect the current release. This task is generally done by the quality assurance team.

Release RC2

After the release of the first release candidate, if there are any major issues, all major bugs p1, p2 are fixed and the next release candidate, i.e. RC2 is released. The release manager has to follow the same steps as in RC1.

Do Final Release

The release manager along with the module leaders decides on the final release. This is generally decided after at least a week of release of the release candidates and all major issues have been fixed. On the other hand if no major issues are found with the previous release candidate, then the release candidate becomes the final release. The release manager again follows the same steps followed for release candidate 1.

Release Post Mortem

The release manager after a release has been successfully deployed, conducts a discussion on the mailing lists about the release. This discussion involves getting feedback from the developers involved in the release about what tasks were carried out well, what was lacking in the release. From this discussion the release manager is supposed to come up with ideas of how to improve on certain factors in the release, this might also serve as a part of redesigning the release process and helping future release managers with the feedback.

Grant CVS Access

The CVS manager grants CVS commit access to the members of the project that are approved by the module maintainers. The module leaders send a request with a list of members to be granted access to a particular branch of the CVS repository. Based on the list the manager sets up CVS accounts for the developers.

Commit Code in CVS

The "contributors" who have "check in" privileges for CVS commit code into CVS, at the feature freeze date. All the features to be included in the release have to be included in the CVS repository. The developers who do not have commit access to CVS can ask some contributor or the maintainer of the module to check-in their code. The code has to be checked into the appropriate branch of the release. The functionality that has to be included has to be complete before committing it into the repository.

Release Information Updates

The release manager is responsible for updating the developers of the release information. This includes all the dates (feature freeze, release date), the proposal, etc. There might be certain issues that prop up after the updates and the release manager is responsible for the task of adjusting the schedule in accordance with the consensus on the mailing list. There have been instances where the release manager had to adjust the feature freeze date for certain modules as they were behind schedule.

Report Bugs

The NetBeans user community is concerned with the task of reporting bugs about the IDE after a new release has been deployed. Here the community can be consisting of developers as well as the end users of the IDE. The bugs have to be reported in a standard format mentioned in IssueZilla. The bugs are then classified as p1, p2, p3 based on the severity and impact of the bugs on the project as a whole. The developers may report bugs based on the nightly builds or any of the release candidates. The users generally report bugs based on the release candidates, beta release or the final release. If some user want to participate using the nightly builds they can do so.

Select Bugs for fixing

The developers based on the bugs identified in their modules, decide on the bugs that have to be fixed for the module to meet the stability criteria. The bugs as mentioned above are grouped on the severity and impact on the project. The bugs once associated with a particular developer are then marked so in IssueZilla so that no other developer should work on the same. The developers have to log into IssueZilla to find out the bugs that have to be fixed.

Test Builds

The QA team is responsible for testing the release before it is released to the public. The QA team takes the latest nightly builds every Friday and executes a set of tests on the same. This test may be manual test or just sanity checking critics that check the usability of the IDE. Based on the results of the various tests, the team categorizes the bugs. The main categories of bugs are - p1 (Critical bugs), p2 (Visible Bugs) and p3 (Minor Bugs). The testing carried out depends on the manual testing criteria that is specified for the QA team.

Release Q-builds

A Q-builds is a build that ensures a certain level of quality in the product. This is generally the result of rigorous testing by the QA team. The QA team after testing the latest nightly build proposes to release a Q-build if it does not identify and p1 and p2 type of bugs in the latest build. The Q-builds may be then used by the developers for further enhancements or by the users as they ensure a certain level of quality.

Deploy Builds

The Site Administrator is responsible for deploying the current builds on the website for download by the community. The builds are obtained by the website administrator from the CVS repository. S/he is also responsible for categorizing the builds based on what branch of the repository they come from and deploy them accordingly on the website. The general categories associated with the builds are, nightly builds, Q-builds, release candidates, and final releases.

Resolve Conflicts

The board is responsible for resolving any conflicts that arise in the project as a whole. For the purpose of this study the issues include deciding on what features have to be included in a particular release to deciding on the actual release dates. Till date the board has not been involved in any major conflicts.

Formal Model

The formal model was developed with Protege, following an ontology based on PML described in [32]. The work done here is identifying instances for all the meta-process components: process model, agents, resources, tools, actions and control flows. The formal specification is based on the process model and has 9 agents, 17 resources, 10 tools and 23 actions. One important part missing from the ontology used was defining of sub-processes of a certain process, such that the actions could not be grouped more conveniently.

Protege offers good support for defining and managing the components of the process model in a manageable form. Defining the agents, resources, tools and the actions was relatively easy. The agents can be linked to the actions performed, the required and produces resources can be linked to actions as also the tools needed to execute a certain action. The facilities of modifying, saving and loading a certain formal specification make from Protege a good tool to manage formal process models visually. Also, the editors for all the elements of the model can be easily extended if the formal model needs to be more precise.

The formal model developed with Protege was exported to an XML format using an XML plug-in, and also a visual representation of the process was created using the Ontovitz plug-in. The image for the whole process is very big and because its size it is not very readable, and the XML file is around 200 lines long. The formal model can be used with special tools and the image can be used with certain filters, but in the actual size they are rather confusing.  The files created with Protege are: the project file, the ontology file and the instance file.

 

Figure 7. The formal process diagram generation

 

Figure 8. The formal process xml generation

Prototype:

Prototyping the process has allowed us to simulate an activity instance of it without actually enacting it.  In doing so, we have been able to detect processes that may be unduely lengthy, which may serve as good candidates for downstream activities such as reorganization and re-engineering.  It allows us to better see the effects of duplicated work.  As an example, we have four agent types that test code.  Users may carry out beta testing from a black box perspective, whereas developers, contributers, and SUN Microsystems QA experts may perform more in-depth analysis, and, in the case of developers and contributers, they will not merely submit a report to IssueZilla, but may also take responsibility for an issue and resolve it.  However, is it really necessary to have so many people doing such similar work?  While, in this case, the benefits of having more eyes  on the problem may justify the costs of involvement (which is voluntary, anyway), in other cases, it may be less clear.

 

We are also able to detect where cycles or particular activities may be problematic for participants.  Prototypes are especially useful when implementing a process or altering a process in that these potential pitfalls may be discovered before they lead to project failure.  Over the cvourse of constructing and executing the prototype we discovered some of the more concrete reasons that there are few volunteers for the release manager position.  The role has an exceptional amount of tedious administrative tasks that are critical to the success of the project.  Between scheduling the release, coordinating module stabilization, and carrying out the build process, the release manager has a hand in almost every part of the requirements and release. This is a good indication that downstream acticities may also uncover a way to better distribute the tasks and lighten his load.

 

The self-selective nature of open source software development community participation has many impacts on the development process they use.  If any member wishes not to follow a given process, the enforcement of the process is contingent on the tolerance of his peers in the matter, which is rarely the case in more corporate development processes.  If the community proves intolerant of the alternative process, the developer is free to simply not participate in the community's development efforts and perform his own build.  So, while exercising a prototype may not provide open source community members with the same sense of empowerment that employees of a more corporate/closed process, since they are the ones that create the process in the first place, it seems not to be an issue.

 

Discussion:

Our study has led us to several interesting insights of the OSSD process. These have generally come up from the varied sources of information ranging from resources from the website to the mailing list archives. Based on the results obtained from modeling and visualizing the open source process it gives us a better idea of how these successful processes can be used for development in an organization. Traditionally software engineering has been thought of as an activity carried out in a single location with a small number of developers working in close coordination with meetings taking place whenever there is a need for communication. The open source paradigm as suggested above is totally different from this view where developers are geographically apart and generally do not even know each other. The primary means of communication is email, which considering the sheer volume is huge. Even with the constraints imposed by such boundaries, these projects are fairly successful (criteria of success being the number of user involved in the community). Previous studies [33] have pointed out how these processes can be used for developing software in a distributed environment, but they have not actually provided a empirically grounded solution for the problem, what has merely been suggested is that these processes can be used. In this section we try to provide our analysis of using the open source process for betterment of the closed source development within an organization.

The main points to be noted from the process described above is the lack of a central authority and having people in charge of themselves. This involves creating a sense of responsibility within the community about their tasks. For example, the developers and module maintainers are responsible for their part of the project and they have to meet the deadline for the release dates (feature freeze, final release date etc.). But these developers are in no way forced by the release manager to complete their tasks, they are asked for their feedback and accordingly the schedule is decided. This serves as the single most important part of the OSSD process. This has also been pointed out by Fred Brooks in his classic paper "No Silver Bullet", where he mentions the insane schedules [34]. Thus by involving the developers in the planning phase lot of the schedule problems can be avoided and the developers have a feeling of responsibility to deliver by the proposed date.

Another important factor as also mentioned by Eric Raymond in [31], is the frequency of the release. In the OSSD release process, we have many different kinds of release; these include nightly builds, Q-builds, release candidates and the final release. By releasing more frequently we have a chance of detecting the bugs early and helps on improving the code. This again ties back to Brooks law of throwing one away [34], whereby the nightly builds are replaced every night after bugs have been fixed. There have been instances where the whole release candidate had to be scrapped due to lack of quality. This needs to be adapted to the conventional process where the trend is on building a final product from the very start.

There is also this trend of exploiting the open source projects by organizations for creating products that are closed source. For the NetBeans project, this is true as SUN Microsystems, the company that funds the project utilizes NetBeans for creating their own product, Forte for Java that is sold commercially. By engaging in such a practice, SUN is able to use the resources of the community in making commercial use of the product. This also enables in saving development time, effort and identifying bugs. This also leads to more people proposing new ideas that are useful and contributing to the code base.

Conclusions:

This study investigates the NetBeans open source process of requirements and release. We began by characterizing the process as in the traditional form of software engineering and then presenting the version of the process followed by the NetBeans community by using various different forms of representation including rich hypermedia, visualization diagrams, process model diagrams and a formal model for the process.

Acknowledgements:

We would like to express our gratitude to the NetBeans community for giving us responses to certain queries. We would also like to thank Prof. Walt Scacchi for his valuable inputs for the project as well as providing us with guides for various tasks such as the rich hypermedia picture as well as coming up with the meta model. John Georgas has been an enormous help by coming up with the meta-model and providing excellent examples needed for creating out formal model using Protege.

References:

1. Contributing Modules to the NetBeans Community (http://www.netbeans.org/devhome/community/contrib-modules.html)
2. Roles within the NetBeans Community (http://www.netbeans.org/about/os/governance.html)
3. The SUN Public License (http://www.netbeans.org/about/os/license.html) 
4. The NetBeans Build Process (http://nbbuild.netbeans.org/nb-build-process.html)
5. The NetBeans Open Source Definition (http://www.netbeans.org/about/os/index.html) 
6. Proposal of NetBeans release process (http://www.netbeans.org/servlets/ReadMsg?msgId=73741&listName=nbdev)
7. The NetBeans Roadmap (http://www.netbeans.org/devhome/docs/roadmap.html)
8. The NetBeans Vision Statement (http://www.netbeans.org/about/os/vision.html) 
9. Module Stability Criteria (http://qa.netbeans.org/processes/stabe_module_criteria_process.html) 
10. NetBeans Release Experience (http://www.netbeans.org/servlets/ReadMsg?msgId=76022&listName=nbdev)
11. NetBeans Release Process (http://www.netbeans.org/devhome/docs/releases/release-process.html)
12. Members of the Development Board (http://www.netbeans.org/about/os/who-board.html)
13. Release 3.4 schedule (http://www.netbeans.org/devhome/docs/releases/34/index.html) 
14. Release Proposal Version 3.4 (http://www.netbeans.org/servlets/ReadMsg?msgId=234728&listName=nbdev)
15. NetBeans: The Definitive Guide (http://www.netbeans.org/about/books/html/tome.html) 
    
16. Suggestion of Release Candidate (http://www.netbeans.org/servlets/ReadMsg?msgId=76022&listName=nbdev)
17. Title Terminology and the Release Manager (http://www.netbeans.org/servlets/ReadMsg?msgId=74564&listName=nbdev) 
18. Request for Creation of Release Manager Position (http://www.netbeans.org/servlets/ReadMsg?msgId=73732&listName=nbdev)
19. The Release Process and Release Manager's Checklist (http://www.netbeans.org/servlets/ReadMsg?msgId=76761&listName=nbdev) 
20. Q Build Program (http://qa.netbeans.org/processes/q-builds-program.html)
21. Access source using CVS (http://www.netbeans.org/devhome/sources/cvs.html)
22. Recent NetBeans distributions (http://www.netbeans.org/devhome/distributions.html) 
23. Issue report (http://www.netbeans.org/devhome/issues.html)
24. Release checklist (http://www.netbeans.org/devhome/docs/release-checklist.html) 
25. Feature freeze (http://www.netbeans.org/servlets/ReadMsg?msgId=141662&listName=nbdev)
26. Overview of hosted tools (http://www.netbeans.org/docs/hostedtools.html) 
27. USENET service (http://www.netbeans.org/devhome/community/lists.html) 
28. The Open Source definition (http://www.opensource.org/docs/definition_plain.html)
29. SUN Microsystems (http://www.sun.com/)
30. J. Noll and W. Scacchi, Supporting Software Development in Virtual Enterprises, Journal of Digital Information, 1(4), February 1999.
31. E. Raymond, The Cathedral and the Bazaar: Musing on Linux and Open Source by an Accidental Revolutionary, O'Reilly and Associates.
32. J. Noll and W. Scacchi,  Specifying Process-Oriented Hypertext for Organizational Computing, J. Network and Computer Applications, 24(1):39-61, 2001.
33. A. Mockus and J. Herbsleb, Why not improve coordination in distributed software development by stealing good ideas from Open Source?, Proc. 2nd Workshop on Open Source Software Engineering, Orlando, FL, May 2002.
34. F. P. Brooks, "No Silver Bullet: Essence and Accident in Software Engineering". IEEE Computer 20(4):10-19, April 1987.
35. S. Bendifallah and W. Scacchi, Work Structures and Shifts: An Empirical Analysis of Software Specification Teamwork, Proc. 11th. Intern. Conf. Software Engineering, IEEE Computer Society Press, Pittsburgh, PA. 260-270, (May 1989).
36. A. Brown and G. Booch, Reusing Open Source Software and Practices: The Impact of Open Source on Commercial Vendors, Proc. 7th International Conference on Software Reuse, 123-136, Austin, TX, USA, April 15-19, 2002. Appears in, C. Gacek (Ed.), Software Reuse: Methods, Techniques, and Tools, LNCS 2319, Spring-Verlag, May 2002
37. T. Halloran and W. Scherlis, High Quality and Open Source Software Practices, Proc. 2nd Workshop on Open Source Software Engineering, Orlando, FL, May 2002.
38. A. Monk and S. Howard,  The Rich Picture: A Tool for Reasoning about Work Context, Interactions , March-April 1998.
39. C.R. Reis and R.P.M. Fortes, An Overview of the Software Engineering Process and Tools in the Mozilla Project, Proc. Workshop on Open Source Software Development, Newcastle, UK, February 2002.
40. W. Scacchi,  Process Models in Software Engineering, in J. Marciniak (ed.), Encyclopedia of Software Engineering (Second Edition), 993-1005, Wiley, New York, 2002.
41. W. Scacchi,  Understanding the Requirements for Developing Open Source Software Systems, to appear in IEE Proceedings--Software, 2002.
42. W. Scacchi and P. Mi, Process Life Cycle Engineering, Intern. J. Intelligent Systems in Accounting, Finance, and Management, 6(1):83-107, 1997.
    

Appendix

The diagrams for the Process Model.

Figure A.1 - The overall process model



Figure A.2 - The Requirements Sub-Process



Figure A.3 - The Release Sub-Process




Figure A.4 - Task for establishing timeline





Figure A.5 - Task for Establishing requirements





Figure A.6 - Task for stabilizing the release





Figure A.7 - Task for deploying the release





Figure A.8 - Task of establishing the release manager





Figure A.9 - Task of reporting bugs