As WordPress gears up for its 7.0 release, a critical discussion emerges around the effectiveness and efficiency of the release party testing phase. With contributors assembling globally to ensure the quality of each release, the community now seeks feedback to refine this cornerstone of the WordPress ecosystem.
Each WordPress release party includes a phase where contributors worldwide engage in synchronized testing, sharing results in the #core Slack channel. This tradition is one of the most accessible entry points for contributors, requiring only basic WordPress update skills. However, as WordPress and its community have grown, several issues during these testing phases have come to light, prompting this call for feedback.
Challenges in the Current Testing Process
The fast-paced nature of the #core channel during testing can be overwhelming, particularly for non-native English speakers. The mix of off-topic messages and late reports further complicates tracking essential updates. Additionally, the testing scenarios provided often lack specific context related to the current release, making it difficult for testers to focus on areas that have changed.
Another significant issue is the lack of detailed testing reports. While testers provide information on installation type and PHP version, specifics such as the steps taken, browser used, and user role tested are often missing. This gap can hinder the ability to reproduce and address reported issues effectively.
Need for Improved Testing Coverage
Determining the overall coverage of testing remains a challenge. Without clear visibility into the number of active contributors and the scenarios tested, identifying gaps in testing coverage is difficult. This lack of clarity extends beyond the release party, as ongoing testing reports disrupt post-party coordination efforts, despite requests for only failure reports.
Ensuring Continued Testing Post-Release
Perhaps the most significant issue is the absence of a clear path for continued testing after the release party. Once the event concludes, testing feedback often gets lost in Slack archives, with no comprehensive summary or steps for ongoing testing. This is a missed opportunity, as each new WordPress version warrants extensive testing across various environments and configurations.
What To Do
- Contributors: Provide feedback on how to improve the testing process in the #core Slack channel.
- Project Leaders: Consider implementing structured summaries and paths for post-release testing.
- Community Members: Participate in discussions and offer suggestions based on personal testing experiences.
- Non-native Speakers: Suggest ways to make the channel more accessible during high-paced testing phases.
