A commitment to Quality

December 15, 2011 – 12:40 am

A lot of good things are currently happening around Quality in Ubuntu. Rick just introduced us to all the efforts the Ubuntu team is working on (Upstream Testing, Acceptance Testing, daily ISO builds and integration tests, automated daily Smoke Test …). Jono has hired a Community QA Coordinator to be the interface between Ubuntu, its community and everybody who is interested in the quality of Ubuntu. Pete has put together a strong Ubuntu QA team and is still increasing its capacity. It is amazing to see how the Platform and Unity teams are working together to make Ubuntu 12.4  the best LTS release ever.

The Unity Quality Team

However, Canonical’s commitment to Quality extends further than addressing quality where Ubuntu is being put together. As a sponsor to upstream projects of Ubuntu, we understand our responsibility that we have for these projects. With that in mind I am happy to update you on our commitment to Quality in the Unity team.

The Unity project will receive help from a dedicated Quality Team which we are currently ramping up. The team consists of various functions that are critical to provide well crafted code.

Alex, Allan, Thomi and Thomas in their role as Quality Engineers are working closely with the Unity Software Engineers. Their main priority is to support their assigned components in creating test frameworks and automated tests, work on integration tests and help us measure the performance of our projects. We are still looking at growing this team!

Omer and Bilal, our Bug Triagers are the first line of defense between our engineering teams and 20 Million users. This powerful team is connecting the dots between a filed defect report and the responsible engineer and is helping us in facilitating the defect management process. Let me know if you would like to participate.

Martin in his role as Quality Lead will be coordinating the efforts between the various engineering teams and the Quality team. He will be defining metrics and targets and his opinion will be sought to determine whether a component is ready to be released.

Open to everybody

With Unity being an open source project the Quality Team is going to follow suit and will work with the community wherever possible. Check out the Ubuntu fridge in January for dates for our Quality meeting, where the team will be giving updates on Unity testing, present test tools and methodologies and discuss Quality questions with everyone who is interested.  We have also started a blog, the Quality Hour, where we will update you with what’s going on in the team. With increasing demand, we are also looking at setting up a dedicated mailing list, but for now we are watching the Ayatana mailing list

Everybody is part of the Quality Team

High quality software is not the product of an individual software engineer nor of the Quality Team but it is the outcome of joint efforts of the whole software engineering team. The desire to produce something great, to produce manageable and maintainable code, the readiness to steadily review and improve test functionality and a commitment to quality is the base for a solid product. The Quality Team is a catalyst as they support the software engineers in getting quality straight into the foundation of our products. It is very exciting to see how the team approaches quality and I wanted to share a few examples of this joint effort with you.

  • Inspired by Thomas’ and Allan’s work on code coverage, Ted went ahead and looked at the code coverage for one of his test suites.  Unhappy with the results he sat down to implement more tests and testability hooks and bumped the coverage by 15%  for line coverage, 8% for function coverage and 22% for branch coverage.
  • Gerry is working on Testability, a test suite that will help us to drive automated GUI tests in Unity 2d
  • Thomas and Chase looked at improving the automated tests for the uTouch stack and needed a way to run tests in a headless X server. The outcome is a sweet little X.org google test extension, which allows to write tests that instrument a dummy or running X server.
  • Alex and others from the Unity and Platform team have been heads down in getting Autopilot v2 for Unity 3d up an running. Autopilot is one of the key tools of the continuous integration testing that Unity releases are put through.
  • Mirco has gone off and defined a process for how to handle design assets in Launchpad and bzr
The list could go on and on and I wanted to thank everybody (Didier, Sebastien, Ken, Evan, Tim, Jay, Neil, Sam, Robert, Martin, everyone who is currently integrating automated tests in projects, everyone who is writing regression tests for fixed bugs or new features… and everyone else whom I hereby forgot) who is putting in extra work to make this happen! You guys rock!