My post on the future of Drupal QA is up on the g.d.o QA group:

http://groups.drupal.org/node/9727#comment-30480

Please read it over and comment!

Dries' newly released Usability Testing first results (here) are in, and it does reinforce my previous point that even as we make the code and patch testing better (via unit tests), many other parts of quality have been overlooked for some time, and developers shouldn't be the ones to force it on. (you can read that previous post here)

In fact, between this and some valuable feedback I received from those in #drupal, I've decided to outline a full proposal for moving Drupal's Quality Assurance forward. As Karoly was kind enough to mention, just posting about this on my development blog isn't enough :) . I'm writing that proposal now, and will be posting it in the Quality Assurance group at groups.drupal.org sometime tomorrow.

I should also note that contrary to some feedback I received from #drupal, I do think unit testing is a great thing, and I do think Drupal is doing good, and I do think we're making progress (one of the (very) few downsides of open source communities is that people take criticism very personally, since everyone who works on Drupal does in a way feel it's a bit of them ;) ) I was, in no way, saying that anyone wasn't doing their jobs, just that we hadn't recognized some of these QA areas as even being needed before. Alright, enough apologizing, I'll see you all tomorrow!

(If there is anything you think the proposal should include, give me a heads up)

Community Segmentation, Bad News?
What brings me to this topic is all the excited talk I've been hearing second hand about splitting groups.drupal.org and drupal.org into many separate chunks (documentation.drupal.org, news.drupal.org, groups.drupal.org, downloads.drupal.org, content.contributed.maintained.6.modules.downloads.drupal.org , you see where this is going). What worries me is that in such an implementation it may become too complex to get to key areas or discussions.

The way I see it, there are a couple major roadblocks for making such a system work (here comes the dreaded Robin-list, grab some coffee):

1. Unified login (no, openid isn't the solution, we have existing accounts on the sites and all those need to be unified).
2. Domain complexity, in my last example above, we can't expect people to remember that. We need to have only primary level subdomains. EG: news.drupal.org, but not, modules.downloads.drupal.org.
   
3. Unified search (this is very important, we want people to be able to find the resources they are looking for). We also need to be able to isolate the searches per content types, per site. So you can search for polls on groups. directly from any other site.
4. Unified look. I don't suspect this will be a problem, bluebeach will need to be modified just slightly to show you're at a different site)
5. Unified input formats. Right now, you can use markdown on groups.drupal, but not at the drupal.org forums. We need to allow the same syntax to be used everywhere
6. Unified front page options. For maximum usefulness, we'd need the moderation options for "Show on this site's frontpage" as well as "Show on drupal.org frontpage".
   
7. No link left behind. We need to implement either redirects from old links we'll be moving, or have a system to search all the sites if a unavailable page is requested. We have a lot of sites that link to nodes on drupal.org, and if we loose those, we loose a lot of useful external documentation.
8. Clear cut reason why this is needed. I personally don't think this level or decimation is necessary, or even useful.
   

Drupal let the quality boat sail by. Hold on, hear me out :) We're a really focused and strong (and almost too commercial) community, and we really have put our focus on features and output, and we've lacked a lot when it's come to making it all work correctly in every possible case.

That's where quality testing comes in. I've been thinking about Drupal's quality assurance process for a couple years now, but in the last few months it's really come to a head for me. My personal "dam-breaking" moment came when the Batch API in Drupal 6 prevented install, and other tasks depending on the API from running automatically, or in text based browsers, or in browsers were people have been privacy conscious and disabled JS and Meta refresh tags.

Without proper quality trails this will continue to happen. And here's another point I'd like to make:
IT'S NOT THE CODE DEVELOPERS FAULT, OR JOB TO DO THIS KIND OF TESTING.

If we expected every patch to have every possible test performed on it by the developer, or even automatically, we're fooling ourselves. Testing.drupal.org, even though it's going to help; even though it's going to catch errors; won't ever be able to replace human testing. Simpletest and DrupalCHK framework is not going to be the do-all and end-all of our problems, nor should we try to market it as such. We risk people feeling that if we have unit testing, we can skimp on the other Quality bits. Just like how a newbie computer user might think Norton 1998 is still protecting them in 2008.

Here's what we need in order to make future releases of Drupal truly "quality assured":

1. We need to have community-created smoketests that members of the community can run on various servers, with various browsers. These are low level things drupal must be able to accomplish in that environment. Each "smoketest", eg, "Create a new node type" would be able to be marked "working", "working with errors", "working with incorrect layout or UI", "failed". We'd log the different environments to find patterns.
   
2. We need user interface quality testing
   • information architecture
     
   • disability-compliance testing (screen readers, color blindness)
     
   • standards compliance (ECMAscript, CSS, X/HTML)
3. We need a group to asses changes to the coding standards, and to modify and refine the standards and time goes by.
4. Quality events (eg, a weekly or monthly online event were everyone can work on bugs, perhaps with prizes)
5. Quality initiatives (bug spotlight is a good start, but a group that can define more initiatives like this)
6. documentation, forums, live irc help and mentor for those new to helping with quality testing. Probably best done as quality.drupal.org, using quality.mozilla.org as a guide. The great thing about quality testing is: you don't need to be a developer, or know how to code!
   

Hopefully, this will spur the Drupal powers that be to make more of a effort and create a full fledged quality team, and not just the appearance of quality by using a single method of quality assurance. I'll be happy to assist in the development of such a team.

I haven't meant this post to be harsh on Drupal, we've done very well, but this will hopefully nudge people in the "quality" direction.

I'd like to submit a proposal for a future location of Drupalcon. Given that Drupalcon Boston is now going on, it seems like now is the opportune time to "whisper" in the ear of attendees, and to suggest the next location on the east coast.

And that spot is..... (drumroll please :) ): Saint John, New Brunswick, Canada (map)

Why Saint John?

• Local History
  - Saint John was the first incorporated city in British North America.
  - Amelia Earheart landed here during her solo trans-continental flight.
  - A wealth of historical buildings, galleries and museums showcasing the architecture and artistry of the east coast, past and present.



• Attractions
  Imperial Theatre
  New Brunswick Museum
  Saint John Arts Centre
  Leisure Guide
  
• Transportation
  - Airport
  - Bus Network
  - Parking
  
• ...and the other stuff you need...
  Places to stay
  Things to do
  Oh, and you can go whale watching too :)
  

Where could it be held?

Glad you asked :) Saint John tourism maintains a list of Meeting and event facilities as well as Convention hotels.

So, the next time someone mentions about the next Drupalcon, mention Saint John!

Disclaimer, I live near Saint John ;)