Stream Wrappers and You


This past weekend, a team of core Drupal developers, including Peter Wolanin (pwolanin), Andrew Morton (drewish), Jon Stacey (jmstacey), Justin Randell (justinrandell), Jody Hamilton (Jody Lynn), Alex UA (Alex UA), and myself, met at the Zivtech headquarters in Philly to hammer out some hot media and file functionality. After much arm-wrestling, teeth-gnashing, and nail-biting, plus some uber-helpful, last-minute, remote spot checks courtesy of Darrel O’Pry (dopry), Dries Buytaert (dries) and Moshe Weitzman (moshe weitzman), we managed to get the initial patch for PHP Stream Wrappers committed to Drupal 7!

As exciting as that is (a year-and-a-half of work from over a dozen contributors), the hard part has just begun. Part of me wants to relax and go celebrate. The other part of me wants to hunker down and get to work, because we only have about six weeks until the Drupal 7 code freeze.

I imagine many readers have two big questions: Just what is a stream wrapper, anyway? And What’s next (and how can I help)? Let’s jump right in, then.

Just what is a stream wrapper, anyway?

PHP, the scripting language that powers Drupal, offers some basic file manipulation capabilities. These include such functions as file_copy, which copies a file from one location to another, and file_delete, which likewise does what it says.

PHP also allows a built-in way to override that functionality, by implementing a class known as a Stream Wrapper. Thus, you could create a Stream Wrapper for a CDN such as Amazon S3, and allow it to manage files as easily as though they were on your local server. Stream Wrappers are identified within Drupal by their scheme (similar to the well-known http://), so that the public and private file systems are known as public:// and private://. Thus, an S3 Stream Wrapper class might use the scheme of s3://.

With the system thus abstracted, it is relatively easy to extend the functionality in further and sometimes novel ways. For instance, a stream wrapper might be implemented for Flickr images (flickr://), YouTube videos (youtube://), or even articles from the New York Times (nytimes://).

What’s next (and how can I help)?

Here’s where you come in. I’m going to present a list of links where you can jump in and help boost development.

  1. File API Stream Wrapper Conversion. Unfortunately, PHP Stream Wrappers was just the first step. We plan next to replace the current file system with stream wrappers, allowing for a wide range of exciting new functionality.
  2. Create the Media Module. An initial boost for all of this came from earlier sprints with Arthur Foelsche (arthurf), Darrel O’Pry (dopry), and others, to create a media asset API, of which stream wrappers is just the inward-facing piece. The outward-facing piece is a robust API to implement a unified GUI for managing stream assets. Sneak peek at some exciting Media module mockups by Maarten Verbaarschot (mverbaar) of the D7UX team.
  3. Version 2 of Embedded Media Field. With stream wrappers now in core, and development of the Media module in full-swing, the thousands of sites using Embedded Media Field to allow editors to easily embed third party content will need to be completely reworked to work with the new system. We still need to create a roadmap, which may involve simply migrating all existing third-party content to FileField.
  4. Converting your favorite media module! If you maintain a media module you want to ensure easily interfaces with the new stream wrapper functionality, feel free to ask for help. If you use a popular media module, and are not sure of its future status, jump on its issue queue and ask the maintainer.

Drupal is an awesome community, and it is this ongoing spirit of collaboration that allows for awesome things like stream wrappers in core to happen! Thanks to all the developers responsible for making it possible. And future thanks to those of you inspired to jump in and help.