GoodBY!

I have recently left Yahoo! to pursue some personal projects.

This is a good time to recap my six and a half years working at Yahoo!

I joined Yahoo! in November of 2000 (the dot-com crash was almost in full swing, but having recently left college, I was oblivious to it).

My first noteworthy project was conceived two years after I started. It was called SmartSort. It saw the light of day as part of Yahoo! Shopping sometime in 2004 after it won Yahoo!’s Innovation Award (thanks to Koshi and Beach). SmartSort was notable in that it was doing a lot of computation in the browser before AJAX was all the rage.

About the same time, I had started working on an interactive map for Yahoo! Travel. That project went on to become Yahoo! Maps SmartView (don’t ask me; I didn’t come up with the SmartX names). SmartView was one of the first interactive, multiple point-of-interest mapping products on the web.

There is an interesting back story behind SmartView. At the exact time we were launching SmartView, a small company called Where 2 (founded by the brilliant Rasmussen brothers) was trying to raise VC funding. Their deal fell through because of SmartView’s launch. They were however introduced to Google who decided to acquire their company. They went on to develop Google Maps and the rest is history…

After SmartView, I started working on SmartTrip (just kidding). The product I worked on next was called Trip Planner and it was one of Yahoo! earliest in-house vertical social applications. It has held up well.

In mid 2005, I worked for a few months at Google with the Google Maps team (I will save the details for another time).

I returned to Yahoo in the end of 2005 in a new role where I was given a lot of latitude to come up with new ideas and pitch them as projects to the management team. I developed several prototypes, including an early prototype called RSS::Pipes. This project was green-lighted by Jerry Yang and become Pipes. Needless to say, the entire Pipes project was an incredible experience for me on many different levels, both during its design & implementation and the months following its launch. Pipes is now being run by Jonathan Trevor (who developed most of the amazing Pipes Editor).

I will try to condense my learnings into a few points:

• Aim high, life is too short for anything else.
• In life, you do what you do, not what you think or say you are going to do.

My new project is called Polyvore.

Pipes #4 on R/WW list of Top 10 Yahoo! Products

More great coverage of Pipes at ReadWriteWeb.

BTW, There have been a lot of new modules added in Pipes, check them out if you have not done so lately.

Safari and iPhone

As one would expect, Safari browser is the main vehicle for extending the iPhone’s functionality. This fits with the general trend of applications migrating from desktop binary applications to ones that run in a browser.

I just hope that Apple decides to expose more of iPhone’s unique functionality as JavaScript objects accessible inside Safari. I know that proprietary browser extensions are frowned upon but I think it is justified in this case. Pages taking advantage of these extension would be exclusively targeted at the iPhone and not intended to run anywhere else to cause cross-browser issues.

So, what is on my wish list of extensions:

img = phone.camera.capture();

light = phone.camera.ambientLightIntensity();

audio = phone.mic.capture();

browsers actually don’t have an native audio representation. so, that is another thing that needs to be added in:

audio.play();

audio.toText(); // wishful thinking

It would be great to have events that expose the phone’s tilt sensors (similar to mouse events).

Also, some kind of off-line storage and computation similar to Google’s Gears would be great for writing more complex applications. Access to the phone book and calendar would not be too shabby either. I guess the list can just go on and on…

Finally, it is ironic that one of the coolest touch sensitive interfaces in existence does not expose any of the touch and drag/drop events into the browser. I think this was put in place for a backward compatible browsing experience for existing web pages, but I wish it would be exposed for applications that want to explicitly take advantage of the touch interface.

The best bit about Pipes

Someone told Daniel tonight that if they ever wanted to work at Yahoo! they would want to work on Pipes. I think that is the ultimate compliment.

Pipes Tech Talk @ Google

We gave a Google Tech Talk about Pipes over at the Googleplex on Monday. Many thanks to the organizers of the Google Tech Talk series for hosting us. They kindly captured the entire thing on video, did a great job on the post-production of the video and finally posted it to Google Video for your viewing pleasure:

New Pipes module: Fetch Data

Jonathan and I made a presentation about Pipes at ETech this past week. Not surprisingly, the Internet connectivity went out just at the moment I was about to give a demo of the Pipes editor. Fortunately, we had a backup video to use instead (thanks to Daniel for being paranoid and preparing it and Jonathan who insisted on  transferring it over to my laptop in the last minutes before the presentation).

We did announce a new module that should be very interesting for developers: Fetch Data. In addition to the regular Fetch module which worked with RSS and Atom feeds, the Fetch Data can work with arbitrary XML or JSON feeds.

The output of the module is still expected to be an array of records. With the help of the rename and regexp modules, it is possible to massage most data into something that can work well in Pipes. For example, you can process KML, XSPF or any other result set oriented REST response (eg: Amazon web service calls). I can’t wait to see how it is going to be used…

Active content

Over the years, web based applications have made it easier for people to create and share content on the web. Examples include:

• usenet: text
• blogs: html
• Flickr: images
• YouTube: videos

There is definitely a progression in the richness of the media being created but they all have one thing in common: they are all passive. You can look at an image but it will not interact with you nor will it change over time.

There is however a new category of active content emerging. The earliest examples I can think of are greasemonkey scripts. These are active in the sense that they operate on other content and their output therefore will change in different situations and over time. Pipes is another example of enabling users to create active content. A Pipe is an instruction set for how to operate on feeds. Its output will change over time based on its inputs.

The Long Tail of applications

Most people who are interested in web technology have no doubt heard about the long tail. Amongst other things, it describes common usage patterns of content or goods on the web.

I believe that by lowering the barriers to creation of simple applications, Pipes (and similar systems) will contribute to an abundance of available applications. The usage pattern of these creations will closely follow the long tail pattern. There will be a few major hits, but the bulk of the usage will consist of very niche applications that solve very specific problems.

Welcome to the long tail of applications.

ps: It turns out other people have already been there and done that. Too bad the mailing list mentioned on the long tail site appears defunct.

Crazy 72 hours

It’s been almost 72 hours since our beta launch. Needless to say, the response to Pipes has been overwhelming for everyone on the team (and our servers!). I have finally had a chance to relax/sleep after addressing some of the immediate issues.

Now we can get back to the main reason for introducing Pipes in beta: to find out what people want to do with them and use that feedback to guide our next steps.

We have already been getting a ton of feedback (thank you!), some of it things that we were aware of but didn’t get around to implementing before the beta(eg: date sorts), some of it about ideas further along on our plans (eg: external modules) and some really cool and wild ideas (eg: image processing modules,…).

I would like everyone to know that we are listening to their requests and we will try to evolve Pipes to meet their needs. It would be great if we could get your feedback through our suggestion boards so that we have it all in one place.

I was also amazed/flattered to find that there are already several Pipes tutorials and how-to posts in addition to the ones made by Brady from ORA.

Pipes Buzz!

Pipes is getting a crazy amount of coverage on the web (and our server load is going up with it). You can follow some of the buzz at these locations:

• Jeremy Zawodny
• TechCrunch
• Technorati

Or you can track them all using Leonard’s buzz aggregator pipe