Pipes!

Pipes Beta

This is my first blog post.

For the past several months, my teammates (ed, jonathan, daniel raffel, kevin) and I have been working on a project called Pipes. It is now live. Since Pipes is all about feeds, I thought that it would only be appropriate if I finally started a blog so that I could produce one too.

I am very excited to see Pipes’ vision resonating with some of the most visionary people on the web. I’d like to think that we pushed some boundarys and I am happy to see both the concept and the execution of Pipes being commented on.

Early on in the development of Pipes we recognized it’s large scope beyond simple processing of feeds. For example, would it not be cool if Pipes could be just as easily be applied to REST web services to allow truely generalized mashups?

Stay tuned for more…

ps: a hello to bradley and caterina, our project sponsors.

Advertisements

Comments

  1. Well, I’ll be your first comment then. And the first to congratulate you publicly on what is a “stunning” “milestone”. I couldn’t be prouder to be working a cube over from y’all. This is Yahoo!

  2. very nice work Pasha. congrats

  3. Ben Clemens says:

    Congratulations Pasha!

  4. Congratulations! It’s beautiful :)

  5. Pasha! What can I say. I’m so happy for you. I’ve been hoping this would come out since the first time I saw it all those months ago.

    Please take in the accolades. You of all people deserve the attention. You’ve been evangelizing and hacking innovation at Yahoo! long before it was encouraged and supported.

    Keep fighting and keep innovating. The internet is better because of your determination and brilliance.

    -beach

  6. Jon Aizen says:

    Awesome work on Pipes. You’ve taken a hard problem and put a great interface on top of it. I’m very impressed.

    Have you seen Dapper? http://www.dapper.net – we have a feature that’s not entirely different called Linked Dapps which lets a user visually link together any number of sources on the Internet (including those that do not have feeds or web services). Would love to hear your input on Dapper.

    Again, great work!

  7. Haven’t seen it yet, but the concept is exactly what I’ve been looking for….almost as exciting as the appearance of the first browser to make it easier for people to connect to and locate info on the Internet. Can’t wait to try Pipes out….

  8. markjvickers says:

    Excellent! Now how about exposing an API for programmatic creation of new Pipes?

  9. Congrats, great work!

    “would it not be cool if Pipes could be just as easily be applied to REST web services to allow truely generalized mashups” – to mix & splice you’ll need a generalized data model, time you started reading Planet RDF!

  10. This is great, Pasha. Inspiring.

    One question is whether Y! intends always to be the processing center for pipe operations. Could the individual processing blocks somehow be distributed as work units in a P2P fashion, so that Y! need only be a pipes directory service/coordination hub?

    It seems reasonable, for example, that a site like Techcrunch might be willing to help out in the filtering/transformation of its own feed, because it would give them valuable insight into how their readership is using their content; remixing it. At the same time, it’s a win for Yahoo! because you needn’t host all the horsepower needed to do the rote grinding.

    Even better might be if some of the processing could be offloaded to clients e.g. via a desktop agent or browser plug-in. If I’m reading a pipe on my desktop (as opposed to e.g. My Y! or Bloglines), my PC may as well help out with some of the heavy lifting that otherwise Pipes has to do.

  11. Hey Pipes is great…but I can think of something that would help it out a lot. One of the strengths is the content analysis…but the output is an element and not text. For those of us who want to built dynamic url’s with the content analysis string, it would be a great help to alllow this.

  12. Hi Jake,

    We are aware of the limitations of our current modules/data types. We are working on introducing more generalized data types and have individual module/functions produce them instead of embedding the results in the items as we do today.

  13. Danny: you are correct :)

  14. Evan: Pipes already relies on a fair bit of processing power from all the web services that it calls while executing. We are planning on supporting external processing modules in the future. In general, I expect Pipes to be part of an eco-system of data sources, processors and consumers.

  15. Hello Pasha,

    Nice work, these Pipes seem very promising! If you would consider my comments I will find useful to add some generators on top of the opérators. It would make things easier, let say, to create dynamic dates and use such basic generator as building blocks for URL etc.

    I suggest this since there are not so much RSS based webservice taht provide date, time, etc.

  16. Hi Denis: agreed.

    The User Input modules provide a way to this. For example, you can use a Date Input module and set the default value to a relative value such as “2 hours ago”. The output date object can be fed to the date formatter and URL builder to construct dynamic URLs which you can feed to the Fetch module.

    This approach has the side-effect that your Pipe will have a date input in its UI which you may not want. We are thinking of adding constructors for all the primitive types to solve this problem.

  17. Great ;)

    Thanks for the explanation I didn’t noticed this !

  18. This is great! How long did it take you to build the system?

    – Sean

  19. sorry sean: yahoo policy does not allow me to comment on that.

  20. Pasha jan this is awesome. You guys have done a fantastic job :-) One input I would give would be to make an additional module to pipes that allows us to query with SQL. Alternativly enabling stored procedure like language inside one of the pipes would be ideal too since that would open up the development door tremendously. Looking forward to building many pipes :-D

    • efat kimiaie says:

      ارش جان سلام امیدوارم حالت خوب باشد

      منو یادت میاد یانه ؟

      سریره را سلام برسون دلم برای شما تنگ شده

      کیمیایی

  21. Hazel Fernandes says:

    Nice Pasha! (and whateva…it’s way over my head!) actually i was just trying to find an email for you just to say hi! what are you up to …(though i can see you’ve been very busy) and if you remembered me from DC.

    -Hazel

  22. Pipes is very impressive I first tried it a couple of weeks ago. The IPhone interface looks great.

  23. very interesting, but I don’t agree with you
    Idetrorce

  24. plz. give me informations on generators. i want to market in bangladesh

  25. thanks

  26. Alexwebmaster says:

    Hello webmaster
    I would like to share with you a link to your site
    write me here preonrelt@mail.ru

  27. tornado suboxone

  28. Oftentimes, the double bench is utilized by artists,
    music Universities and aficionados. When utilized in institutions and music universities, this can be
    the ideal bench mainly because it can face up
    to every day use, and wear; along with longer lasting than vinyl.
    Piano is the costliest musical instrument, the person should be
    very cautious while buying the piano.

  29. My brother suggested I would possibly like this web site.
    He was once totally right. This post truly made my day.
    You cann’t consider simply how a lot time I had spent for this information!

    Thanks!

  30. Thanks for sharing!

  31. Thanks for sharing!

  32. Curso Mini Site Ninja Thanks for sharing!

Trackbacks

  1. […] for me, my favorite pipe is European Performance Car News. When Pasha Sadri first told me about his idea, I immediately wanted to have this pipe. It pulls automotive news from […]

  2. […] Pasha Sadri: the man with the idea, who also came up with Yahoo! Trip Planner is a brilliant thinker and draws great caricatures to boot. […]

  3. […] Pipes was built by Pasha Sadri, Ed Ho, Jonathan Trevor, Kevin Cheng, and Daniel Raffel. I work in the row of cubes right beside […]

  4. […] to Pasha (who seems to have taken the opportunity to launch a blog) and the rest of the team that made this […]

  5. Yahoo! Pipes

    Last night Jeremy announced Yahoo! Pipes. The news spread fast and now the site is down.
    Before it went down I played with Pipes a bit last night. What amazed me the most from the start is that it isn’t a flash app, all HTML and JavaScript. Ya…

  6. […] especially cool about Pipes is how it came about. Pasha Sadri first had the idea. Bradley Horowitz, Chad Dickerson, and Caterina Fake all heard about it, said, […]

  7. […] los “gurús” que hablan del tema: Tim O´Reilly, jeremy Zawodny, Matt Cuts de google, Pasha Sadri (el de la idea) que tuvo el soporte de Caterina Fake (la fundadora de […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: