Jonathans stuff

Well its been a while since my last post but I’ve been experimenting with some new hardware and its SOOOOO disappointing and frustrating I thought I’d vent here.

Here are a set of throughput measurements using two Apple Airport Extreme Base Stations (AEBS) in a very simple setup. Both AEBS’ are gigabit versions running 7.2.1 firmware (latest).

I have two network segments connected via two AEBS like this:

Segment A: DSL modem -> Linksys B+G router running DD-WRT -> AEBS -> iMac
Segment B: Buffalo NAS <- AEBS -> Mac Mini

So the AEBS connected in A, AEBS(A), is very simply configured to be in bridge mode, and to use wireless-N (5Ghz) only. The AEBS(B) is configured to “extend” AEBS(A) and not to accept wireless connections (the reason becomes very apparent in a minute). WDS is out as that doesn’t support N.

The Linksys router is providing B&G networks, etc, so I can leave the AEBS(A) happily in 5Ghz N only doing very basic routing. All wired connections are gigabit.

So heres a series of transfers using FTP from different machines in this network:

macbook to imac (wireless N via AEBS(A) to wired iMac)
ftp: 9.45 MB/s
drag copy: (135Mb in 35s) 3.5MB/s
GREAT for raw transfer. Whats going on with AFP?

macbook to buffalo nas (wirless N to AEBS(A) to AEBS(B) to wired NAS)
ftp: 780kB/s
MISERABLE

macbook to mac mini (wireless N to AEBS(A) to AEBS(B) to wired mac mini)
ftp: 645kB/s
MISERABLE

mac mini to buffalo nas (wired to AEBS(B) wired to NAS)
ftp: 16.44MB/s
GOOD, but seems slower than expected (gigabit all-round)

mac mini to imac (wired to AEBS(B) over N to AEBS(A) wired to imac)
ftp: 3.26MB/s
OK at best. Faster than G, but why not as fast as macbook to imac?

mac mini to macbook (wired to AEBS(B) over N to AEBS(A) over N to MacBook)
ftp: 1.98MB/s
MISERABLE, two N’s are trampling on each other?

imac to buffalo nas (wired to AEBS(A) over N to AEBS(B) to wired NAS)
ftp: 1.35MB/s
MISERABLE

imac to mac mini (wired to AEBS(A) over N to AEBS(B) to wired mini)
ftp: 1.15MB/s
MISERABLE

imac to macbook (wired to AEBS(A) over N to macbook)
ftp: 9.4MB/s
GREAT, same as reverse

What makes the MISERABLE ones even worse is that going via AFP is 3 times slower in most cases.

My final test was to disable my N card on the MacBook and connect the MacBook directly to AEBS(B) and copy from AEBS(A): 3MB/s. Eh? What? I get 9MB/s via N but when going via the “extended” AEBS(B) via N I only get 3MB/s

So my conclusions, so far, based on this are:

• EXTEND MODE DOESNT WORK anywhere near N and is far far worse than G in most instances. I can’t believe how broken the “extend” seems to be.
• The overhead of AFP hurts throughout very badly
• I cant use this setup as copying anything on Segment B to A (from A) takes HOURS for 350Mb. Truly bad.

Anyone offer any tips? This is the worst experience Ive had in a long time with moderately expensive hardware…

Im presenting at the Open Hack Day in London this weekend and as part of it I’ve knocked up a quick example script for executing pipes from Javascript on your web site, and getting the results as an object in a JS function (so you can loop and display the contents for example). You can check out the sample code and a very simple example here.

The interesting thing is you can modify this quite easily to work with any JSON callback based API (those that support a callback parameter) like many Yahoo APIs, as well as Flickr now…

Im happy to announce that we’ve massaged (to borrow from Flickr) the Pipes web site and other parts of the system quite a bit in the last few weeks and pushed out a new version tonight. Its part of ongoing work to make the site easier to use and we hope to keep the changes coming over the next few weeks. There are of course some new modules, and additional fixes to bugs and issues that are reported on our message boards or feedback page. If there’s something you think we’ve missed, please add it to our feedback page, or add a comment here.

Google recently invited the team to give a tech talk about Pipes.

If you’re interested in some background about the project and some of the issues we were thinking about you can see Pasha and myself talking here

I noticed a few blogs mentioning that Pipes uses Google’s open source canvas wrapper library for drawing the editor lines in IE, and wanted to comment on that.

Yup, its true.

Yup, its a nice little open source library that solved a small problem that would have taken a little while to work around. It draws the nice looking lines in IE. That’s pretty much it (for Pipes). Every other browser does it with our code and the canvas tag.

Yup, we could have written our own, and we may do so in the future, but there’s no great mystery in the technique or the code, and they did a good job so I decided not to reinvent the wheel.

As mentioned here we’re seeing Yahoo!s YUI libraries being mixed with Google code too.

P.S. For those of you using IE….what are you doing using IE? Pipes looks much nicer in Firefox and Safari:-)

Quite a number of people have asked about how the lines were done in the editor so heres a quick summary.
The answer is actually pretty simple, although getting it “right” is quite hard.

The lines are drawn using the canvas tag, which is supported in firefox and safari (and webkit - which is the fastest renderer of them all). Its also supported in opera but debugging opera is nearly impossible currently. We make use of a wrapper for IE which converts many of the canvas operations transparently into VML.

Now the problems kick in: safari doesn’t redraw when the canvas gets resized; “sizing” the canvas needs to be done in the right way; the canvas tag obscures the DHTML elements very easily (the bounding box for the canvas can be large) - so you need a solution for drawing the lines and still be able to interact with the other DOM elements (FWIW the editor has the 4th variation we tried). There are lots of small issues

Another sporadic blog update, and again about work…

My colleagues and I have just launched Pipes after many months of hard work, and Im really pleased with what we’ve created. I think its got a lot of possibilities and we’ve only begun to scratch the service.

“Pipes is a service platform for processing well-structured data
such as RSS, Atom and RDF feeds in a Web-based visual programming
environment. Developers can use Pipes to combine data sources and
user input into mashups without having to write code. These mashups,
analagous in some ways to Unix pipes, can power badges on personal
publishing sites, provide core functionality for existing or new Web
applications, or serve as reusable components within the Pipes
platform itself.” (thanks Maciej)

While we’re a small team (Pasha, Edward and I working across the whole system and design), I focused mainly on the editor which has been fun and frustrating at the same time. I’ll try and comment some more on that particularly working on the “line” connectors which I hope people will enjoy

Others who’ve talked about the release (and provided great context for where this has come from and can go to):

• Tim O’Reilly
• Jeremy Zawodney
• My boss’ boss Bradley Horowitz

Enough for now… we’re pushing changes as I type…

We’ve finished developing a mobile prototype called “Checkmates”. The Checkmates prototype is a mobile application for telling your friends where you are and seeing where they are on a scrolling mobile map. You can even use a custom map from inside a building - which is one of the really cool new things. You’ll be able to see your friends’ locations, what their status is, and when they broadcasted their location.

The main site is here: http://tinyurl.com/pya4x

The current version comes with data for the eTech conference, local bars, restaurants and hotels - and a couple of floors of the hotel with the eTech conference setup.

One of the other nice things we’ve been working on is the radial/pie menu system. I’ve been dying to find a good application where we could try one of these out and a phone is a really nice place to do it. A mobile device has really restrictive input capabilities and everyone is used to using the 5-way. I think it works pretty nicely in the app, and it’s even capable of selecting an entry from a long list.

Its built in J2ME (MIDP2) for the broadest platform support on mobile devices and uses Yahoo!s external Flickr API for defining your “friends” (using Flickrs Friends and Family social network). J2ME turns out to be a bit of a pain to develop with and I wish there was far better emulator support that mimics real phones better.

One nice side effect of using Flickr is that we can add map tiles into your photostream that represent where you are. These can only be seen by those in your social network (the same people who can see where you are using the app).

More blogs about the prototype from my colleagues:
Edward Ho
Chad Dickerson

This is coming a little late, but I’ve been pretty excited by some of my recent work in a new team at Yahoo!, namely the Local Events Browser. It’s an Ajax heavy super-mashup across a lot of REST APIs. What makes the application “cool� (IMHO) is that is all client-side Javascript – the only server involvement is Yahoo!s public APIs serving up XML data. The whole app fits into less than 100k of downloadable code. It seems likely that we’ll see more and more client-side code for the “presentation� aspects of a web application and less and less server-side UI creation. Another trend will be to more and more multiple server-side API usage – the Event Browser used 5 and we could see how others could have been used:

1. Maps 4.0 AJAX
2. Term Extraction
3. Image Search
4. Geocoding
5. Local Search

My previous life (as a Senior Researcher at FXPAL) involved working in cross-disciplinary teams, and its great to see the same advantages of differing strengths and perspectives coming together in such a short-term project – with everyone involved throughout all stages of the design and implementation. Thanks Ed, Karon, Sam and Nate!

Related bloggy things (from Eds post about the EB):

Jeremy Zawodny has some nice things to say about the Local Event Browser
YSB announcement with all the new Maps API links