Friday, October 21, 2011

Adventures in the Celestial City - Off to Beijing

Yes, that's right - it's time for another road trip.  This time, it's my first-ever visit to the People's Republic of China; I'll be spending two weeks in Beijing.  I'll be sure to hit the major sights--the Great Wall, the Palace Museum, Tian'anmen Square and the Forbidden City--but you know I'll be looking for the interesting stuff-on-the-street and delicious hole-in-the-wall restaurants that hide in every city.  Of course, there will also be Chinese beer, which will be something of an adventure in itself if the various reviews I've read are to be believed.

Nonetheless, I'm going to try to take the same approach as I did with my recent visit to Osaka.  I'll take candid shots with my Blackberry and upload them to Flickr for your entertainment and comment.  I'll also be tweeting with the #BeijingWes hashtag, if you want to search for it (or filter it!)...

I wonder if they'll sell me a copy of Mao's Little Red Book?  That would make an interesting conversation piece...

Ah, well, time to rest up for my flight; it's 13 hours, and I don't sleep on airplanes...

Wednesday, October 12, 2011

Data Throughput with HTTP, SMB and NRPC - It Isn't Just Network Latency

Customers are often puzzled by the "speed difference" seen when transferring data via Lotus Notes, as compared to HTTP or SMB file transfers; these differences are most often seen in conditions of high network latency, but they are always evident to some degree.  What most folks DON'T realize is that "network latency" is only an exacerbating factor; the real "root cause" reasons are inherent differences among the protocols.  Let's go deeper...

HTTP is, for all intents and purposes, a streaming protocol.  A single request, such as an HTTP GET, results in a (sometimes) lengthy, but uninterrupted, stream of data in response.  It's just "GET", "200 OK" and a blast of data; the protocol performs no metering or interruption of the inbound data flow.  Basically, it looks like this:

GET / HTTP/1.0

200 OK <followed by all of in one stream>

The limiting factors are, in this case, at the TCP/IP layer itself, in the form of small TCP windows, TCP's "slow start" behavior, and/or congestion avoidance mechanisms.

SMB has its own limiting factor - one that most users of "disk shares" don't know.  SMB/CIFS requests are limited to 64Kb in size; thus, any file transfer is a series of requests, each of which is up to 64Kb in size.  So, no matter how "fast" one's network might be, SMB still operates in 64Kb "chunks" and must issue a new request for each "chunk."  Transferring our now looks like this:

Create AndX

Create AndX

Read AndX offset 0x0 data 0xf000

Read AndX (64K chunk of data)

Read AndX offset 0xf000 data 0xf000

Read AndX (next 64K chunk of data)

(Repeat until all data transferred...for a 2Mb file, about 32 iterations)

Now, each of these is a network transaction, so now we have a full network round-trip added for each ReadX request, plus the 64Kb blocking overhead on both ends; you can see why large data transfers with SMB are more sensitive to network latency than are HTTP transfers. (The astute among you also see why I NEVER recommend the use of SMB shares for frequently accessed data, such as personal Notes mailfiles...)

Turning to Notes NRPC (aka "Lotus Notes"), we find ourselves somewhere between these two extremes.  Behind the scenes, Lotus Notes/Domino is a transactional database system; even your "mailfile" is a database, and what you see as "an email message" is actually dozens of entries in that database.  It's all fields and attributes, which are handled individuallly.  They're all individual transactions, which means that--while individual transactions may involve large volumes of data--there's a constant back-and-forth between client and server.  Even opening (or replicating) a single email message triggers "Give me this part"..."OK, now this part"..."I'm ready for this part"...and, while not subjected to any particular metering or "chunking" (as is SMB), each of those transactions still requires an extra round-trip between server and client - which is where network latency can hit and hit hard - just as it does with SMB.

Unfortunately, there are no easy solutions for such environments.  You can adjust TCP windows, and (in the case of Lotus Notes), increasing the size of the TCP port buffers can provide some relief, but you'll eventually come up against the laws of physics - the electrons can only move so fast...and now you know how choice of protocols can make a BIG difference in performance when the network is slow...

Tuesday, October 04, 2011

Tidying up the Home Office with Synergy

OK, my wife kind of likes the idea of me working from home, but she definitely did NOT approve of the maze of keyboards, mice and laptops that make our dining room look like a spaghetti factory (no, not this one).  I tried to satisfy her aesthetic sense with a KVM switch, but the dog and cat decided that the cables behind the table were JUST the place to conduct their mock battles.  To top it off, my wife actually looked BEHIND the was I to maintain efficiency AND earn Spousal Points?

Enter Synergy.  It's a wonderful little client-server app that farms one machine's keyboard/mouse/clipboard out across an entire range of systems.  Want to do all your big FTP stuff on that box over there?  Cut-and-paste across the monitors, and you're there.  Want to tidy things up further?  Shut the laptops and run on external monitors.  It's cross-platform, with binaries available for Windows, Macintosh, Debian/Ubuntu Linux and Fedora/Red Hat Linux.  The configuration (I used a WIndows machine as the server) was almost effortless; basically, you just define rules about the positions of one's monitors by their hostnames.  For example, one of my rules says "reliant is left of disruptor", and another says "galileo is right of disruptor"...and that's basically it.

Did I mention that Synergy is FREE?  If you want to clean up your workspace and get back to a single keyboard/mouse pair, make haste to the Synergy website, linked below.