robgillen.Blog()

As part of our cloud computing initiative we have been investigating the use of containerized computing and exploring if and where it might play a role in the computational environment where I work. In the context of this effort I have had the privilege of visiting a few different locations and seeing the containers first hand - an experience which has both answered and generated a number of questions for me.

We have a unique opportunity in that the SGI ICE Cube demo truck is going to be on-site here Thursday and Friday September 9th and 10th. During that time the container will be available both for walk-in traffic as well as pre-scheduled, in-depth presentations. While there are a handful of different container vendors and approaches, seeing one in person will give you a baseline and framework by which to analyze others.

For those of you not familiar with containerized computing, it (as an approach/concept, not necessarily this particular design) is being used in some of the largest datacenters being built and is a key component in Microsoft’s 3rd and 4th generation datacenter designs.

Some interesting characteristics of the SGI design (other vendors have other distinguishing features although there are some common threads such as high density, energy efficiency, etc):

• Provide the ability to operate at elevated temperatures (ambient – cold aisle - air of 85F)
• Incredibly high density, surpassing 46,080 cores within one container
• Reduced cooling cost
• 20’ or 40’ containers, can be stacked 3 to 5 high
• Dual row, universal and air-cooled designs
• Can contain compute, storage, network, and cooling
• Just plug in power, network, and chilled water (if needed)
  

SGI has recently added to their suite of designs a totally air cooled unit that simply requires a garden hose for intake water (read: “massive energy savings”).

More information on the SGI container can be found here: http://www.sgi.com/products/data_center/ice_cube/ and a PDF datasheet is here: Datasheet PDF

If any of you live near where I work, and are interested in seeing this in person, contact me and I’ll see what I can work out (note: you must be a US citizen).

Cloud Computing cloud, azure

I had a conversation with a friend last week who was messing around with a non-SOAP based HTTP service and was fighting with the C# necessary for rudimentary interactions. The problem was compounded by the fact that he needed to associate a certificate with the request to authenticate properly to the server.

I had recently been doing this exact thing based on some work with the Azure management API so I promised him some samples. As I was assembling them this morning, I decided to drop them here in case they could be beneficial to others.

The first sample shows a simple call wherein I build some XML and send it along with the request. In this case we are creating a deployment in Windows Azure. We then grab a value from the response header collection for use in the second call that has a simple request format but returns an XML blob in the body that I then parse to get the results I need.

Both requests are signed with an X509 client certificate – you’ll notice it referred to as “managementCertificate” – this is a variable passed in that was generated using the following code:

var managementCertificate = new X509Certificate2(manifest.CertificateFile);

Where manifest.CertificateFile is the path to the pem file on my local machine.

In the sample below, you’ll see the target URL built, some base64 encoding of some of the parameters (included just for completeness but just a requirement of the service I was calling). I then use a StringBuilder to build up an XML block and then setup the request with the certificate, xml blob, and other properties set. Finally, you’ll see the submission and then pulling a value from the headers collection to be sent back to the caller.

// Build uri string
// format:https://management.core.windows.net/<subscription-id>/services/
//                hostedservices/<service-name>/deploymentslots/
//                <deployment-slot-name>
var url = string.Format(
    "{0}{1}/services/hostedservices/{2}/deploymentslots/{3}",
    Constants.AzureManagementUrlBase,
    subscriptionId,
    serviceName,
    deploymentSlot);

// Base64 encode configuration label and file
var base64label = EncodeAsciiStringTo64(configurationLabel);
var base64config = GetSettings(
    instanceCount,
    accountName,
    accountKey,
    queueSleepTime,
    maxJobLength,
    container,
    queueName);

// build request body
StringBuilder blob = new StringBuilder();
blob.Append("<?xml version=\"1.0\" encoding=\"utf-8\"?>\n");
blob.Append("<CreateDeployment " +
    "xmlns=\"http://schemas.microsoft.com/windowsazure\">\n");
blob.AppendFormat("\t<Name>{0}</Name>\n", deploymentName);
blob.AppendFormat("\t<PackageUrl>{0}</PackageUrl>\n", packageUrl);
blob.AppendFormat("\t<Label>{0}</Label>\n", base64label);
blob.AppendFormat("\t<Configuration>{0}</Configuration>\n", base64config);
blob.Append("</CreateDeployment>\n");

// encode request body then put it in a byte array
byte[] byteArray = Encoding.UTF8.GetBytes(blob.ToString());

// make request
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);

// header info
request.Method = "POST";
request.ClientCertificates.Add(managementCertificate);
request.Headers.Add(Constants.VersionHeader, Constants.VersionTarget);
request.ContentType = Constants.ContentTypeXml;
request.ContentLength = byteArray.Length;

Stream dataStream = request.GetRequestStream();

// write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);

// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

// Get the x-ms-requestID
string requestID = response.GetResponseHeader(Constants.RequestIdHeader);

// Clean up the streams
dataStream.Close();
response.Close();

return requestID;

In this next sample, we make a rather simple request but do more with the result in parsing the returned XML blob which is fairly trivial although it does have custom namespaces which have to be accounted for when you crawl the XML tree.

// Build uri string
// format:https://management.core.windows.net/<subscription-id>/services
//          /hostedservices/<service-name>/deploymentslots
//          /<deployment-name/
var url = string.Format(
    "{0}{1}/services/hostedservices/{2}/deploymentslots/{3}",
    Constants.AzureManagementUrlBase,
    subscriptionId,
    serviceName,
    deploymentSlot.ToString());

// make uri request using created uri string
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);

// make header, method, and certificated requests
request.Method = "GET";
request.ClientCertificates.Add(managementCertificate);
request.Headers.Add(Constants.VersionHeader, Constants.VersionTarget);
request.ContentType = Constants.ContentTypeXml;

// Get the response
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

// put response into string text
StreamReader dataStream = new StreamReader(response.GetResponseStream());
string text = dataStream.ReadToEnd();

// create an xml document
XmlDocument xml = new XmlDocument();

// load up the response text as xml
xml.LoadXml(text);

// get the NS manager
XmlNamespaceManager ns = new XmlNamespaceManager(xml.NameTable);
ns.AddNamespace("az", Constants.AzureXmlNamespace);

// return the status
DeploymentStatus currentStatus;
var statusText = xml.SelectSingleNode("//az:Status", ns).InnerText;

if (Enum.TryParse<DeploymentStatus>(statusText, true, out currentStatus))
{
    FullDeploymentStatus fullStatus = new FullDeploymentStatus() 
        { MainStatus = currentStatus };

    // now try to get the status values for each instance
    XmlNodeList instances = xml.SelectNodes("//az:RoleInstance", ns);

    foreach (XmlNode instance in instances)
    {
        var instanceStatus = new InstanceDetails()
        {
            RoleName = 
                instance.SelectSingleNode("az:RoleName", ns).InnerText,
            InstanceName = 
                instance.SelectSingleNode("az:InstanceName", ns).InnerText,
            Status = (InstanceStatus)Enum.Parse(typeof(InstanceStatus),
                instance.SelectSingleNode("az:InstanceStatus", ns).InnerText)
        };

        fullStatus.Instances.Add(instanceStatus);
    }

    return fullStatus;
}
else
{
    throw new ArgumentOutOfRangeException("Status",
        "The status returned for the deployment is outside the range of " +
        "acceptable values");
}

That is about it. Hopefully this is helpful and give you more comfort interacting with HTTP-based services that aren’t simply a matter of pointing at a WSDL and having magic happen.

General Development samples, .net

On June 5, 2010, I had the opportunity to join just over 400 other folks and ride in the American Diabetes Association’s (ADA) Tour de Cure (TDC) in Knoxville, TN. The TDC is a fundraiser supporting the mission of the ADA in diabetes research and prevention. There were four ride distances available: 12.5 miles (family ride), 25 miles, 62 miles (metric century), and 100 miles (century). I elected to try the 100 mile ride as I had never participated in a ride of that length before and wanted the challenge.

As many of you know, I had planned on riding in honor of my mother-in-law, Ann Black, who had type-2 diabetes for over 25 years. In April Ann passed away, causing the ride to have more importance as a memorial and tribute to the strength and perseverance she demonstrated over the past 10 years in her fight with cancer and subsequent/related health struggles. It was great to be able to have Gary Black (Ann’s husband of over 40 years) present with Julie and the kids as I crossed the finish line, just over 6 hours after having started the ride.

On the fundraising front, I cannot say thanks enough to those who supported me in this effort. Together we raised over $1,350 which made me the 14th highest individual fund raiser. The team I was associated with (UT-Battelle – from the Lab where I work) was the 10th highest with just under 5,000 raised. These numbers may not seem that large, but these were small parts of the entire TDC Knoxville effort that raised over $148,000. This is a great number and I’m thrilled to have been a small part.

The ride was certainly challenging, and I’m glad I did the 100 mile distance. While it was a ride and not a race (a distinction that may not mean much to most, but basically means that timing is less precise and placement isn’t tracked that closely) I was able to learn that there were approximately 45 individuals who started the 100 mile  route and I was the 9th across the finish line with a personally-recorded time of 6:09:05. During that time I climbed a total of 6,128 feet, burned 6,578 calories, had an average speed of 16.2 mph, and drank alot of liquid. If you want, you can view the data collected by my bike computer (GPS/map data, etc.) here: http://connect.garmin.com/activity/35784608. As you can imagine, I learned quite a bit about long distance rides. While I consumed a significant amount of liquid, I had my hydration approach skewed a bit and didn’t consume enough salt compared to what I lost. This resulted in some pretty severe cramping in my legs that caused me to have to stop a few more times than I had intended. I also should have paced myself better as I started off a bit strong and tapered pretty badly, especially in the last hour. I probably should have planned my stops ahead of time and stuck to them regardless. As it was, I decided to stop “when I felt I needed it” and didn’t end up stopping at all until mile 62 and at this point I was probably a little past the point of proper salt consumption and it was a little late to fix it.

All that said, it was a great ride and I really enjoyed participating in it. A special thanks again to all who were involved in supporting me in this – it was certainly a team effort and I couldn’t have done it without you. This last picture is me crossing the finish line… pleased to be done.

I spoke at CodeStock (http://codestock.org) a few weeks ago and one of my talks was focused on AWS from the perspective of the .NET developer. The slides are available here:

And the video of the session is available here:

I just finished reading an article (http://searchcloudcomputing.techtarget.com/news/article/0,289142,sid201_gci1512394,00.html) discussing some of the power issues and related outages at one of Amazon’s (http://aws.amazon.com) data centers last week. While much of the article was fine and factual, I take a bit of issue with the way the article wraps up:

Users may not like being told they should fend for themselves on disaster preparedness, but that appears to be part of the price for getting everything else AWS offers.

This highlights a sentiment that is unfortunately pervasive within the community of those evaluating or adopting cloud computing – that of believing that cloud computing is a panacea for all scale and datacenter problems.

What the users of these platforms need to understand is that they are toolkits. While the various cloud computing vendors provide important services and features, the consumer of said platforms must do their homework to understand the technical tradeoffs of various decisions so that they can appropriately reap the benefits of the selected platform. Simply uploading your code/application and expecting it to be always available is unrealistic. The consumer must understand what high availability features are offered by their particular cloud vendor and exploit those features to ensure that their app has the appropriate availability. In the case of the Amazon outage(s), if users had followed the high-availability guidelines provided by Amazon, they would not have experienced any outage at all. Cloud providers such as Microsoft, Amazon, and others provide the notion of availability zones, or regions, and – much like you would if you were hosting the app yourself – you need to distribute your application across such to ensure that a failure in one location doesn’t mean a complete outage for your application.

Rather than a magic wand that solves all scaling and availability issues, cloud computing provides a democratized toolset that informed consumers can use to develop a highly available, scalable, and fault-tolerant application. The key word here is “democratized” – meaning – these features are available to anyone, at a fraction of the cost of doing it yourself. I experience similar frustration when reading complaints from folks about the pricing of Windows Azure (i.e. “Why can’t I host my simple website there fore $10/month?”). The question illustrates that the inquirer doesn’t understand the fundamental architecture of the platform (both how it works, and what its primary use cases are). Neither Amazon’s EC2 nor Windows Azure are designed to compete with a low-cost web hoster… rather they are designed to provide the tools by which a company that needs features not available from a low-cost hoster, but doesn’t have (or wish to spend) the capital to build those features themselves.

They are great platforms that provide you the ability to build a very solid offering, but you have to understand how to properly utilize those features. Cloud computing should not be approached with ignorance or any less planning than you would if you were building out the infrastructure yourself (of course the level of detail will differ).

Cloud Computing, Theory

I’m wrapping up a bit of the work we’ve been doing on data movement optimizations for cloud computing and the latest set of data yielded some interesting points I thought I’d share. The work done here is not really rocket science but may, in some ways, be slightly counter-intuitive and therefore seemed worthy of posting.

Summary: for those who don’t like to read detailed posts or don’t have time, the synopsis is that if you are uploading data to Azure, block your data (even down to 1MB) and upload in parallel. Set your block size based on your source file size, but if you must choose a fixed value, use 1MB. Following the above will result in significant performance gains… upwards of 10x-24x and a reduction in overall file transfer time of upwards of 90% (eg, uploading a 1GB file averaged 46.37 minutes prior to optimizations and averaged 1.86 minutes afterwards).

Detail: For those of you who want more detail, or think that the claims at the end of the preceding paragraph are over-reaching, what follows is information and code supporting these claims. As the title would indicate, these tests were run from our research facility pointing to the Azure cloud (specifically US North Central as it is physically closest to us) and do not represent intra-cloud results… we have performed intra-cloud tests and the overall results are similar in notion but the data rates are significantly different as well as the tipping points for the various block sizes… this will be detailed separately).

We started by building a very simple console application that would loop through a directory and upload each file to Azure storage. This application used the shipping storage client library from the 1.1 version of the azure tools. The only real variation from the client library is that we added code to collect and record the duration (in ms) and size (in bytes) for each file transferred. The code is available here.

We then created a directory that had a collection of files for the following sizes: 2KB, 32KB, 64KB, 128KB, 512KB, 1MB, 5MB, 10MB, 25MB, 50MB, 100MB, 250MB, 500MB, 750MB, and 1GB (50 files for each size listed). These files contained randomly-generated binary data and do not benefit from compression (a separate discussion topic). Our file generation tool is available here.

The baseline was established by running the application described above against the directory containing all of the data files. This application uploads the files in a random order so as to avoid transferring all of the files of a given size sequentially and thereby spreading the affects of periodic Internet delays across the collection of results.  We then ran some scripts to split the resulting data and generate some reports. The raw data collected for our non-optimized tests is available via the links in the Related Resources section at the bottom of this post.

For each file size, we calculated the average upload time (and standard deviation) and the average transfer rate (and standard deviation). As you likely are aware, transferring data across the Internet is susceptible to many transient delays which can cause anomalies in the resulting data. It is for this reason that we randomized the order of source file processing as well as executed the tests 50x for each file size. We expect that these steps will yield a sufficiently balanced set of results.

Once the baseline was collected and analyzed, we updated the test harness application with some methods to split the source file into user-defined block sizes and then to upload those blocks in parallel (using the PutBlock() method of Azure storage). The parallelization was handled by simply relying on the Parallel Extensions to .NET to provide a Parallel.For loop (see linked source for specific implementation details in Program.cs, line 173 and following… less than 100 lines total). Once all of the blocks were uploaded, we called PutBlockList() to assemble/commit the file in Azure storage. For each block transferred, the MD5 was calculated and sent ensuring that the bits that arrived matched was was intended. The timer for the blocked/parallelized transfer method wraps the entire process (source file splitting, block transfer, MD5 validation, file committal). A diagram of the process is as follows:

We then tested the affects of blocking & parallelizing the transfers by running the updated application against the same source set and did a parameter sweep on the block size including 256KB, 512KB, 1MB, 2MB, and 4MB (our assumption was that anything lower than 256KB wasn’t worth the trouble and 4MB is the maximum size of a block supported by Azure). The raw data for the parallel tests is available via the links in the Related Resources section at the bottom of this post.

This data was processed and then compared against the single-threaded / non-optimized transfer numbers and the results were encouraging. The Excel version of the results is available here.

Two semi-obvious points need to be made prior to reviewing the data. The first is that if the block size is larger than the source file size you will end up with a “negative optimization” due to the overhead of attempting to block and parallelize. The second is that as the files get smaller, the clock-time cost of blocking and parallelizing (overhead) is more apparent and can tend towards negative optimizations. For this reason (and is supported in the raw data provided in the linked worksheet) the charts and dialog below ignore source file sizes less than 1MB.

(click chart for full size image)

The chart above illustrates some interesting points about the results:

• When the block size is smaller than the source file, performance increases but as the block size approaches and then passes the source file size, you see decreasing benefit to the point of negative gains (see the values for the 1MB file size)
• For some of the moderately-sized source files, small blocks (256KB) are best
• As the size of the source file gets larger (see values for 50MB and up), the smallest block size is not the most efficient (presumably due, at least in part, to the increased number of blocks, increased number of individual transfer requests, and reassembly/committal costs).
• Once you pass the 250MB source file size, the difference in rate for 1MB to 4MB blocks is more-or-less constant
• The 1MB block size gives the best average improvement (~16x) but the optimal approach would be to vary the block size based on the size of the source file.

 (click chart for full size image)

The above is another view of the same data as the prior chart just with the axis changed (x-axis represents file size and plotted data shows improvement by block size). It again highlights the fact that the 1MB block size is probably the best overall size but highlights the benefits of some of the other block sizes at different source file sizes.

This last chart shows the change in total duration of the file uploads based on different block sizes for the source file sizes. Nothing really new here other than this view of the data highlights the negative affects of poorly choosing a block size for smaller files.

Summary

What we have found so far is that blocking your file uploads and uploading them in parallel results in significant performance improvements. Further, utilizing extension methods and the Task Parallel Library (.NET 4.0) make short work of altering the shipping client library to provide this functionality while minimizing the amount of change to existing applications that might be using the client library for other interactions.

Related Resources

• Source code for upload test application
• Source code for random file generator
• ODatas feed of raw data from non-optimized transfer tests
• Experiment Metadata
• Experiment Datasets
• 2KB Uploads
• 32KB Uploads
• 64KB Uploads
• 128KB Uploads
• 256KB Uploads
• 512KB Uploads
• 1MB Uploads
• 5MB Uploads
• 10MB Uploads
• 25MB Uploads
• 50MB Uploads
• 100MB Uploads
• 250MB Uploads
• 500MB Uploads
• 750MB Uploads
• 1GB Uploads
• Raw Data
• OData feeds of raw data from blocked/parallelized transfer tests
• Experiment Metadata
• Experiment Datasets
• Raw Data
• 256KB Blocks
• 512KB Blocks
• 1MB Blocks
• 2MB Blocks
• 4MB Blocks
• Excel worksheet showing summarizations and comparisons

Cloud Computing, Theory, General Development azure, cloud

Today marks my 10-year anniversary at eQuest/Planet Technologies. It almost seems odd to write that… 10 years seems like a long time, and particularly to be at the same company in the Internet/Technology field.

During my tenure at Planet, I’ve had the privilege to work and cross paths with a number of great people – too many to name specifically but I do want to mention Scott Tucker, Steve Winter, and Dan Nelson. These men are passionate about their work and a pleasure to work with. I’ve also had the opportunity to hold a number of different roles ranging from “server build guy” to dev team lead, to world traveler, and now get to play around as a research scientist. The work has rarely (read: “never”) been boring and it seems like nearly every day presents another opportunity to solve a difficult challenge.

Today doesn’t represent any sort of change, or adaptation in my career path (I’ve never been particularly good at/fond of career planning). I simply am grateful for the opportunities I have had and am looking forward to what the coming years hold.

Miscellaneous

I’ve been reading a paper this morning published by Microsoft Research on Quality of Service Aware Clouds. If you are engaged in the cloud computing field, I would suggest that it is worth the time to read (14 pages) if for no other reason than to get your mind rolling (as it did mine) on the topic. Further, I’d be keenly interested in follow-on conversations from the community as to the issues/remedies put forth in that paper.

I’m finding myself split on the topic… academically, there are some interesting points being made:

• VMs from different customers running on the same physical host can negatively impact each other (think, last-level cache contention, memory bandwidth, I/O paths, etc.) (this isn’t a surprise to anyone, just the premise on which the rest of the paper is based)
• They suggest an intelligent VM placement algorithm based on resource utilization models of the applications running in the VMs
• They suggest reserving a certain amount of headroom on each physical host to allow for dynamic compensation and CPU throttling adjustments to maintain QoS
• They suggest addressing the “wasted” resources represented by the headroom in the normative case by means of a bid-based “higher quality” service level available to users for additional fees
• The key premise is that many apps hosted in the cloud are *not* CPU bound, and putting those that are along side those that are not will provide everyone with a reasonable level of service (as compared to putting all CPU-bound VMs on the same nodes resulting in resource contention issues for those nodes whereas other nodes with lighter workloads are, effectively, sleeping).

However, I find myself struggling with a few things:

• The solution they propose is based on the ability to accurately model the workload of a given VM (they suggest in a non-contended staging area) and to then to the initial production VM placement based on a balancing algorithm. I struggle with this as I think the majority of cases are going to be either “quick hits” (user is just posting a handful of VMs to use for awhile after which they will be torn down) or scenarios in which the cost of eeking out 100% QoS consistency is going to be greater than simply “spinning up another VM”. I’m guessing that the later case is more likely, and that most users will simply accept the balanced performance for what it is, and adjust their total number of VMs accordingly. The exception to this rule will likely be permanent (or nearly so) installations (wherein the cost of running nodes for a long time is higher)
• They propose that the cloud providers insert “head room” into their resource deployment strategy (as a means of compensating for contention) and then mitigate this “waste” by selling higher levels of Quality to interested customers who are willing to bid if you will for increased performance when available but content to live with a lesser service level in the normative case. I struggle with this as it, in my mind, inserts another level of complexity to the pricing model of cloud computing that simply will not survive in the market place. From the vantage point of the user, this would further introduce variance into my overall QoS as, the normative case *might* be that I run at the higher service level (normally the host is not experiencing high levels of interference) but I will be dropped to the lower level without notice. This could add significantly to my auto-scaling complexity as I now no longer need to scale up/down simply based on traffic or load, but also have to monitor the QoS state I’m currently being provided by the cloud provider.
• They discuss using market-based “bidding” for higher QoS levels which I think is problematic due to the fact that they are, in this case, asserting that Quality of Service 0 (Q0) (the normative state) is something less than a full core (in their examples, something around 50%) and that Q1 is higher (maybe 75%). The problem here is perception vs. reality in that most users of cloud platforms would assume that when they get a single-core VM, they are getting ALL of that VM. Therefore, asserting that what you are really getting is 50% of said core, and that if you pay more, we’ll periodically give you more (except for when we don’t due to someone else on the same host over utilizing their portion) seems a difficult sell.
• The algorithm and approach targets the “behaving” code rather than punishing the misbehaving code. Rather than determining which VM on the chip is overrunning the rest and curtailing that, they attempt to simply help those who are under performing.

I think, that in the end, I’m more in favor of simply having more intelligent hypervisors that provide better isolation for VMs, but I’m still thinking this all through. There are some interesting points made in this paper, and intelligent allocations could be interesting…

Cloud Computing, Theory

I am in Redmond this week and am participating in two workshops being hosted by different groups within Microsoft Research. Along with a handful of others, I was asked to participate in a panel discussion on Friday dealing with new experiences that cloud computing would facilitate, as well as things we felt were road blocks to seeing those experiences realized. He specifically challenged us to think "outside the box" and to look beyond (the now typical) conversations surrounding raw performance and to dream a little. I wrote out the following as a means of working through my thoughts for my 5-7 minute portion of the panel discussion and, as it took me longer than 7 minutes to read, I thought I'd post it here as a expansion of the talk and possibly an anchor on which to hang subsequent conversations. Please forgive the casual nature of the talk as it is intended to be, essentially, a script read delivered to a group rather than a formal written version of the same.

---

This topic is certainly interesting to me as I am convinced that cloud computing is here to stay and also presents a platform that can be disruptive to the scientific/technical computing industry (although I would qualify this by saying “disruptive in a constructive sense” – meaning that the disruption leads to the additive good and not the removal of existing work). I have spent a considerable amount of time over the past week contemplating this question (how do we imagine cloud computing facilitating new usage scenarios), and have chosen to present my reply by means of a few examples.

The first example is that of Lego MindStorms. Are you familiar with these? They are kits that provide kids (regardless of how old they are :) ) the ability to build robots using a familiar (although slightly altered) Lego metaphor. These kits come with motors, sensors, and a "brain" that is programmable via a drag-and-drop software tool but also supports more complex tools such as Microsoft's Robotics Studio. Do you know what is so great about these (besides the obvious)? They allow common people, with no prior robotics or electronics experience, to dabble in the field. It is, a gateway, if you will, to a much broader field.

The second example is more of an experience that happened to me recently in that I had the privilege of running into my high school science teacher this past weekend - a quiet, rather unassuming fellow named Randy White. Randy's brilliance is that he has a passion for science and did (at least in my case) an excellent job of transference. If I am ever able to accomplish anything interesting in the scientific domain, a large portion of the credit will lie with him. Probably the most important thing he taught us, was how to think about, or to tackle the complex. I can't tell you how many times I heard him say, "Start with what you know". The idea being, that most often, incredibly complex problems were comprised of nothing more than a series of far simpler, and additive problems. He taught us to focus on solving what we could, rather than attempting to "swallow the entire elephant" if you'll allow me to strain a metaphor.

If you find yourself wondering what these two examples have to do with each other, or more germanely, what do they have to do with my vision for the scenarios that cloud computing will open, let me see if I can explain...

You see, much in the same manner as Lego MindStorms have introduced an otherwise unlikely audience to the world of robotics, I believe that cloud computing (based on its cost model and popular programming paradigms) is a means of introducing normal people (and by this, I mean those not formally trained in scientific or technical computing) to the notion of using computation as a tool for solving complex problems. Possibly to the dismay of some in the field, I think that this will, at least initially be done in a means void of the topics of MPI, or Fortran, much in the same way as a 15 year old "programming" his robot doesn't have to understand the inner workings of concurrency runtimes nor the physics at work when his robot "walks" for the first time. I will be the first to admit that these (MPI, Fortran, concurrency topics, race conditions) are important topics, but I would submit that they should not be gating factors to one's ability to explore the arena and determine if he/she is interested in further study in that field. I think we will see paradigms that are far simpler to adopt, such as master-worker, map/reduce, etc. (or even cloud-backed applications that are hidden behind more accessible tools such as Excel, or MatLab) take hold in significant ways and that we will see the development of novel approaches to solving problems using this new platform. The tired-and-true tools will remain, and will be used when necessary and appropriate, but I think if we force them down the throats of the next generation of researchers as "the only way to accomplish science", we are doing them a great
disservice.

As to where Mr. White and high-school science comes into play - well, this can best be summarized by a comment made by a friend of mine, Wally McClure when he, almost flippantly, referred to Windows Azure as a "poor man's supercomputer". Being one that had been working with Azure for quite a bit at the time, I took a little offense at the accolade due to its semi-pejorative nature, and prefer the "common man", but the point is the same regardless: Cloud Computing (at least as currently manifested in both Windows Azure and Amazon's AWS platform), has a great potential to democratize high-performance computing. You see, the high-school I grew up in was small... we had 23 in my graduating class. While Randy has moved on, he still teaches in a comparatively small school that certainly has no funds for a cluster on which to run experiments. However, with the advances in cloud computing, Randy could devise a collection of simple experiments and actually execute them as part of a class project. He could have a significant computational cluster for the equivalent of a few dollars. He can present "Scientific" computing as something obtainable to his students, and hopefully foster an interest that will develop into the next generation of computational thinkers - solving one problem at a time, incrementally, on the way to solving massive problems that we have trouble even describing today.

It is, in my opinion, incumbent upon us - the current generation of computational researchers and domain-specific scientists - to look at cloud computing not as a threat to the establishment, but as facilitating a new means of scientific discovery. We should consider ways to make large-scale computation more accessible to "normal" people. We should be opening up the community, sharing wherever possible, reducing the barriers to entry. Challenge yourselves and your students to push boundaries, to consider non-traditional approaches, and to enjoy "playing" with computational resources.

Conferences, Theory, Cloud Computing

The danger with using a framework is that sometimes it does things that you aren’t aware of that can send you in circles for  quite some time before you figure them out.

I’ve been working on some tests of some “creative” ways to get data in and out of cloud platforms at rates above the norm and I’ve written a test harness that I’ve been using that will grab a bunch of files, one at a time, and record the file size, duration, etc. for the transfer. I’ve been doing this in a single-threaded fashion for quite some time with reasonable success. The problem began when I attempted to use to run a test that did multi-threaded downloads (multiple threads each grabbing a portion of the file).

NOTE/Crazy Quirk: I don’t yet know why this is the case, but the problem I’m preparing to explain did *not* appear while I had Fiddler running… only when it was *not* running. I’m guessing that this is due to some “magic” that Fiddler does to the HTTP/networking stack..

The behavior I was seeing, was that after two threads would execute, all subsequent threads would fail or timeout. Obviously, when one is doing a significant amount of data movement, this is sub-ideal. The culprit turned out to be the ServicePointManager’s DefaultConnectionLimit. By default, this is configured to 2 which means you can, at most, have 2 open connections to the same TLD at the same time. When I was doing this in serial, there was no problem as the connections were managed/re-used on the main (only) thread.  When doing a number of operations to the same URL (TLD) from multiple threads (especially when you are setting up/tearing them down quickly), it appears that the ServicePointManager is unable to re-use them (not surprising) but neither is it able to determine that the thread is now gone as should be the connection count. (yes, I was behaving and closing my connections).

The solution I came up with was to first shorten the time to live for idle threads, next to monitor the number of threads currently “consumed” and to increase the limit based on how many I needed for the current operations, all while ensuring an upper bound and stand-off mechanism should things get too far out of bounds.

// ensure that we don't have lingering connections that will hamper our ability to continue...
// Start by getting the ServicePoint for our current Url 
ServicePoint servicePoint = ServicePointManager.FindServicePoint(new Uri(url));

// see how many connections currently exist...
int existingConnections = servicePoint.CurrentConnections;

// if we are above our upper bound, wait a bit to let things settle down...
while (existingConnections >= 64)
{
    Console.WriteLine("Connection count too high... sleeping for a bit...");
    Thread.Sleep(1000);
}

// ensure that we have enough room to do what we need
if ((existingConnections + options.ConcurrentThreads + 1) > servicePoint.ConnectionLimit)
{
    servicePoint.ConnectionLimit = existingConnections + options.ConcurrentThreads + 1;
}

// only give them a few seconds (5) to time out...
ServicePointManager.MaxServicePointIdleTime = 5000;

Console.WriteLine("Pre-Existing Connections: {0}", existingConnections);
Console.WriteLine("Connection Limit: {0}", servicePoint.ConnectionLimit);

Hopefully, this will be helpful for someone else hitting the same issue.

Miscellaneous c#, tips, tricks, tools