InterClypse

We promise a vibrant and passionate presentation outlining an open source six step initiative named Cultivation (teamcultivation.org) that helps successfully place junior professionals into various work environments. The founder, Brian Walsh, has used this innovative process to help place junior software engineers within the Intelligence Community; however, this process can be used across multiple verticals to include the DOD, health-care, etc..

Additionally, with sequestration and the fast pace of technology there is a trend toward automated testing and Technical Software Configuration Management (TSCM technicalsoftwareconfigurationmanagement.org). Presently, these two tasks are performed manually which causes them to be very time intensive and worse than that they are often the slow points in trying to achieve our customer's mission. In this presentation, we are going to outline a path for Cultivating our personnel and processes toward the automated time and mission saving approach.

Who should attend this presentation and why. Your business development team might want to consider using Cultivation and TSCM on upcoming proposals as differentiators. Small business owners and other executives that want to create a cultivating environment within their company. Project managers and technical leads can also use these concepts and initiatives on a smaller scale to achieve success. In the end, Cultivation promises to help a broad spectrum of businesses successfully hire and retain junior talent while building the superstars of the future

Congressman Elijah Cummings leads off the presentation on March 28th followed by a briefing on Cultivation and Technical Software Configuration Management (TSCM) from Brian E. Walsh the CTO of Interclypse, followed by an update on the BASE Business Initiative (BBI) and networking activities. These meetings are free, open to the public, and require no RSVP. 

On March 15, 2013, two Interclypse Software Engineers attended Puppet Camp Baltimore hosted by Puppet Labs (https://puppetlabs.com/).
Puppet Camps are community-oriented, regional gatherings of Puppet users and developers. This provides an opportunity to talk to a diverse group of Puppet users, benefit from presentations delivered by prominent community members, and be able to share experiences and discuss potential implementations of Puppet with your peers. Special guests included James Turnbull (Author of "Pro Puppet" and VP of Technology Operations), Ryan Coleman (Puppet Forge Blacksmith) and various Puppet associates whom discussed the current state and future of Puppet. Interclypse visited Puppet Camp to refine our use of Puppet when configuring Open Stack (http://www.openstack.org/). This will play a key role in Interclypse new data center build and design, which will showcase Puppet’s capabilities and the partnership between Interclypse and Puppet Labs.
 

Recently, Interclypse, Inc., attended the MongoDC 2013 conference - an annual appearance of the open-source, no-SQL community surrounding 10Gen's product, MongoDB. The conference was hosted at the very-cool Newseum in downtown Washington DC. What can I say about Mongo...what can't I say about Mongo!? If you are in the data management arena and you haven't played with MongoDB or any other no-SQL solution yet, what are you waiting for?
The Conference
First, about the conference: I attended only the general event (just to set the context). Obviously, there were a range of presentations, but these were the ones that I found to stand out:
1. Max Schireson's (the CEO of 10Gen) discussion on aggregation and indexing was excellent. First of all, the CEO gave it! Outstanding! The CEO is a technical guy with plenty of database experience - both structured and unstructured. Max was able to demonstrate new features of the MongoDB product that really make it a disruptive force in the data management /analysis market. Chief among these is the ability to aggregate documents given different criteria. This would be like constructing a view or a join. In addition, to producing new aggregate documents at query time was the ability to chain this creation to other mutations, such as sums and sorts. For example, let's assume the following JSON documents in the same collection:
{"_id: "1",
"org" : "JUG",
"state":"MD",
"size":"200"}

{"_id: "2",
"org" : "MUG",
"sponsors":["company1","company2"],
"state":"MD",
"membership":120}

{"_id: "3",
"org" : "JUG",
"state":"DE",
"membership":21}

{"_id: "4",
"org" : "JUG",
"state":"CA",
"membership":2000}

The following query would produce an aggregate from this collection:
db.collection.aggregate({$group : { _id : "$org", totalMembership : { $sum : "$membership" } } }, { $match : { org : { $eq : "JUG"} } })

This is equivalent to the SQL statement:
SELECT org, SUM(membership) as totalMembership FROM collection
GROUP by org
WHERE org is "JUG";

{"_id: "JUG",
"totalMembership": 2221}

...where the results are grouped and sorted. As you can see, the aggregate is a reduced set despite the fact that one of the documents having slightly different structure. The "." notation acts as a "|" or pipe function, similar to that found in UNIX shells, allowing for the chaining of operations.

Finally, Max covered index construction and optimization, how not to index everything, and finally how to profile your queries through the "explain plan" to understand what indices’, if any, are being used during your queries and when.
2. Rob Moore's presentation on tricks-to-teach MongoDB showcased potential depth with the product. Imagine MongoDB acting as a pub-sub message broker! Building upon his asynchronous (and super-fast) Mongos client, he was able to demonstrate callback functionality within Mongo so that modification events to a document or collection were passed back to subscribed mongos clients much a JMS broker would do so. Although this is not part of the supported product, it did entice us to dream of certain possibilities.
3. Auto-Sharding and matters of scale. The sharding capability of MongoDB makes it attractive and a viable alternative to some big data solutions! Now, with MongoDB 2.4 release, the pain of shard management has gotten easier with automated shard extension support. So, as my Mongo cluster grows and I need to add more members to my shard collections, I "simply" startup a new MongoDB server instance and add it as a member of my shard collection. The distribution of key space across the new member is handled incrementally during normal shard collection rebalancing. Of course, if you didn't know, MongoDB also has a mapReduce implementation using JavaScript. Before you bash it, consider the performance of mapReduce across a shard collection filtered by time where all the working documents are in memory - yep, that's how MongoDB rolls. MapReduce, given your use case and tuning, can plow through documents faster than you would expect. (Sorry for the caveats, but that's life, right?)
There were other topics discussed, of course - spatial querying, security, a RedHat OpenShift highlight, and others. One thing new I did do that day was connect to the mongodb IRC on FreeNode through a Chrome add-in and help someone with their document "_id" (the shard key) strategy. Yeah old-school collaboration!