Architecture enabling business

Low Ceremony Architecture

Posted in Uncategorized by jeromyc on December 7, 2010

Below is the abstract that I submitted as a presentation proposal for the Software Engineering Institute‘s SATURN 2011 conference.

(more…)

Measuring Software Productivity

Posted in Uncategorized by jeromyc on August 20, 2010

An interesting Twitter conversation I had with Martin Fowler yesterday.

Martin: Just ran into a request to measure productivity of software projects using Line of Code per hour. *sobs* #zombieswontdie

Me: @martinfowler But software productivity IS measurable, in context. http://bit.ly/9ZqidP

Martin: @sjcarriere I disagree: http://www.martinfowler.com/bliki/CannotMeasureProductivity.html

Me: @martinfowler I expected. :-) I believe there are meaningful measures that we can use to improve productivity within the software process.

Martin: @sjcarriere true, but that’s very different to saying you can measure productivity itself.

Me: @martinfowler You have to measure something to improve it. For software productivity, it starts with the long-term value of the output.

(Aside: there’s got to be a better way to transcribe Twitter conversations…)

Keyboards

Posted in Uncategorized by jeromyc on August 15, 2009

I’m a keyboard junky.  I have dozens. I sometimes switch between keyboards multiple times during a single day.  Yes, it’s a problem.

Here are my four favorites.  Clearly, I like the tactile feedback of clicky keyboards:

  1. IBM Model M – the most obnoxious for my neighbors, but with the best feel ever.  Unusable if you have a headache.  One of two keyboards on my list with a Wikipedia page.
  2. DAS Keyboard Professional – came second because I think the keyclick is a bit high-pitched, and my spacebar squeaks.  I don’t need the key labels, but I’m not geek enough, I guess, to go blank.
  3. NMB RT8255C – good click, solid. This keyboard is so obscure I couldn’t even find something to link to.   I came across this keyboard at a business center in the Albany airport, and it took me several years to find one (I bought two, actually).   My one complaint is the weird placement of the backslash key, in the top row, left of backspace.
  4. Kinesis Advantage – something completely different, for when the clicking gets to be too much, or my wrists start to hurt. I’ve got the cool Qwerty/Dvorak one, just for the key labels.  I’ve been typing Dvorak for 20 years, and this is my first and only Dvorak-labeled keyboard.

And the runners up:

  1. Microsoft Wireless Entertainment Desktop 7000 – love the action, hate the F-keys.
  2. Dell AT101 – the “Black Alps” switches are quieter than the “buckling spring” switches in the Model M, and I find the keys a bit “loose” – they rattle.

Finally, the ones you can’t pay me to use:

  1. Logitech MX3200 – totally flat, uninspired.  Not sure why I bought it.
  2. The new Apple Keyboard – I don’t even know where to start.

Just thought I’d share.

It’s pronounced like “lamb”, not like “lame”

Posted in Uncategorized by jeromyc on May 11, 2009

With that out of the way, it’s time for an update on the Lightweight Architecture Alternative Assessment Method (if you’re judging – negatively – the quality of the acronym, you wouldn’t be the first), which I first blogged about four years and three jobs ago.  Since then, LAAAM has undergone some refinement, shaped by collaboration and experience in its use at Fidelity and VistaPrint. This week is Microsoft TechEd, and  I’m giving a presentation on LAAAM this Tuesday at 10:15AM: ARC314.  I’m also doing a Tech Talk Interview with Bob Familiar on Thursday.

Why do LAAAM?

The first question should actually be: “why evaluate architectures?” Generally speaking, analyzing architectures gives us an opportunity to make good decisions early in the software development lifecycle, reducing risk and improving the quality of what we deliver. This benefit is possible because architectural decisions – the earliest decisions in a system’s life – are the primary determiners of quality: performance, scalability, reliability, flexibility, and so forth.  Unfortunately, the state of the practice in making architectural decisions is most senior/loudest/most patient debater wins. Sometimes this produces good decisions, but it’s not very predictable or repeatable, and we don’t usually end up with a clear and documented rationale.

LAAAM’s first, and perhaps most important, contribution to the software development lifecycle is its focus on defining what quality means for a given system or product.  The "–ilities" are commonly used, coarsely, to talk about architectural quality, and they usually give us a nice warm-and-fuzzy; who wouldn’t want a system that exhibits scalability and reliability and flexibility and maintainability?

The second important contribution of LAAAM is the way it brings together stakeholders –  i.e. anyone that cares about the outcome of a decision – from the business,the development team, operations, QA, etc.  LAAAM helps achieve a common language and decision framework. LAAAM helps to build consensus. That doesn’t mean everyone has to agree on everything, but it does mean that everyone’s perspective is on the table and the final result represents a fusion of those perspectives into a rational whole.

Finally, LAAAM produces a set of artifacts that represent a rigorous, rational decision-making process. These artifacts are extremely valuable in justifying architectural decisions; yes, ideally all of the stakeholders in those decisions would participate in the process as described above, but obviously that’s not always possible. So the LAAAM artifacts give us a way to express the reasoning that went into a given decision, with a foundation in the definition of quality for a system.

Quality

The first, and most important, step in LAAAM is to figure out what quality means for a given decision.  In other words, when all is said and done, and we look back on the decision, what’s going to make us say “that was the right decision”?  The –ilities give us a starting point, but they’re not enough.  LAAAM takes a page from the ATAM playbook here, and has us build a quality tree (ATAM calls it a utility tree, and LAAAM used to, but the term confused people) to structure our thinking.  This is easiest by example:

image 

Here, performance (an honorary –itility) and maintainability are the two important quality attributes for whatever decision we’re trying to make (yes, there are always more than two in the real world).  Performance has been refined to have two sub-qualities: response time and scalability.  You get the idea.

Scenarios

Now the punch line.  LAAAM doesn’t let us stop here, but rather forces another level in the tree: scenarios.  Scenarios (also borrowed from ATAM) give us the mechanism to be precise in defining quality, by ensuring each of our quality attributes are refined to a point of measurability.  In other words, we can look at the alternatives we’re considering and assess how well they achieve a scenario.  You can’t do this when you stop at “flexibility” or “scalability’”.  Scenarios have a consistent structure: a context, a stimulus and a response.  The context is the state of the environment when the scenario is applied, the stimulus is something happens to the system, and the response is how we want the system to react.  Again, by example:

image

A couple more:

  • The system network is partitioned and restored to normal condition; complete database resynchronization occurs within 30 minutes.
  • For a new release, integrate a new component implementation within two days.

The context/stimulus/response identification is left as an exercise (hopefully trivial) for the reader.  But note that the last example is different from the first two: it doesn’t apply at run-time, but rather at development time.

(Sidebar: while I’ve presented this a top-down exercise – first level quality attributes, second level quality attributes, scenarios – it’s not always done this way; in fact, it’s probably rare.  More often, the quality definition comes cyclically – you start with a few first level quality attributes that you think are important, someone throws out a scenario or two, you add some second level quality attributes to group them, and so on.  LAAAM doesn’t insist on any specific approach to building the quality tree, so long as it gets done.)

Now, let’s put some scenarios into our example quality tree:

image

Great, I want it all!  But wait …

Knowing What’s Important

Unfortunately, quality attributes trade off against one another.  There come times when you have to choose to achieve versus another, and it’d be nice to know in advance how to know which to is the right one.  In the most general case, we could try to weight all of our scenarios, but this is really hard, especially when trying to get people with competing concerns to agree.  Instead, LAAAM follows an approach of ranking at each node in the quality tree, like so:

image

So performance is more important than maintainability, scalability is more important than response time, and the steady state scenario is more important than the peak load scenario.  Easy, right? Everyone’s happy to agree?  Not always, but at least we we have a much more constrained series of problems to solve, rather than trying to tackle one big problem.  Sometimes you have to resort to tricks like voting to get to closure, but I’ve seen this be pretty rare.

Scenario Weights

The next step is completely mechanical.  Using a formula called “rank order centroids” ROC, we compute a weight for each scenario.  I came across ROC in an MSDN Magazine Test Run article called “Competitive Analysis Using MAGIQ”, but it originated with Barron and Barrett’s 1996 paper “Decision Quality Using Ranked Attribute Weights”.  Anyhow, ROC computes a weight for a given ranking like this:

image

Where N is the number of options and k is the rank.  So above, flexibility was ranked 2 out of 3 options:

imageIf you’re thinking “wow, this really favors the highly ranked options”, you’re right – that’s the point.  The idea is that this is, psychologically speaking, how people actually apply importance to rankings. There’s lots of interesting other work in this area.  Check out: Attribute Weighting Methods and Decision Quality in the Presence of Response Error: A Simulation Study.

LAAAM isn’t wed to rank order centroids, and sometimes we use linear, or even manual (danger, Will Robinson) weightings at one node in the utility tree when the stakeholders decide it’s more reflective of the actual relative importance of sub-qualities or scenarios. Rank order centroids has proven pretty effective though, so it’s the default.  Whichever approach you use to determine the weight for a given ranking, the last step is to compute the product of the rankings for each path through the utility tree, to come up with the scenario weights:

image

So the steady state scenario has a total weight of .140625 and the peak load scenario has a total weight of .046875.  Nothin’ to it.

Alternatives

Excellent, now we’ll know what we like when we see it. The next step is to figure out what the choices are – what are the alternative decisions, solutions, platforms, whatever, that we’re trying to choose among.  But first, note that all of the work we’ve done up to this point (i.e. creating a fully ranked quality tree with scenarios) is completely independent of the decision itself – this makes it much more persistent through time, even able to outlive a single instantiation of a system.

Generally, compared to defining quality, it’s pretty easy to identify our alternatives; usually the participants in the process have a pretty good idea about the options – .NET or Java or Ruby? On-premise or in the cloud? Hibernate or iBATIS? Entity Framework or LINQ to SQL or NHibernate? SharePoint or Documentum?  The common challenge at this point isn’t in identification of the alternatives, but in defining them to a sufficiently precise degree that we can assess them.  Sometimes this means we have to do more investigative work during the LAAAM process, but the scenarios guide us to where this investigation is required.

Assessment

Finally, we get to the third A in LAAAM – Assessment.  Somewhat counter-intuitively, this part is actually the easiest.  All we need to do is fill in this table:

Scenario

Alternative 1

Alternative 2

Alternative 3

Scenario 1      
Scenario 2   How well does alternative 2 achieve scenario 2?  
     

Each cell will contain a very simple assessment measure: usually we use a scale like poor, fair, adequate, good, excellent. This part of the process is done collaboratively with the subject matter experts for each of the alternatives, plus whatever additional supporting players are required.  Comparing alternatives is fine; in fact, it commonly makes the process go very quickly when you can establish a firm baseline (usually with the alternative that’s best understood) and ask questions like “is this solution better or worse?”

Now turn the ratings into numeric values (it doesn’t matter what they are, as long as they’re linearly distributed), multiply each by the weight, and you’re done (d’oh, I just realized the math in my TechEd deck is wrong):

Scenario

Weight

Alternative 1

Alternative 2

Alternative 3

Scenario 1 .140625 Poor (0) Fair (1) Excellent (4)
Scenario 2 .046875 Good (3) Adequate (2) Fair (1)
Total   .140625 .234375 .609375

(Footnote: if you read my earlier blog on LAAAM, you’ll notice that each cell actually contained three assessment “dimensions”: fit, development cost and operations cost. Repeated application of the method made it clear that cost is almost always smeared across the scenarios in a way that makes asking about it in each cell very difficult.  Effectively incorporating cost back into the method is an important subject for future work,  but it hasn’t inhibited adoption yet.)

Final Analysis

Unfortunately, we don’t get to simply declare that Alternative 3 is the winner, even though it had the highest score.  We do have an ordering of our alternatives, and we can get an idea about their relative adequacy.  To be thorough with the method, though, we need to ask a question about sensitivity: which scenarios are heavily weighted and exhibit significant differentiation among the alternatives?  These are the important ones, where we want to be sure about our assessment.  A second pass to validate sensitive scenarios is a good follow-up.

Also, in cases where a narrow margin exists between highly-rated alternatives (e.g. when they’re within 20%), one approach to refining the analysis is to add scenarios to further differentiate the alternatives. One way to do this is to decompose sensitive scenarios into sub-scenarios to probe related concerns. Sometimes it’s necessary to refine the definition of one of the alternatives and iterate on the analysis; such refinement may require, for example, a proof-of-concept implementation.

Tool

At TechEd I showed a demo of the Quality-Oriented Decision Assessment (QODA) tool, developed by Gary Chamberlain, the lead for Platform Architecture here at VistaPrint.  Although it’s pretty new, there’s already plenty of evidence that adoption of the method (which we’re now calling QODA inside VistaPrint) is substantially assisted with this tool support.  Previously the only “support” was an Excel spreadsheet, which was painful to work with. The tool isn’t currently generally available, but we’re hoping to make it so sometime soon.  Stay tuned.

Lessons learned

Posted in Uncategorized by jeromyc on April 9, 2009

I believe every developer and architect should keep track of their personal "lessons learned" for software/process/organization design and, broadly, how they practice as a software professional. Further, I expect that I’m like a lot of other architects in doing so somewhat irregularly, and therefore almost certainly retreading and relearning more than necessary. So what better place to capture my lessons learned than here?

I actually made a first pass at this, somewhat sideways, as part of a talk called "An Architecture Journey" I gave at the SATURN conference at the Software Engineering Institute in 2007. The abstract and slides are posted. The gist of the talk was to share lessons I had learned during my various jobs as a developer and architect, along with some observations on how my definition and practice of software architecture evolved over that time. I gave a slightly perturbed version of this talk at Google Canada in Waterloo, where my good friend Steve Woods is the site director. Below are some of the lessons I came up with, paraphrased, and in chronological order.  Sure, these don’t really stand on their own (and some may be downright cryptic), so the more interesting ones will likely become posts of their own.

Nortel

  • Big systems are tough to get right; thinking about “architecture stuff” up front is required for success (like avoiding measuring time-to-dialtone in minutes after a major system reengineering effort).

SEI

  • Architecture is the bearer of quality, but reasoning about architecture is reasoning about potential.

Quack/AOL

  • Enabling autonomy of organizations and systems is the way you scale.

Microsoft

  • You don’t need to choose between reasoning top-down and bottom-up: do spiral.
  • You’ll rarely (never?) know in advance if a decision is right, but make sure you know afterward.
  • Technology doesn’t matter (much); it’s about the people, the process, and the consistency of practice.

Fidelity

  • Figure out who defines goodness of your work and make them happy.
  • Don’t let “pragmatism” become a disguise for shortsightedness.
  • Communication and understanding trump every technical problem.
  • Plan for technology retirement, not just adoption.

VistaPrint

  • Don’t boil the frog with standards. (As the most cryptic, there will definitely be a post coming on this one.)
  • Be dumb. Ask smart questions.
  • Don’t undervalue slack.
  • Don’t forget your lesson about autonomy: define boundaries and common language when crossing them.

I’m currently particularly excited about the autonomy and slack lessons (not coincidentally the last two on the list), so expect posts on them before too long.

Technogility!?

Posted in Uncategorized by jeromyc on February 11, 2009

Maybe this should have been the first post on my new blog, but I didn’t want to show any (overt) signs of mental weakness too soon.

The totally bogus pseudo-word "technogility" was one of those that jumped into my head and blocked any rational thought about better alternatives (that reminds me: go read Pragmatic Thinking and Learning: Refactor Your Wetware, if you haven’t already). My only excuse is that somehow, at least to me, the word has an intuitive connection to how technology enables business – specifically the topic I think is most important to our industry (and the subtitle of this blog).

Technology organizations are always challenged to continue to deliver value to their business partners (i.e. the proxy for the end customer, trying to figure out what they want), particularly as both the technology organization and their partners scale. Sometimes they scale differently, and sometimes the scaling is transformational – that is, one of the organizations has to radically restructure in order to continue to deliver. Agility is therefore fundamental, and brittleness is the enemy. When we create cultures, organizations, processes, tools, frameworks, etc. that are rigidly aligned to a particular mode of delivery, we put the entirety of the business at risk.

Talk at TechEd 2009

Posted in Uncategorized by jeromyc on January 29, 2009

I’ve had a talk on the Lightweight Architecture Alternative Assessment Method accepted for the Architecture track at TechEd 2009.  Here’s the abstract:

Making good architectural decisions, fast!

This session will present and demonstrate the Lightweight Architecture Alternative Assessment Method (LAAAM), a fast and efficient way to make good architectural decisions – decisions that are aligned with the needs of your business. Making good architectural decisions early in the software development lifecycle helps reduce project risk and ensures that precious time and energy isn’t wasted delivering solutions that miss the mark. This is critical in our ever-accelerating software development world. LAAAM’s most unique characteristic is its focus on defining what quality means for a given system or product. We all want to create high-quality products, but LAAAM helps us go further in determining the aspects of quality that are important to the stakeholders for a given system. The "-ilities" are commonly used, coarsely, to talk about architectural quality; however, LAAAM won’t let us simply say "my system has to be scalable" or "I need flexibility". Instead, LAAAM forces us to be a. precise in the kinds of scalability and flexibility that are important; and b. prioritize these quality attributes so that appropriate tradeoff decisions can be made. LAAAM produces a set of artifacts that represent a rigorous, rational decision making process. These artifacts are extremely valuable in justifying and communicating architectural decisions, since it’s rarely possible (or even desirable) to have every stakeholder involved in every conversation. The LAAAM artifacts give us a way to express the reasoning that went into a given decision, with a foundation in the definition of quality for a system. LAAAM has been successfully applied to make decisions large and small at Microsoft, Fidelity and VistaPrint. LAAAM draws its roots from the Architecture Tradeoff Analysis Method developed at the Software Engineering Institute at Carnegie Mellon University.

Simple Architectures for Complex Enterprises

Posted in Uncategorized by jeromyc on January 26, 2009

Last Wednesday I attended a meeting of the New England chapter of the International Association of Software Architects, where Roger Sessions, CTO of ObjectWatch, gave a great talk based on his “Simple Architectures for Complex Enterprises” book. (I bought the book during the talk using the Amazon iPhone app … the last thing I needed was more enablement to buy stuff from Amazon.)

Below are my raw notes. Anything annotated “Thought” is something I was thinking, but didn’t say; generally I haven’t edited these, so some are addressed later in the talk (and some are just plain wrong). Further, I offer no guarantee that I haven’t misinterpreted something Roger said.

  • Address the “problem of complexity” – the “biggest problem in IT”
  • Federal IT
    • $71B in federal IT projects; $47B invested in “at risk” projects
      • Opportunity cost? (What government can’t do without the projects delivered)
        • Estimated at 5x; a company would demand 5x ROI on an IT project
    • 100% of defense-related projects on the “at risk” list
  • Map to private sector
    • $14T GDP; assume 5% in IT => $700B in IT (? something wrong)
    • Assume 33% at risk (no evidence) => $231B in at-risk projects
    • Thought: aren’t there already good studies on this stuff?
    • Now assume 5x in opportunity cost again => $1.115T
  • Back to federal IT
    • FEA rollout in 2006, specifically to address the problem if at-risk IT projects
    • Trend in IT projects considered at risk:
      • 30% iin 2007; 43% in 2008; 66% in 2009
    • Thought: why would anyone think that FEA would address this?
  • Epidemiology
    • In medicine, we assume some common factor when many exhibitions of common symptoms (illness, death)
    • How?
      • Look at all the characteristics of the individuals involved to identify commonality, and assume that it is the common factor or a contribution to it
  • What is the common factor for IT risk/failures?
    • Not geography, industry, methodology, technology
    • Roger’s belief: complexity: the only common factor across all failures
  • Thought: what are the long-term trends? Are we at a new peak of complexity relative to failures?
  • Questions
    • What’s complexity? all-in: organizational, data schema, code
    • My question: presume you consider also organizational complexity as well as systems complexity?
      • A: Mostly organizational, actually; can rebuild systems
    • Is communication the problem?
      • A: A sympton, not the cause.
    • Why not identified sooner? What happens to at-risk projects?
      • A: Some fail because they run out of money. Some because they don’t meet the needs of the business.
  • Can’t build architectural quality attributes (security, as an example) into a complex system. Much harder than doing so for a simple system.
  • Complexity
    • Glass’s Law
      • For every 25% increase in problem complexity, there is a 100% increase in solution complexity.
        • (Not stated by Glass, but implied by his “Facts and Fallacies of Software Engineering”)
    • If you have a system with 10 functions, how many functions can you add before doubling the complexity? 2 or 3.
    • If you double the number of functions in a system, you increase complexity by 8x.
  • Implications
    • Methodologies that work well for small projects don’t work well for large projects.
      • e.g. agile
  • Thought: isn’t the solution to make our “systems” smaller. This is what SOA was about – autonomy.
  • Methodologies/frameworks don’t explicitly address complexity, so won’t succeed.
  • Questions
    • Isn’t agile about breaking large projects/systems into small ones?
      • A: Agile doesn’t tell you how to do that, but if you can conceive small systems/projects, agile be effective.
    • Examples of projects/companies that have succeeded?
      • A: A current client: a state government replacing their IT system. Using “this” methodology to decompose.
        • Who could respond to a 400 page RFP? Only 3 or 4 firms. Predetermines the solution.
    • Problem is lack of cohesion and loss of knowledge? Doesn’t this increase complexity?
      • A: Problem is when we lose knowledge of complex systems, not the root.
      • A: If complexity is managed as a primary concern, the others won’t manifest.
    • Doesn’t this imply that we just spend more time on up-front design?
      • A: Yes, but we need to design explicitly to manage complexity.
    • Survivability of requirements/stability of organizations?
      • A: Also symptoms. Need to address holistically.
  • Model for complexity: a function of the number of states a system can be in
    • A penny has two; two have four (or three?); a six-sided die has 6
    • Thought: this is unmeasurable for a software system of any relevance. Does that matter?
    • Generally: (number of states)^(number of objects)
    • Justifies Glass’s law: if we add functionality, we add variables linearly. If we add variables linearly, we add complexity exponentialy.
  • Questions
    • What’s a database row in this model?
      • A: <shrug/> Consider database columns as variables.
  • If we split up “systems”, we manage:
    • One system of 12 6-state objects => >2B states
    • 2 systems of 6 6-state objects each => < 2 x 45K states
  • Glass’s Law
    • 100 -> 1164 functions => 2048x increase in complexity
    • 2 x 582 functions => 2 x 250 complexity measure
    • Keep going …
  • (Mentions SOA in passing; presumes independence of systems; clearly motivated by previous work on Software Fortresses)
    • Thought: Estimate independence with autonomy?
  • As number of subsystems increases, you reach a complexity minimum, then complexity begins increasing
    • => need to know when to stop decomposing
    • Workflow and management complexity increases – need to address coordination among subsystems
  • How to partition? Different complexity curves for each:IMG_0062.JPG
  • Goal of architects should be to find the simplest possible solution
    • Thought: slight misuse of Einstein’s “but no simpler” quote; impossible to be “too simple” in this model, but possible to over-decompose such that simplicity is lost
  • Ha: “decibel-driven decision making” – whoever yells loudest
  • How do we know which partitioning is simplest?
    • Equivalence relations
      • Five characteristics
        • Binary function: F(x, y)
        • Boolean: F(x, y) in {true, false}
        • Reflexive: F(x, x) = true
        • Symmetric: F(x, y) = F(y, x)
        • Transitive: F(x, y) = true ^ F(y, z) = true => F(x, z) = true
      • Can drive a unique partition with an equivalence function; e.g. same-category-as
      • But can you convince yourself that there’s an equivalence function that is correlated to simplicity?
      • Example: synergy
        • Two functions are synergistic if, from the business perspective, neither is useful without the other
        • Perhaps the optimal equivalence relation
        • Issue is separating business function and IT realization
          • Need collaboration between IT and business to get it right
  • Process
    • SIP: Simple Iterative Partitions
    • Value proposition: dramatically improves return on IT investment by addressing complexity.
    • Thought: a motivation for SOA. Ok.
  • Recap
    • IT complexity is a major problem
    • Cannot be solved by existing methodologies
      • Can’t judge architectures today without building them
        • Thought: scenario-based assessments (like LAAAM and ATAM) can help
    • To solve complexity, must understand it
      • SOA by itself isn’t the answer, because it pre-supposes a decomposition (?)
    • Can only be solved by “best possible” partitioning
    • Best partitions are driven by equivalence relations
    • SIP is based on best possible partitioning through equivalence relations
  • Thought: partitions today are temporal – groups of things built at the same time
  • My question: is there stability in the synergy equivalence relation?
    • Argued not, Roger suggested that its the implementations that change, not the partitions
    • Same argument as documented in Enterprise Architecture as Strategy, Motion (Microsoft’s business architecture methodology – unsure of its current state)
    • Neeraj Sangal of Lattix: multi-allocation based on equivalence relation?
      • IT is a problem, because they want to select a single solution (implementation) for multiple business problems. e.g. scheduling, but scheduling of different kinds of things
  • Partition: “Autonomous Business Capability”
  • Thought: (on my way home) What about on-the-same-P&L-as as an equivalence function? Substantially stable

A domain-specific language for … interval workouts

Posted in cycling, dsl by jeromyc on January 26, 2009

This is something of a cross-over personal/technology post, hopefully interesting nonetheless. Perhaps a something-for-everyone post … or a nothing-for-anyone post.

I’m a fairly serious amateur cyclist. I raced for one season when I was in college (1992!), but I totally burned out and quit riding, almost entirely, until the 2007 season. I was helped back into riding through what I call “motivation through financial commitment” – I bought a $5000 (at the time) Cervelo Soloist Carbon. But my biggest motivation comes from the group I ride with – Crack o’ Dawn – that meets every weekday at 5:45 for a 25-30 mile ride and does weekend rides of 50-100 miles; the group convenes within 2 miles of my house in Newton, MA. I rode 5000ish miles in 2007 and 6500ish in 2008 (the CoD record this year was set by Michael Mercier, at 14105.69 miles, and many CoDers break 10K every year).

Now the bad news. It’s winter here in the Boston area. It’s a snowy winter. Unlike some of my CoD mates, I’m not all that fond of the cold or rolling the dice with icy roads (I had a crash late in the 2007 season, broken collarbone). I’m happy to ride if it’s above freezing, even if wet, and down to 20ish if it’s dry. But lately this has left precious little opportunity to ride outside. I’m coping by riding spin classes at Spynergy, where follow CoDer and good friend Andy Schiller is an awesome instructor. I also have a CycleOps Pro 300PT indoor trainer, which I ride 4-6 times a week. The coolest thing about about the trainer is that it has an integrated power meter, so you can track your wattage while you ride. There’s a lot of science behind training based on power; the Peaks Coaching Group is pretty well known, and their principal, Hunter Allen, is co-author of book on the topic: Training and Racing with a Power Meter (Allen and Coggan). This provides an excellent opportunity for me to integrate my bike self and my geek self into one well-trained, intellectually satisfied unit.

Here’s how: unlike some indoor training systems, like the CompuTrainer, which many of my cohorts swear by, the CycleOps doesn’t provide any facilities for guiding a workout. It just shows you your heart rate, cadence and power output in watts. I filled this gap with a small Windows Forms application (hopefully soon to be ported to WPF) that has a very simple UI (yes, some would say go so far as “lame”, and they’d be right, but that’s not the point):

picture-2This says: ride at 200 watts for 4 minutes and 44 seconds more; you’ve been riding for 19 seconds and have 54:44 to go. The pull down lets you select different workouts. You can skip over an interval (wimp) and pause/resume (say, to change the TV channel). Easy, to the point. But, there’s some underlying extra fun. When I first wrote this, I basically hardcoded workouts in a config file. They looked like this:

        180     00:10:00
        220     00:10:00
        250     00:05:00
        280     00:02:00
        150     00:03:00
        220     00:10:00
        250     00:05:00
        280     00:02:00
        150     00:03:00

Again, simple and to the point. But repetitive and not very flexible. The opportunity expands…

I’ve been working with domain-specific languages in various forms for a long time, including with Microsoft’s DSL Tools, Boo and Oslo. As I’ve mentioned before, my good friend Harry Pierson is a Product Manager with the DLR group at Microsoft, so I’ve been playing with IronRuby and IronPython for a while (Harry gave an excellent talk at a brown bag session at VistaPrint on the DLR and IronPython a couple of months ago). I had read a few articles about implementing DSLs in Ruby, so … <light bulb/> Construct a Ruby DSL to express interval workouts and dispatch out from the WinForms app to the IronRuby runtime to fetch workout definitions.

Here’s a simple example of a definition in the resulting DSL:
        workout "W - intervals" do
        interval.target(0).time('00:01:00')
        interval.target(180).time('00:05:00')
        repeat 4 do
              interval.target(220).time('00:10:00')
              interval.target(280).time('00:02:00')
              interval.target(150).time('00:03:00')
        end
end

This defines a workout called “W – intervals”, made up of a 1 minute 0-power “click start, get on the bike and clipped in” step, 5 minutes at 180 watts, then 4 repeats of 10 minutes at 220, 2 at 280 (what I believe to be my functional threshold power) and 3 at 150. Aside: I’m not proposing this is a good workout – I’m studying the Allen and Coggan book to figure out how to actually use power properly in training. Again, not the point.

Simple enough, and the “repeat” construct achieves a lot of the intended benefit of reducing repetition. But there’s so much more potential. As I was reading the training book today, I came up with this one:

ftp = 280
work = 0.9*ftp
warmup = 0.68*ftp
rest = warmup

workout "L4 - Ride A" do
        interval.target(0).time('00:01:00')
        interval.target(warmup).time('00:05:00')
        interval.target(ftp).time('00:02:30')

        repeat 2 do
                interval.target(work).time('00:13:00')
                interval.target(rest).time('00:02:00')
        end

        repeat 6 do
                interval.target(work).time('00:01:00').message("high cadence")
                interval.target(rest).time('00:02:00')
        end

        interval.target(rest).time('00:05:00')
end

This is an adaptation (shortening to 1 hour) of a “typical level 4 threshold” workout from Coggan and Allen, on page 83.

The power of using an embedded DSL is clear: there are very few domain-specific implementation constructs required, making the DSL specification extremely short. All that was required was the definition of the workout and interval classes and the simple “repeat” construct, and that’s really only a personal preference (6.times would have done just as well).

The core of the Ruby DSL looks like this:

$intervals = []

class RInterval
        def initialize()
                @interval = Interval.new
                $intervals.push self
        end

        def target(t)
                @interval.Target = t
                self
        end

        // message and time look the same as target

        def Interval
                @interval
        end
end

def repeat(count, &block)
        count.times do
                yield
        end
end

def workout(name, &block)
        yield

        intervals = System::Collections::ArrayList.new

        $intervals.each {|interval| intervals.Add interval.Interval}

        $intervals.clear

        workouts.Add name, intervals
end

def interval
        RInterval.new
end

On the .NET side, there’s a simple Interval class:

public class Interval
{
        public TimeSpan Time { get; set; }
        public int Target { get; set; }
        public string Message { get; set; }
}

At runtime, the .NET application instantiates the IronRuby engine, sets up a Dictionary<string, ArrayList> in the engine’s ScriptScope (no interoperable generics right now :-() and loads and executes the DSL spec and the DSL instance:

            _scope = _engine.CreateScope();
            _scope.SetVariable("workouts", new Dictionary<string, ArrayList>());

            // ...

            _engine.Execute(code, _scope); // code is the DSL definition + the DSL instance
            object ret = _scope.GetVariable(returnVar);

And that’s it.  The UI code is dead simple, and again, not the point.  I’ve obviously got some more to learn about doing DSLs in Ruby, but this has proven to me the utility of the approach.  And it helps with the winter riding blues.

Update: Here’s the code from Jim’s comment, which I couldn’t make show up in his comment:

;; User setting
(setq *ftp_limit 280)

;; Workout routine
(progn
  (wait 1 "click start, get on the bike and clipped in")
  (warmup 5)
  (ftp 2.5)
  (repeat 2
                (work 13)
                (relax 2))
  (repeat 6
                (work 1 "high cadence")
                (relax 2))
  (relax 5))

;; Here’s the DSL implementation

(defmacro interval (pace duration &optional message)
  `(progn
                 (format t "~&~6a (~2$) for ~5 minutes" (car ,pace) (cadr ,pace) ,duration)
                 (and ,message (format t " *** ~a ***" ,message))))

(defmacro proportional-interval (name proportion duration &optional message)
  `(interval (list ',name (* ,proportion *ftp_limit*)) ,duration ,message))

(defmacro deffactor (name factor)
  `(defmacro ,name (duration &rest args)
                 `(proportional-interval ,',name ,',factor ,duration ,@args)))

(defmacro repeat (times &body body)
    (let ((x (gensym)))
      `(dotimes (,x ,times)
         ,@body)))

;; You can argue about whether the following is in the DSL designer
;; land, or in user land. Since it seems effort ratios would be
;; uniform across many workouts, I’ve put it here (though users could
;; override for their own workout easily enough.)

(deffactor ftp           1)
(deffactor work          0.9)
(deffactor relax         0.68 )
(deffactor warmup        0.68 )
(deffactor wait          0)
(defvar    *ftp_limit* 280)

Update: I couldn’t leave this alone, after Jim’s elegance put me to shame. Here’s the same workout I had before in my new DSL:

workout "L4 - Ride A" do
	setup 1
	warmup 5
	work 2

	2.times do
		work 13
		rest 2
	end

	6.times do
		work 1, "high cadence"
		rest 2
	end

	rest 5
end

Not quite ready to share the DSL definition yet – it’s still a bit ugly, but improving. I’m also trying to smooth out the IronRuby integration so that I can test the DSL independent of a .NET runtime.

Update: Here’s the new DSL definition.  Much much cleaner.  Thanks, Jim, for driving me to get this right.


$ftp = 280
$effort = {"setup" => 0, "ftp" => $ftp, "easy" => 0.8*$ftp, "work" => 0.9*$ftp, "warmup" => 0.68*$ftp, "rest" => 0.68*$ftp}

class WorkoutDSL
    def method_missing(sym, *args)
          interval = {:target => $effort["#{sym}"], :time => args[0], :message => args[1]}
          @workouts[@name].push interval
    end

    def workout(name, &block)
          @name = name
          @workouts[@name] = []

        yield
    end

    def load(filename)
          @workouts = {}
           instance_eval(File.read(filename), filename)
        @workouts
  end
end

WorkoutDSL.new.load("intervals.dsl")

Welcome Jim!

Posted in Uncategorized by jeromyc on January 13, 2009

My VistaPrint colleague, Jim Sokoloff, has started his blog: IT @ A Profitable Dot Com with an excellent first post about VistaPrint’s recent data center move (roughly 8 tons of gear). Jim runs our customer-facing IT organization, but was down there in the trenches with his team moving servers over New Year’s. An awesome reflection of how committed VistaPrint employees are to the success of the business.  

Follow

Get every new post delivered to your Inbox.