Our talented team of engineers here at Zignal builds and maintains the split-second processing engine that ingests and enriches millions of news and social media stories each day from multiple sources. We index and aggregate tweets, stories, blogs and news video clips so that front-end interfaces can display beautiful, dynamic visualizations that are customized to the needs of each client.

Rather than having to manually sift through multiple websites, emails and apps, our customers are empowered to act on news as it happens based on accurate, up to the split-second information.

Many of our customers use Zignal Command Center to showcase our visualizations: an engaging multi-screen wall display that updates in realtime as tweets and stories are created.

As it turns out, engineering teams face exactly the same challenges that we are solving for our customers.

We are inundated with information and it is a challenge to see what’s really important at a glance. Huge amounts of data in logs, unstructured conversations in chatrooms, alerts in emails, comments in Jira. The list goes on. We need to stay on track during sprints, write quality code and tests that get continuously integrated, manage environments, and react quickly to problems. The flood of unstructured, hidden, unindexed information can be seriously overwhelming.  So what do we use to overcome this problem?

Zignal Command Center, of course.

(yes, we use one of our own products here at Zignal Labs!)

command-center

The Zignal Command Center

Feedback and transparency are key principles for effective lean and agile teams.

All the way back to my first experience of agile and XP the ‘visual workspace’ was key to giving us rapid feedback on progress as well as obstacles blocking our path. Great agile teams devise all sorts of creative methods to shine the spotlight on pertinent information that would otherwise be lost inside a terminal window, chat, webserver or email. For example, actual traffic lights to flag failing build status, wall mounted burndown charts or large-format kanban boards with colored paper showing who’s working on what.

The beauty of these ‘information radiators’ (a term coined by Alistair Cockburn, Crystal Clear, 2004) is their simplicity: large, easy to understand, easy to keep up to date and conspicuous displays that the entire team (and anyone visiting the space) can see and interact with at standup and any other time throughout the day.It’s difficult to ignore blocks and problems when lights are flashing or a yellow card is in the wrong column right in front of you.

Since we’ve created an incredible “information radiator” in the form of the Zignal Command Center we wanted to make use of the same technology and put it to work in our team room. Our expert installers spent the weekend drilling and wiring up six huge screens and we got together and thought about what we should display on them. An important step since the information we display needs to convey the right level of feedback and cues to the team. We came up with the following categories:

  • Continuous integration status for all our projects — immediately react to red builds and fix them.
  • Sprint board and charts — giving everyone visibility of progress towards our sprint goals.
  • Code quality metrics and test coverage — transparency around our work and get continuous feedback on quality improvements we make.
  • Github activity — see open pull requests and avoid delays by reviewing and merging quickly.
  • Our application (Zignal Enterprise) — the best feedback of all is seeing the system we’re building streaming story updates in realtime.
  • Operational data — metrics about our infrastructure’s health and rate of data flowing through the pipeline helps us identify and address problems before our clients are affected by them.
  • Webservice status for third party feeds we rely on — notice as soon as external systems stop responding.
  • Visualizations of logs showing trends and errors across our many components — early warning of errors and transparency of incorrect or inappropriate log levels.

Our wall of screens has made a huge difference to how we work together as a team.

engineers-command-center

Problems get our attention more quickly and we’re proactive in fixing them. Breaking builds are addressed and blocked sprint tasks get resolved. Which leads to greater transparency and communication across the team and beyond as well as faster delivery of tested, integrated features. We spend less time chasing up problems in chat rooms and more time implementing new features that add business value.

Time is money. So is speed to resolution.

We will keep iterating on our Engineering team Command Center (pictured above), looking for ways to simplify what we present and get even better high feedback and transparency.

Watch this space for future posts that drill into our tech stack and other innovations we’d like to share.

Other references: