Category: Community Blog

Reynold Miguel is a QA engineering leader with a strong focus on bringing automation to organizations. His deep technical knowledge and understanding of technology enables him to discuss more complex concepts, like using cron to setup automation, while maintaining a focus on the value and broader organizational importance of the technique. 

Using Unix Cron to Run Automated Tests, by Reynold Miguel 

The software utility cron is a time-based job scheduler in Unix-like operating systems. People use a cron to schedule jobs (commands or shell scripts) to run periodically at fixed times, dates, or intervals.  Using cron jobs are a great way to run scheduled tests. Running tests in Jenkins provides more functionality, but crons are a quick shortcut in case you want to run your initial set of tests right away on a schedule. The benefit of this approach is the ability to schedule tests in a matter of minutes, without need for a whole lot of setup. You can also utilize crons for any external processes that you need for your tests, e.g. data loading, sending emails, etc. At our company we initially used crons for running tests, but since transitioning to Jenkins we use crons primarily for running various processes like data loading and sending test emails in our QA environment.

Needed Tools:

• Unix/Linux/Mac
• Automated Tests that are runnable from command line, or any runnable process
• Basic knowledge of Unix and vi editor

Step 1: Have a Job You’d Like to Schedule

Here’s our job that we use for running our tests, we use a customized framework built on top of Selenium Web driver and Ruby.  This test can be run from the Unix command line. $ bin/autobot –connector=firefox –env=qa 2>&1 | tee data.log What do these commands mean?

• This line is how we run our customized set of selenium tests, on firefox and in our qa environment: bin/autobot –connector=firefox –env=qa
• And these are Unix commands that allow us to append the test results to a log file so we can look at it later: 2>&1 | tee data.log

Step 2: Setting up a Cron Job

Type the following in the terminal: crontab -e
If you do not have a crontab file, one will be created for you and opened in the vi editor. Setting up a cron job consists of creating, or adding to, your own crontab file.  Each job needs a line containing six entries, each of which is separated by spaces or tabs. The first five entries are integer patterns specifying the following:

• Minute (0-59),
• Hour (0-23),
• Day of the month (1-31),
• Month of the year (1-12),
• Day of the week (0-6 with 0=Sunday).

Each of these patterns may be either an asterisk (meaning all legal values) or a list of elements separated by commas. An element is either a number or two numbers separated by a minus sign to indicate an inclusive range.

A minute specification of 0,30 would indicate the job is to be run on the hour and half hour. Likewise, a day of the month entry of 1,15 would initiate execution on the first and fifteenth of the month. The sixth field of a line in a crontab file is a string that is executed by the shell at the specified times. A percent character in this field (unless escaped by) is translated to a new-line character. Cron jobs are being executed from a system directory. Make sure you include and explicit path to your script. In our case, let’s say we want to run the tests every 30 min. You can also use a “#” to comment your scheduled jobs.

#This will run our tests every 30 min
0,30 * * * * bin/autobot –connector=firefox –env=qa 2>&1 | tee data.log

Step 3: Save the Cron and Let it Run

And once you save it you’re done!  You’ll receive a message that should say something like: crontab: installing new crontab

Crontab Commands:

• crontab -e   The -e option spawns an editor containing the user’s current crontab
• crontab -r   The -r option removes a user’s crontab from the crontab directory
• crontab -l   To see the list of jobs use the -l option

Remember, when you save the file, it automatically updates the crontab when exiting the editor.

Jay is a technology executive who runs application development and operations for a large team at Edgecast, a widely used and fast growing content delivery network based in LA. With broad responsibilities ranging from strategic decision making to hands-on management, Jay provides technical leadership and direction for his team. Jay shares his perspective on how and why he implemented Feature Branching at Edgecast.

Is Feature Branching Evil? by Jay Dorman

Remember when deploying a web application meant mapping a network drive and xcopying (or even dragging and dropping!) files between your development system and a production server?  And then dealing with all the files that didn’t copy because your webserver had them locked?  Come on…you can admit it…you’ve been there.  While my teams have moved FAR beyond those days, we’re still struggling with some aspects of deploying new features. We generally operate in two week sprints, which are followed by about two weeks of testing, break-fix and deployments before a release hits production.  That means there’s a nearly four week window between the time a feature bubbles to the top of the backlog and it sees the light of day…a pretty painful delay in anybody’s book.  It also means that some dozens of features are bundled together for release.  In the case where a feature needs to be delayed because of external dependencies, or because of a last minute bug, it forces us to roll back the entire release.  Yes, we can exclude the particular feature, but with a mature base of shared objects and base functionality, the altered deployment really needs to roll through QA and Staging again before it’s safe to deploy.  Even with a relatively high degree of automation (we use a combination of Jenkins, Octopus Deploy and RedGate’s tools), rollbacks and re-deployments can be pretty painful. We’ve decided to move towards feature branching.  If you ask The Google, it won’t take long before you’ll read/hear the phrase “…feature branching is evil.”  You’ll also read articles that describe feature branching as being in direct opposition to continuous integration.  I just don’t see it.  Lots of talking heads will tell you that in order to do continuous integration, you need to build logic gates around each feature so that development code can be disabled until it’s ready, then flipped on once it’s passed through all the quality gates, after which you’d remove those logic gates.  (Yes, I know…that’s a pretty simplistic explanation, but I think accurate for the purposes of this discussion).  This makes me nervous for a couple reasons.  First, I don’t like the idea of all these logic gates sprinkled throughout the code.  With a team of say, 10 developers writing features with broad scope, it seems like a lot of ‘meta code’ to write, test, remove and then re-test.  Second, what if you need to make changes to a broadly used core business object?  Wrapping conditional gates around the datatype of a property that’s accessed by dozens of methods and pages seems like a lot of risk. So, we’re moving towards a deployment methodology that we feel incorporates the best of feature branching and continuous integration.  Here’s some of the key points of the direction we’re heading:

• We’ve migrated from svn to git, as it’s a much better fit for the way we want to branch and merge changes.
• Each feature gets its own branch off of the main develop branch.  Incidentally, we’re naming each feature branch after the Jira ticket that tracks the effort.  Seems intuitive to us, and helps with automation.
• Throughout the development cycle, the developers merge up from the main branch into the feature branch.  This insures that features play nice with other changes that have already made their way into production.
• On a daily basis, we use Jenkins to compare each feature branch to main in order to make sure that developers are merging up every day.  With the linear nature of git, it’s easy enough to determine if a given branch is behind.
• Once a feature is code complete, we build to QA from the feature branch, before we merge to main.  That allows us to deploy urgent hotfixes and such, without inadvertently deploying untested features.  Also, when Jenkins initiates a QA build, it also spins up an OpenStack VM environment to test the feature.  That way we can have multiple changes going through QA independently and simultaneously.
• Once QA certifies a feature as ready for production, we deploy the same package to integration, staging and eventually production.  When that’s complete, we finally merge into main.
• Hard not to emphasize this again…with each deployment, we compare the build to what’s currently in main, merging up to the feature branch if necessary.

It’s taken a lot of automation, and loads of adjustment and practice, but we’re getting there.  Each feature goes through CI within its own branch, and if there’s problems, that’s where we catch them.  The developer(s) working in that branch have to address any integration issues before moving their feature towards production.  Every night we run automated unit tests and functional tests on each feature branch, and in a performance environment, we run performance, load and analytics tests.  Now features go out the door as soon as they’re ready.  Product feels empowered to prioritize and actually impact deployment timelines.  QA is happy because they don’t necessarily have to run full regression with each change (depending on the scope).  Execs don’t wonder “Why does it take so long to develop features.”  Maybe best of all, customers feel like we’re listening and quickly acting on their needs. Is it perfect?  I don’t know.  We’ll figure that out over time.  But we’re feature branching.  AND we’re doing CI.  And so far, we don’t feel evil.

Ensuring a culture of innovation as companies grow has become an increasingly important focus for executives. Jim Connolly, SVP of Technology at Zynx Health, provides insight into the unique challenge of building a company structure that breeds creativity and ideas.

“The Innovation Sandwich”, by Jim Connolly

In many organizations there are processes in place to corral and organize ideation and follow-through activities.  These processes are often centralized using internal R&D and/or Product Management teams.  Some companies may adhere to activities such as those taught in the Pragmatic Marketing Framework.  These outside-in activities are useful because they take a market driven, ROI based approach for product planning.

There are downsides of such centralized and rigid processes that could actually diminish the latent innovation potential of company employees. For example, game changing opportunities may be lost because ideation requiring creativity, inspiration, and suspension of belief are slowed down, gated and filtered too early by the very processes put into place to encourage and manage them.

Finding ways to work alongside rigid annual and product plans is what Hearst CTO Phil Wiser referred to when he said, “…Of all our challenges, that’s been our single biggest challenge–to operate our businesses in a way that really supports this real-time innovation.”

How can both a market/strategy driven (top-down) and a real-time employee (bottom-up) approach to innovation co-exist?  One approach could be to add a second process that gives responsibility for ideation follow-up activities to the people that come up with product ideas or new ways to leverage technology.

Here is how to get started.  First, create a time boxed process for idea exploration and follow-through using contemporary methodologies such as Design Thinking or Lean Startup.

Second, form partnerships which cross multiple functional boundaries in order to build additional domain and customer knowledge. These partnerships should include sales teams, account managers, product teams and other company domain experts and thought leaders. Build a culture of communication that makes it safe for people with diverse viewpoints to interact without filters and the loss of information that can occur when functional boundaries exist.

Next, provide access to readily available resources such as contemporary prototyping tools, APIs, cheap computing power, and inexpensive engineering support.  Establish a service center to provide mentorship in such areas as prototyping, design thinking and lean startup techniques.

Finally, create a culture of learning and give technology employees a mandate to stay current with technical skills, processes, and emerging standards and to find innovative uses of technology that could contribute to business growth and competitive advantage.

By following these steps, in combination with market driven approaches, businesses can achieve real-time innovation.  The key to success is to provide the organizational and technical support necessary to allow passionate and creative people to directly follow through and vet their ideas and solutions.

Jim Connolly is the SVP of Technology for Zynx Health.  For more management and leadership ideas, please visit http://www.theagilelife.org.