Improve Linux performance

GovernmentSecurity.org > System Security > Linux & Unix Systems

ComSec

Apr 15 2003, 04:04 AM

Level: Intermediate

Cameron Laird (claird@phaseit.net)
Vice president, Phaseit, Inc.
April 10, 2003
Cameron Laird presents a collection of useful examples that are apt models for the sorts of performance problems likely to arise in your own application development.

Performance breakthroughs seem to come in two varieties: easy and hard. That's no platitude; the boundary between the two is surprisingly clear.

When you hear about some -- the easy ones -- you clap your hands and say, "wow" or "of course" or "slick." Although in some cases it has taken considerable genius to realize their first application, they're easy to understand.

The other kind involve careful measurement, specific knowledge, and a fair amount of tuning. These are often both frustratingly and rewardingly contingent on "local conditions" such as hardware specifics.

Good programmers can operate in either the "hard" or "easy" mode. As performance is an important topic for Linux programmers, I offer a paired collection of four hard and easy tales from real (programming) life.

It always starts with requirements
Death and taxes have reputations for inevitability. Of nearly equal importance for developers is careful consideration of requirements; that's one subject that comes up in every kind of programming.

While performance is certainly important, the best way to handle this requirement is not always obvious. Time and again, I've experienced a software challenge that followed roughly this pattern: a program is in use. Its functionality is correct. A user stops in, though, to report that it's "too slow" and needs acceleration. Someone on the team quickly hacks in a "monitor" that slows performance a bit but keeps the user informed about how much time remains for a long-running computation. Satisfaction settles in.

I first observed this phenomenon late in the 1960s and have seen it every decade since. My guess is that in some sense it's been going on since the beginning of computing. The point is not that end users are gullible or can be distracted with shiny decorations. They simply have larger goals than our narrow, specialists' notions of "performance." In many cases, when they say a calculation needs to be faster, what they really mean is that they need to know more quickly and reliably how long the calculation will take. With accurate information on that score, end users can be happy scheduling other tasks around the computer's delay.

This suggests a "warmup exercise" for everyone involved with performance. First, read or reread your favorite references on requirements management; the Resources section below provides a couple of useful starting points.

Second, keep in your toolkit a few little "progress monitors," "progress bars," or "stopwatches" that you can quickly drop into applications when you find you've left your users too much in the dark. These don't have to be expensive at all, as the two following examples illustrate.

Two countdown examples
Performing a long computation while simultaneously keeping a user informed of its progress models an interesting problem in application design. It's an instance of concurrency, or "multitasking." Many developers believe concurrency demands elaborate mechanisms, and difficult threading code in particular, for a proper solution.

It's not true. The Resources section below lists an earlier developerWorks column in which I sketch the range of technologies involved in concurrency. What's more, simple concurrent programming has been available on essentially all UNIX hosts since at least 1988, which is when ksh standardized its "co-processes." If you save the ksh source in Listing 1, below, as ex1.ksh and then run it, you'll see a "countdown" -- "ten, nine, eight, etc." -- display, then the result of the ksh subprocess: "All done." In practice, you might use something like this for a long-running chemical calculation or database retrieval: launch the operation as a subprocess, but keep users informed about what's going on or how much time is left before completion. It only takes a couple of lines of source code to update the display....

full details and codes

http://www-106.ibm.com/developerworks/linu...w914FasterLinux

This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.


Help - Search - Member List - Calendar Full Version: Improve Linux performance GovernmentSecurity.org > System Security > Linux & Unix Systems ComSec Apr 15 2003, 04:04 AM Level: Intermediate Cameron Laird (claird@phaseit.net) Vice president, Phaseit, Inc. April 10, 2003 Cameron Laird presents a collection of useful examples that are apt models for the sorts of performance problems likely to arise in your own application development. Performance breakthroughs seem to come in two varieties: easy and hard. That's no platitude; the boundary between the two is surprisingly clear. When you hear about some -- the easy ones -- you clap your hands and say, "wow" or "of course" or "slick." Although in some cases it has taken considerable genius to realize their first application, they're easy to understand. The other kind involve careful measurement, specific knowledge, and a fair amount of tuning. These are often both frustratingly and rewardingly contingent on "local conditions" such as hardware specifics. Good programmers can operate in either the "hard" or "easy" mode. As performance is an important topic for Linux programmers, I offer a paired collection of four hard and easy tales from real (programming) life. It always starts with requirements Death and taxes have reputations for inevitability. Of nearly equal importance for developers is careful consideration of requirements; that's one subject that comes up in every kind of programming. While performance is certainly important, the best way to handle this requirement is not always obvious. Time and again, I've experienced a software challenge that followed roughly this pattern: a program is in use. Its functionality is correct. A user stops in, though, to report that it's "too slow" and needs acceleration. Someone on the team quickly hacks in a "monitor" that slows performance a bit but keeps the user informed about how much time remains for a long-running computation. Satisfaction settles in. I first observed this phenomenon late in the 1960s and have seen it every decade since. My guess is that in some sense it's been going on since the beginning of computing. The point is not that end users are gullible or can be distracted with shiny decorations. They simply have larger goals than our narrow, specialists' notions of "performance." In many cases, when they say a calculation needs to be faster, what they really mean is that they need to know more quickly and reliably how long the calculation will take. With accurate information on that score, end users can be happy scheduling other tasks around the computer's delay. This suggests a "warmup exercise" for everyone involved with performance. First, read or reread your favorite references on requirements management; the Resources section below provides a couple of useful starting points. Second, keep in your toolkit a few little "progress monitors," "progress bars," or "stopwatches" that you can quickly drop into applications when you find you've left your users too much in the dark. These don't have to be expensive at all, as the two following examples illustrate. Two countdown examples Performing a long computation while simultaneously keeping a user informed of its progress models an interesting problem in application design. It's an instance of concurrency, or "multitasking." Many developers believe concurrency demands elaborate mechanisms, and difficult threading code in particular, for a proper solution. It's not true. The Resources section below lists an earlier developerWorks column in which I sketch the range of technologies involved in concurrency. What's more, simple concurrent programming has been available on essentially all UNIX hosts since at least 1988, which is when ksh standardized its "co-processes." If you save the ksh source in Listing 1, below, as ex1.ksh and then run it, you'll see a "countdown" -- "ten, nine, eight, etc." -- display, then the result of the ksh subprocess: "All done." In practice, you might use something like this for a long-running chemical calculation or database retrieval: launch the operation as a subprocess, but keep users informed about what's going on or how much time is left before completion. It only takes a couple of lines of source code to update the display.... full details and codes http://www-106.ibm.com/developerworks/linu...w914FasterLinux This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.